2024-04-26T20:43:57.495 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T20:43:57.495 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675182 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/asok_dump_tree} --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675182/orig.config.yaml 2024-04-26T20:43:57.541 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T20:43:57.630 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675182 branch: wip-rishabh-testing-20240426.111959 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/asok_dump_tree} email: null first_in_suite: false job_id: '7675182' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi028.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIK+zg/JWra1uD612P625cNOww2OHY02pxIF8/9BwvxAKT3XK7tUAMyzrzCJUhku57hQKvrn2sqmYng7u9kbJ64= smithi103.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEEKkQDpg0Qr3CZiGw53Rqbpec5xVz6/xjXT8sCdl24BbF/tT3cu5+b/frYYHVCRg/0ymyHAxlTesvOs1Y95tMc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_dump_tree teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T20:43:57.631 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T20:43:57.631 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T20:43:57.631 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T20:43:57.633 INFO:teuthology.task.internal:Checking packages... 2024-04-26T20:43:57.658 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T20:43:57.658 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T20:43:57.659 INFO:teuthology.packaging:ref: None 2024-04-26T20:43:57.659 INFO:teuthology.packaging:tag: None 2024-04-26T20:43:57.659 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T20:43:57.659 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:43:57.659 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T20:43:57.794 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T20:43:57.796 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T20:43:57.803 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T20:43:57.803 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T20:43:57.852 INFO:teuthology.task.internal:Saving configuration 2024-04-26T20:43:57.869 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T20:43:57.876 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T20:43:57.903 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi028.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675182', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 20:38:37.850409', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIK+zg/JWra1uD612P625cNOww2OHY02pxIF8/9BwvxAKT3XK7tUAMyzrzCJUhku57hQKvrn2sqmYng7u9kbJ64='} 2024-04-26T20:43:57.930 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675182', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 20:38:37.847730', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEEKkQDpg0Qr3CZiGw53Rqbpec5xVz6/xjXT8sCdl24BbF/tT3cu5+b/frYYHVCRg/0ymyHAxlTesvOs1Y95tMc='} 2024-04-26T20:43:57.930 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T20:43:57.937 INFO:teuthology.task.internal:roles: ubuntu@smithi028.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-26T20:43:57.937 INFO:teuthology.task.internal:roles: ubuntu@smithi103.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-26T20:43:57.937 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T20:43:58.007 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7feee4ac9160>, signals=[15]) 2024-04-26T20:43:58.007 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T20:43:58.013 INFO:teuthology.task.internal:Opening connections... 2024-04-26T20:43:58.014 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T20:43:58.015 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:43:58.093 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi103.front.sepia.ceph.com 2024-04-26T20:43:58.094 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:43:58.168 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T20:43:58.175 DEBUG:teuthology.orchestra.run.smithi028:> uname -m 2024-04-26T20:43:58.194 INFO:teuthology.orchestra.run.smithi028.stdout:x86_64 2024-04-26T20:43:58.194 DEBUG:teuthology.orchestra.run.smithi028:> cat /etc/os-release 2024-04-26T20:43:58.255 INFO:teuthology.orchestra.run.smithi028.stdout:NAME="CentOS Stream" 2024-04-26T20:43:58.255 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION="9" 2024-04-26T20:43:58.255 INFO:teuthology.orchestra.run.smithi028.stdout:ID="centos" 2024-04-26T20:43:58.255 INFO:teuthology.orchestra.run.smithi028.stdout:ID_LIKE="rhel fedora" 2024-04-26T20:43:58.256 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION_ID="9" 2024-04-26T20:43:58.256 INFO:teuthology.orchestra.run.smithi028.stdout:PLATFORM_ID="platform:el9" 2024-04-26T20:43:58.256 INFO:teuthology.orchestra.run.smithi028.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T20:43:58.256 INFO:teuthology.orchestra.run.smithi028.stdout:ANSI_COLOR="0;31" 2024-04-26T20:43:58.256 INFO:teuthology.orchestra.run.smithi028.stdout:LOGO="fedora-logo-icon" 2024-04-26T20:43:58.256 INFO:teuthology.orchestra.run.smithi028.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T20:43:58.256 INFO:teuthology.orchestra.run.smithi028.stdout:HOME_URL="https://centos.org/" 2024-04-26T20:43:58.256 INFO:teuthology.orchestra.run.smithi028.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T20:43:58.256 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T20:43:58.256 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T20:43:58.257 INFO:teuthology.lock.ops:Updating smithi028.front.sepia.ceph.com on lock server 2024-04-26T20:43:58.278 DEBUG:teuthology.orchestra.run.smithi103:> uname -m 2024-04-26T20:43:58.295 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64 2024-04-26T20:43:58.296 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2024-04-26T20:43:58.353 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Stream" 2024-04-26T20:43:58.353 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="9" 2024-04-26T20:43:58.353 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos" 2024-04-26T20:43:58.353 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora" 2024-04-26T20:43:58.353 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="9" 2024-04-26T20:43:58.353 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el9" 2024-04-26T20:43:58.353 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T20:43:58.353 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2024-04-26T20:43:58.353 INFO:teuthology.orchestra.run.smithi103.stdout:LOGO="fedora-logo-icon" 2024-04-26T20:43:58.353 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T20:43:58.353 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/" 2024-04-26T20:43:58.354 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T20:43:58.354 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T20:43:58.354 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T20:43:58.354 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2024-04-26T20:43:58.378 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T20:43:58.388 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T20:43:58.394 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T20:43:58.395 DEBUG:teuthology.orchestra.run.smithi028:> test '!' -e /home/ubuntu/cephtest 2024-04-26T20:43:58.397 DEBUG:teuthology.orchestra.run.smithi103:> test '!' -e /home/ubuntu/cephtest 2024-04-26T20:43:58.413 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T20:43:58.419 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T20:43:58.419 DEBUG:teuthology.orchestra.run.smithi028:> test -z $(ls -A /var/lib/ceph) 2024-04-26T20:43:58.455 DEBUG:teuthology.orchestra.run.smithi103:> test -z $(ls -A /var/lib/ceph) 2024-04-26T20:43:58.564 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T20:43:58.612 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T20:43:58.632 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T20:43:58.632 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-04-26T20:43:58.633 DEBUG:teuthology.orchestra.run.smithi028:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T20:43:58.633 DEBUG:teuthology.orchestra.run.smithi103:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T20:43:58.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:43:58.649 DEBUG:teuthology.task.kernel:sha1 for mon.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:43:58.649 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T20:43:58.649 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-04-26T20:43:58.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:43:58.650 DEBUG:teuthology.task.kernel:sha1 for mon.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:43:58.651 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T20:43:58.651 DEBUG:teuthology.orchestra.run.smithi028:> uname -r 2024-04-26T20:43:58.705 INFO:teuthology.orchestra.run.smithi103.stdout:5.14.0-437.el9.x86_64 2024-04-26T20:43:58.706 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:43:58.706 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T20:43:58.706 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:43:58.710 INFO:teuthology.orchestra.run.smithi028.stdout:5.14.0-437.el9.x86_64 2024-04-26T20:43:58.710 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:43:58.711 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T20:43:58.711 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:43:58.845 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum upgrade -y linux-firmware 2024-04-26T20:43:58.856 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum upgrade -y linux-firmware 2024-04-26T20:44:00.901 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-26T20:44:03.694 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-04-26T20:44:03.698 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 2.5 MB/s | 8.0 MB 00:03 2024-04-26T20:44:06.166 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-26T20:44:14.536 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - CRB 890 kB/s | 6.2 MB 00:07 2024-04-26T20:44:16.502 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - Extras packages 21 kB/s | 16 kB 00:00 2024-04-26T20:44:17.324 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 843 kB/s | 6.2 MB 00:07 2024-04-26T20:44:18.023 INFO:teuthology.orchestra.run.smithi028.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-26T20:44:19.292 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-04-26T20:44:20.293 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-04-26T20:44:23.271 INFO:teuthology.orchestra.run.smithi028.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-26T20:44:25.317 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T20:44:25.318 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2024-04-26T20:44:25.318 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T20:44:25.429 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-26T20:44:25.430 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:44:25.443 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-26T20:44:25.577 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T20:44:25.577 DEBUG:teuthology.orchestra.run.smithi028:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T20:44:25.580 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi028.front.sepia.ceph.com to complete... 2024-04-26T20:44:27.372 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-04-26T20:44:27.373 INFO:teuthology.orchestra.run.smithi103.stdout:Nothing to do. 2024-04-26T20:44:27.373 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-04-26T20:44:27.521 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-26T20:44:27.522 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T20:44:27.761 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T20:44:27.761 DEBUG:teuthology.orchestra.run.smithi103:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T20:44:27.764 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi103.front.sepia.ceph.com to complete... 2024-04-26T20:44:57.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26 20:44:57 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T20:44:57.287 INFO:teuthology.orchestra.run.smithi028.stderr:FINISHED --2024-04-26 20:44:57-- 2024-04-26T20:44:57.287 INFO:teuthology.orchestra.run.smithi028.stderr:Total wall clock time: 32s 2024-04-26T20:44:57.287 INFO:teuthology.orchestra.run.smithi028.stderr:Downloaded: 1 files, 341M in 31s (10.9 MB/s) 2024-04-26T20:44:57.290 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-26T20:44:57.290 DEBUG:teuthology.orchestra.run.smithi028:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T20:44:59.483 INFO:teuthology.orchestra.run.smithi028.stdout:Verifying... ######################################## 2024-04-26T20:44:59.612 INFO:teuthology.orchestra.run.smithi028.stdout:Preparing... ######################################## 2024-04-26T20:44:59.737 INFO:teuthology.orchestra.run.smithi028.stdout:Updating / installing... 2024-04-26T20:45:03.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26 20:45:03 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T20:45:03.521 INFO:teuthology.orchestra.run.smithi103.stderr:FINISHED --2024-04-26 20:45:03-- 2024-04-26T20:45:03.521 INFO:teuthology.orchestra.run.smithi103.stderr:Total wall clock time: 36s 2024-04-26T20:45:03.521 INFO:teuthology.orchestra.run.smithi103.stderr:Downloaded: 1 files, 341M in 35s (9.64 MB/s) 2024-04-26T20:45:03.524 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-26T20:45:03.524 DEBUG:teuthology.orchestra.run.smithi103:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T20:45:05.629 INFO:teuthology.orchestra.run.smithi103.stdout:Verifying... ######################################## 2024-04-26T20:45:05.784 INFO:teuthology.orchestra.run.smithi103.stdout:Preparing... ######################################## 2024-04-26T20:45:05.896 INFO:teuthology.orchestra.run.smithi103.stdout:Updating / installing... 2024-04-26T20:45:08.028 INFO:teuthology.orchestra.run.smithi028.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T20:45:13.836 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T20:45:45.795 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi028.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T20:45:45.796 DEBUG:teuthology.orchestra.run.smithi028:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T20:45:45.890 INFO:teuthology.orchestra.run.smithi028.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T20:45:45.891 INFO:teuthology.orchestra.run.smithi028.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T20:45:45.891 INFO:teuthology.orchestra.run.smithi028.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T20:45:45.891 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T20:45:45.891 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T20:45:45.891 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T20:45:45.891 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T20:45:45.891 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T20:45:45.891 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T20:45:45.891 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T20:45:45.891 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T20:45:45.891 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T20:45:45.891 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T20:45:45.891 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T20:45:45.891 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T20:45:45.892 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T20:45:45.892 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T20:45:45.892 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T20:45:45.892 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T20:45:45.892 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T20:45:45.892 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T20:45:45.892 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T20:45:45.892 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T20:45:45.892 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T20:45:45.892 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T20:45:45.892 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T20:45:45.892 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T20:45:45.892 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T20:45:45.892 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T20:45:45.892 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T20:45:45.893 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T20:45:45.893 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T20:45:45.893 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T20:45:45.893 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T20:45:45.893 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T20:45:45.893 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T20:45:45.893 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T20:45:45.893 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T20:45:45.893 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T20:45:45.893 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T20:45:45.893 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T20:45:45.893 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T20:45:45.893 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T20:45:45.893 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T20:45:45.894 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T20:45:45.894 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T20:45:45.894 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T20:45:45.894 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T20:45:45.894 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T20:45:45.894 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T20:45:45.894 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T20:45:45.894 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T20:45:45.894 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T20:45:45.894 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T20:45:45.894 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T20:45:45.894 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T20:45:45.894 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T20:45:45.894 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T20:45:45.895 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T20:45:45.895 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T20:45:45.895 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T20:45:45.895 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T20:45:45.895 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T20:45:45.895 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T20:45:45.895 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T20:45:45.895 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T20:45:45.895 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T20:45:45.895 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T20:45:45.895 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T20:45:45.895 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T20:45:45.895 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T20:45:45.896 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T20:45:45.896 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T20:45:45.896 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T20:45:45.896 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T20:45:45.896 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T20:45:45.896 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T20:45:45.896 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T20:45:45.896 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T20:45:45.896 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T20:45:45.896 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T20:45:45.896 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T20:45:45.896 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T20:45:45.896 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T20:45:45.897 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T20:45:45.897 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T20:45:45.897 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T20:45:45.897 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T20:45:45.897 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T20:45:45.897 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T20:45:45.897 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T20:45:45.897 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T20:45:45.897 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T20:45:45.897 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T20:45:45.897 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T20:45:45.897 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T20:45:45.897 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T20:45:45.897 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T20:45:45.897 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T20:45:45.898 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T20:45:45.898 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T20:45:45.898 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T20:45:45.898 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T20:45:45.898 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T20:45:45.898 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T20:45:45.898 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T20:45:45.898 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T20:45:45.898 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T20:45:45.898 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T20:45:45.898 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T20:45:45.898 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T20:45:45.898 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T20:45:45.899 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T20:45:45.899 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T20:45:45.899 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T20:45:45.899 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T20:45:45.899 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T20:45:45.899 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T20:45:45.899 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T20:45:45.899 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T20:45:45.899 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T20:45:45.899 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T20:45:45.899 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T20:45:45.899 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T20:45:45.899 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T20:45:45.899 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T20:45:45.899 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T20:45:45.900 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T20:45:45.900 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T20:45:45.900 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T20:45:45.900 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T20:45:45.900 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T20:45:45.900 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T20:45:45.900 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T20:45:45.900 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T20:45:45.900 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T20:45:45.900 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T20:45:45.900 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T20:45:45.900 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T20:45:45.900 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T20:45:45.900 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T20:45:45.901 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T20:45:45.901 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T20:45:45.901 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T20:45:45.901 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T20:45:45.901 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T20:45:45.901 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T20:45:45.901 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T20:45:45.901 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T20:45:45.901 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T20:45:45.901 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T20:45:45.901 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T20:45:45.901 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T20:45:45.901 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T20:45:45.901 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T20:45:45.901 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T20:45:45.902 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T20:45:45.902 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T20:45:45.902 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T20:45:45.902 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T20:45:45.902 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T20:45:45.902 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T20:45:45.902 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T20:45:45.902 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T20:45:45.902 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T20:45:45.902 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T20:45:45.902 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T20:45:45.902 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T20:45:45.903 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T20:45:45.903 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T20:45:45.903 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T20:45:45.903 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T20:45:45.903 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T20:45:45.903 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T20:45:45.903 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T20:45:45.903 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T20:45:45.903 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T20:45:45.903 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T20:45:45.903 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T20:45:45.903 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T20:45:45.903 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T20:45:45.903 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T20:45:45.904 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T20:45:45.904 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T20:45:45.904 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T20:45:45.904 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T20:45:45.904 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T20:45:45.904 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T20:45:45.904 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T20:45:45.904 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T20:45:45.904 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T20:45:45.904 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T20:45:45.904 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T20:45:45.904 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T20:45:45.904 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T20:45:45.904 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T20:45:45.905 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T20:45:45.905 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T20:45:45.905 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T20:45:45.905 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T20:45:45.905 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T20:45:45.905 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T20:45:45.905 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T20:45:45.905 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T20:45:45.905 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T20:45:45.905 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T20:45:45.905 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T20:45:45.905 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T20:45:45.905 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T20:45:45.905 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T20:45:45.906 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T20:45:45.906 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T20:45:45.906 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T20:45:45.906 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T20:45:45.906 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T20:45:45.906 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T20:45:45.906 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T20:45:45.906 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T20:45:45.906 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T20:45:45.906 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T20:45:45.906 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T20:45:45.906 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T20:45:45.907 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T20:45:45.907 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T20:45:45.907 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T20:45:45.907 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T20:45:45.907 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T20:45:45.907 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T20:45:45.907 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T20:45:45.907 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T20:45:45.907 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T20:45:45.907 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T20:45:45.907 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T20:45:45.907 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T20:45:45.907 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T20:45:45.907 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T20:45:45.908 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T20:45:45.908 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T20:45:45.908 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T20:45:45.908 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T20:45:45.908 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T20:45:45.908 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T20:45:45.908 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T20:45:45.908 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T20:45:45.908 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T20:45:45.908 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T20:45:45.908 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T20:45:45.908 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T20:45:45.908 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T20:45:45.908 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T20:45:45.909 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T20:45:45.909 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T20:45:45.909 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T20:45:45.909 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T20:45:45.909 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T20:45:45.909 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T20:45:45.909 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T20:45:45.909 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T20:45:45.909 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T20:45:45.909 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T20:45:45.909 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T20:45:45.909 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T20:45:45.909 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T20:45:45.909 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T20:45:45.910 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T20:45:45.910 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T20:45:45.910 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T20:45:45.910 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T20:45:45.910 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T20:45:45.910 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T20:45:45.910 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T20:45:45.910 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T20:45:45.910 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T20:45:45.910 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T20:45:45.910 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T20:45:45.910 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T20:45:45.910 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T20:45:45.910 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T20:45:45.911 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T20:45:45.911 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T20:45:45.911 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T20:45:45.911 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T20:45:45.911 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T20:45:45.911 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T20:45:45.911 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T20:45:45.911 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T20:45:45.911 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T20:45:45.911 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T20:45:45.911 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T20:45:45.911 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T20:45:45.911 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T20:45:45.911 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T20:45:45.911 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T20:45:45.912 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T20:45:45.912 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T20:45:45.912 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T20:45:45.912 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T20:45:45.912 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T20:45:45.912 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T20:45:45.912 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T20:45:45.912 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T20:45:45.912 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T20:45:45.912 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T20:45:45.912 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T20:45:45.912 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T20:45:45.912 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T20:45:45.912 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T20:45:45.912 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T20:45:45.913 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T20:45:45.913 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T20:45:45.913 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T20:45:45.913 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T20:45:45.913 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T20:45:45.913 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T20:45:45.913 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T20:45:45.913 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T20:45:45.913 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T20:45:45.913 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T20:45:45.913 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T20:45:45.913 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T20:45:45.913 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T20:45:45.913 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T20:45:45.913 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T20:45:45.914 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T20:45:45.914 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T20:45:45.914 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T20:45:45.914 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T20:45:45.914 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T20:45:45.914 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T20:45:45.914 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T20:45:45.914 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T20:45:45.914 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T20:45:45.914 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T20:45:45.914 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T20:45:45.914 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T20:45:45.914 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T20:45:45.914 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T20:45:45.915 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T20:45:45.915 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T20:45:45.915 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T20:45:45.915 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T20:45:45.915 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T20:45:45.915 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T20:45:45.915 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T20:45:45.915 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T20:45:45.915 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T20:45:45.915 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T20:45:45.915 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T20:45:45.915 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T20:45:45.915 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T20:45:45.915 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T20:45:45.915 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T20:45:45.915 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T20:45:45.916 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T20:45:45.916 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T20:45:45.916 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T20:45:45.916 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T20:45:45.916 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T20:45:45.916 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T20:45:45.916 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T20:45:45.916 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T20:45:45.916 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T20:45:45.916 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T20:45:45.916 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T20:45:45.916 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T20:45:45.916 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T20:45:45.916 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T20:45:45.917 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T20:45:45.917 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T20:45:45.917 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T20:45:45.917 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T20:45:45.917 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T20:45:45.917 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T20:45:45.917 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T20:45:45.917 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T20:45:45.917 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T20:45:45.917 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T20:45:45.917 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T20:45:45.917 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T20:45:45.917 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T20:45:45.918 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T20:45:45.918 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T20:45:45.918 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T20:45:45.918 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T20:45:45.918 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T20:45:45.918 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T20:45:45.918 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T20:45:45.918 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T20:45:45.918 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T20:45:45.918 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T20:45:45.918 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T20:45:45.918 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T20:45:45.918 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T20:45:45.918 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T20:45:45.919 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T20:45:45.919 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T20:45:45.919 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T20:45:45.919 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T20:45:45.919 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T20:45:45.919 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T20:45:45.919 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T20:45:45.919 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T20:45:45.919 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T20:45:45.919 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T20:45:45.919 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T20:45:45.919 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T20:45:45.919 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T20:45:45.919 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T20:45:45.919 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T20:45:45.920 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T20:45:45.920 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T20:45:45.920 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T20:45:45.920 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T20:45:45.920 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T20:45:45.920 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T20:45:45.920 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T20:45:45.920 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T20:45:45.920 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T20:45:45.920 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T20:45:45.920 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T20:45:45.920 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T20:45:45.920 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T20:45:45.920 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T20:45:45.920 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T20:45:45.921 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T20:45:45.921 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T20:45:45.921 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T20:45:45.921 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T20:45:45.921 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T20:45:45.921 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T20:45:45.921 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T20:45:45.921 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T20:45:45.921 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T20:45:45.921 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T20:45:45.921 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T20:45:45.921 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T20:45:45.921 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T20:45:45.921 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T20:45:45.921 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T20:45:45.922 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T20:45:45.922 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T20:45:45.922 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T20:45:45.922 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T20:45:45.922 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T20:45:45.922 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T20:45:45.922 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T20:45:45.922 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T20:45:45.922 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T20:45:45.922 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T20:45:45.922 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T20:45:45.922 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T20:45:45.922 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T20:45:45.922 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T20:45:45.923 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T20:45:45.923 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T20:45:45.923 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T20:45:45.923 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T20:45:45.923 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T20:45:45.923 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T20:45:45.923 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T20:45:45.923 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T20:45:45.923 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T20:45:45.923 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T20:45:45.923 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T20:45:45.923 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T20:45:45.923 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T20:45:45.923 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T20:45:45.923 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T20:45:45.923 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T20:45:45.924 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T20:45:45.924 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T20:45:45.924 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T20:45:45.924 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T20:45:45.924 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T20:45:45.924 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T20:45:45.924 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T20:45:45.924 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T20:45:45.924 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T20:45:45.924 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T20:45:45.924 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T20:45:45.925 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T20:45:45.925 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T20:45:45.925 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T20:45:45.925 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T20:45:45.925 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T20:45:45.925 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T20:45:45.925 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T20:45:45.925 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T20:45:45.925 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T20:45:45.925 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T20:45:45.925 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T20:45:45.925 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T20:45:45.925 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T20:45:45.925 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T20:45:45.926 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T20:45:45.926 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T20:45:45.926 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T20:45:45.926 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T20:45:45.926 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T20:45:45.926 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T20:45:45.926 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T20:45:45.926 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T20:45:45.926 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T20:45:45.926 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T20:45:45.926 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T20:45:45.926 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T20:45:45.926 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T20:45:45.926 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T20:45:45.927 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T20:45:45.927 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T20:45:45.927 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T20:45:45.927 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T20:45:45.927 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T20:45:45.927 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T20:45:45.927 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T20:45:45.927 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T20:45:45.927 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T20:45:45.927 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T20:45:45.927 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T20:45:45.927 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T20:45:45.927 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T20:45:45.927 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T20:45:45.928 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T20:45:45.928 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T20:45:45.928 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T20:45:45.928 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T20:45:45.928 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T20:45:45.928 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T20:45:45.928 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T20:45:45.928 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T20:45:45.928 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T20:45:45.928 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T20:45:45.928 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T20:45:45.928 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T20:45:45.928 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T20:45:45.928 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T20:45:45.928 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T20:45:45.929 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T20:45:45.929 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T20:45:45.929 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T20:45:45.929 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T20:45:45.929 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T20:45:45.929 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T20:45:45.929 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T20:45:45.929 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T20:45:45.929 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T20:45:45.929 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T20:45:45.929 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T20:45:45.929 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T20:45:45.929 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T20:45:45.929 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T20:45:45.930 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T20:45:45.930 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T20:45:45.930 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T20:45:45.930 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T20:45:45.930 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T20:45:45.930 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T20:45:45.930 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T20:45:45.930 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T20:45:45.930 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T20:45:45.930 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T20:45:45.930 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T20:45:45.930 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T20:45:45.930 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T20:45:45.930 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T20:45:45.931 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T20:45:45.931 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T20:45:45.931 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T20:45:45.931 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T20:45:45.931 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T20:45:45.931 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T20:45:45.931 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T20:45:45.931 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T20:45:45.931 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T20:45:45.931 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T20:45:45.931 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T20:45:45.931 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T20:45:45.931 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T20:45:45.931 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T20:45:45.931 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T20:45:45.932 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T20:45:45.932 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T20:45:45.932 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T20:45:45.932 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T20:45:45.932 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T20:45:45.932 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T20:45:45.932 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T20:45:45.932 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T20:45:45.932 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T20:45:45.932 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T20:45:45.933 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T20:45:45.933 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T20:45:45.933 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T20:45:45.933 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T20:45:45.933 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T20:45:45.933 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T20:45:45.933 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T20:45:45.933 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T20:45:45.933 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T20:45:45.933 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T20:45:45.933 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T20:45:45.933 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T20:45:45.933 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T20:45:45.933 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T20:45:45.933 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T20:45:45.934 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T20:45:45.934 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T20:45:45.934 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T20:45:45.934 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T20:45:45.934 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T20:45:45.934 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T20:45:45.934 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T20:45:45.934 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T20:45:45.934 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T20:45:45.934 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T20:45:45.934 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T20:45:45.934 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T20:45:45.934 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T20:45:45.934 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T20:45:45.934 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T20:45:45.935 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T20:45:45.935 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T20:45:45.935 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T20:45:45.935 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T20:45:45.935 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T20:45:45.935 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T20:45:45.935 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T20:45:45.935 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T20:45:45.935 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T20:45:45.935 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T20:45:45.935 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T20:45:45.935 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T20:45:45.935 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T20:45:45.935 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T20:45:45.936 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T20:45:45.936 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T20:45:45.936 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T20:45:45.936 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T20:45:45.936 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T20:45:45.936 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T20:45:45.936 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T20:45:45.936 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T20:45:45.936 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T20:45:45.936 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T20:45:45.936 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T20:45:45.936 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T20:45:45.936 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T20:45:45.936 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T20:45:45.937 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T20:45:45.937 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T20:45:45.937 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T20:45:45.937 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T20:45:45.937 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T20:45:45.937 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T20:45:45.937 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T20:45:45.937 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T20:45:45.937 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T20:45:45.937 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T20:45:45.937 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T20:45:45.937 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T20:45:45.937 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T20:45:45.937 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T20:45:45.938 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T20:45:45.938 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T20:45:45.938 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T20:45:45.938 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T20:45:45.938 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T20:45:45.938 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T20:45:45.938 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T20:45:45.938 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T20:45:45.938 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T20:45:45.938 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T20:45:45.938 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T20:45:45.938 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T20:45:45.938 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T20:45:45.938 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T20:45:45.939 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T20:45:45.939 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T20:45:45.939 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T20:45:45.939 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T20:45:45.939 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T20:45:45.939 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T20:45:45.939 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T20:45:45.939 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T20:45:45.939 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T20:45:45.939 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T20:45:45.939 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T20:45:45.939 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T20:45:45.939 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T20:45:45.939 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T20:45:45.939 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T20:45:45.940 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T20:45:45.940 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T20:45:45.940 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T20:45:45.940 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T20:45:45.940 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T20:45:45.940 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T20:45:45.940 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T20:45:45.940 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T20:45:45.940 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T20:45:45.940 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T20:45:45.940 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T20:45:45.940 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T20:45:45.940 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T20:45:45.940 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T20:45:45.941 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T20:45:45.941 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T20:45:45.941 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T20:45:45.941 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T20:45:45.941 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T20:45:45.941 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T20:45:45.941 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T20:45:45.941 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T20:45:45.941 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T20:45:45.941 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T20:45:45.941 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T20:45:45.941 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T20:45:45.941 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T20:45:45.941 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T20:45:45.941 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T20:45:45.942 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T20:45:45.942 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T20:45:45.942 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T20:45:45.942 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T20:45:45.942 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T20:45:45.942 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T20:45:45.942 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T20:45:45.942 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T20:45:45.942 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T20:45:45.942 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T20:45:45.942 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T20:45:45.942 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T20:45:45.942 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T20:45:45.942 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T20:45:45.942 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T20:45:45.943 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T20:45:45.943 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T20:45:45.943 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T20:45:45.943 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T20:45:45.943 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T20:45:45.943 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T20:45:45.943 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T20:45:45.943 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T20:45:45.943 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T20:45:45.943 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T20:45:45.943 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T20:45:45.943 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T20:45:45.943 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T20:45:45.943 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T20:45:45.943 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T20:45:45.944 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T20:45:45.944 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T20:45:45.944 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T20:45:45.944 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T20:45:45.944 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T20:45:45.944 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T20:45:45.944 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T20:45:45.944 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T20:45:45.944 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T20:45:45.944 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T20:45:45.944 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T20:45:45.944 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T20:45:45.944 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T20:45:45.944 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T20:45:45.944 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T20:45:45.945 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T20:45:45.945 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T20:45:45.945 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T20:45:45.945 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T20:45:45.945 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T20:45:45.945 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T20:45:45.945 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T20:45:45.945 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T20:45:45.945 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T20:45:45.945 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T20:45:45.945 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T20:45:45.945 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T20:45:45.945 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T20:45:45.945 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T20:45:45.945 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T20:45:45.946 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T20:45:45.946 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T20:45:45.946 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T20:45:45.946 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T20:45:45.946 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T20:45:45.946 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T20:45:45.946 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T20:45:45.946 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T20:45:45.946 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T20:45:45.946 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T20:45:45.946 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T20:45:45.946 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T20:45:45.946 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T20:45:45.946 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T20:45:45.947 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T20:45:45.947 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T20:45:45.947 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T20:45:45.947 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T20:45:45.947 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T20:45:45.947 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T20:45:45.947 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T20:45:45.947 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T20:45:45.947 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T20:45:45.947 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T20:45:45.947 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T20:45:45.947 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T20:45:45.947 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T20:45:45.947 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T20:45:45.947 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T20:45:45.948 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T20:45:45.948 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T20:45:45.948 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T20:45:45.948 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T20:45:45.948 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T20:45:45.948 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T20:45:45.948 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T20:45:45.948 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T20:45:45.948 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T20:45:45.948 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T20:45:45.948 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T20:45:45.948 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T20:45:45.948 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T20:45:45.948 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T20:45:45.948 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T20:45:45.949 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T20:45:45.949 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T20:45:45.949 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T20:45:45.949 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T20:45:45.949 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T20:45:45.949 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T20:45:45.949 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T20:45:45.949 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T20:45:45.949 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T20:45:45.949 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T20:45:45.949 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T20:45:45.949 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T20:45:45.949 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T20:45:45.949 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T20:45:45.949 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T20:45:45.950 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T20:45:45.950 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T20:45:45.950 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T20:45:45.950 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T20:45:45.950 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T20:45:45.950 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T20:45:45.950 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T20:45:45.950 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T20:45:45.950 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T20:45:45.950 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T20:45:45.950 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T20:45:45.950 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T20:45:45.950 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T20:45:45.950 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T20:45:45.951 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T20:45:45.951 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T20:45:45.951 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T20:45:45.951 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T20:45:45.951 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T20:45:45.951 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T20:45:45.951 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T20:45:45.953 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T20:45:45.953 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T20:45:45.953 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T20:45:45.953 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T20:45:45.953 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T20:45:45.953 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T20:45:45.953 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T20:45:45.954 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T20:45:45.954 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T20:45:45.954 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T20:45:45.954 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T20:45:45.954 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T20:45:45.954 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T20:45:45.954 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T20:45:45.954 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T20:45:45.954 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T20:45:45.954 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T20:45:45.954 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T20:45:45.954 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T20:45:45.954 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T20:45:45.954 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T20:45:45.954 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T20:45:45.955 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T20:45:45.955 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T20:45:45.955 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T20:45:45.955 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T20:45:45.955 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T20:45:45.955 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T20:45:45.955 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T20:45:45.955 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T20:45:45.955 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T20:45:45.955 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T20:45:45.955 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T20:45:45.955 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T20:45:45.955 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T20:45:45.955 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T20:45:45.955 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T20:45:45.956 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T20:45:45.956 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T20:45:45.956 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T20:45:45.956 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T20:45:45.956 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T20:45:45.956 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T20:45:45.956 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T20:45:45.956 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T20:45:45.956 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T20:45:45.956 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T20:45:45.956 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T20:45:45.956 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T20:45:45.956 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T20:45:45.956 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T20:45:45.957 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T20:45:45.957 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T20:45:45.957 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T20:45:45.957 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T20:45:45.957 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T20:45:45.957 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T20:45:45.957 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T20:45:45.957 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T20:45:45.957 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T20:45:45.957 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T20:45:45.957 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T20:45:45.957 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T20:45:45.957 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T20:45:45.957 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T20:45:45.957 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T20:45:45.958 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T20:45:45.958 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T20:45:45.958 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T20:45:45.958 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T20:45:45.958 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T20:45:45.958 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T20:45:45.958 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T20:45:45.958 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T20:45:45.958 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T20:45:45.958 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T20:45:45.958 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T20:45:45.958 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T20:45:45.958 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T20:45:45.958 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T20:45:45.958 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T20:45:45.959 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T20:45:45.959 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T20:45:45.959 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T20:45:45.959 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T20:45:45.959 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T20:45:45.959 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T20:45:45.959 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T20:45:45.959 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T20:45:45.959 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T20:45:45.959 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T20:45:45.959 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T20:45:45.959 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T20:45:45.959 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T20:45:45.959 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T20:45:45.959 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T20:45:45.960 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T20:45:45.960 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T20:45:45.960 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T20:45:45.960 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T20:45:45.960 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T20:45:45.960 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T20:45:45.960 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T20:45:45.960 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T20:45:45.960 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T20:45:45.960 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T20:45:45.960 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T20:45:45.960 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T20:45:45.960 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T20:45:45.960 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T20:45:45.960 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T20:45:45.961 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T20:45:45.961 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T20:45:45.961 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T20:45:45.961 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T20:45:45.961 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T20:45:45.961 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T20:45:45.961 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T20:45:45.961 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T20:45:45.961 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T20:45:45.961 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T20:45:45.961 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T20:45:45.961 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T20:45:45.961 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T20:45:45.961 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T20:45:45.961 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T20:45:45.962 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T20:45:45.962 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T20:45:45.962 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T20:45:45.962 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T20:45:45.962 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T20:45:45.962 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T20:45:45.962 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T20:45:45.962 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T20:45:45.962 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T20:45:45.962 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T20:45:45.962 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T20:45:45.962 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T20:45:45.962 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T20:45:45.962 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T20:45:45.962 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T20:45:45.963 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T20:45:45.963 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T20:45:45.963 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T20:45:45.963 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T20:45:45.963 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T20:45:45.963 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T20:45:45.963 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T20:45:45.963 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T20:45:45.963 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T20:45:45.963 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T20:45:45.963 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T20:45:45.963 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T20:45:45.963 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T20:45:45.963 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T20:45:45.963 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T20:45:45.964 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T20:45:45.964 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T20:45:45.964 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T20:45:45.964 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T20:45:45.964 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T20:45:45.964 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T20:45:45.964 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T20:45:45.964 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T20:45:45.964 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T20:45:45.964 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T20:45:45.964 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T20:45:45.964 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T20:45:45.964 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T20:45:45.964 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T20:45:45.964 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T20:45:45.965 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T20:45:45.965 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T20:45:45.965 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T20:45:45.965 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T20:45:45.965 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T20:45:45.965 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T20:45:45.965 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T20:45:45.965 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T20:45:45.965 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T20:45:45.965 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T20:45:45.965 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T20:45:45.965 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T20:45:45.965 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T20:45:45.965 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T20:45:45.965 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T20:45:45.966 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T20:45:45.966 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T20:45:45.966 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T20:45:45.966 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T20:45:45.966 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T20:45:45.966 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T20:45:45.966 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T20:45:45.966 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T20:45:45.966 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T20:45:45.966 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T20:45:45.966 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T20:45:45.966 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T20:45:45.966 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T20:45:45.966 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T20:45:45.966 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T20:45:45.967 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T20:45:45.967 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T20:45:45.967 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T20:45:45.967 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T20:45:45.967 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T20:45:45.967 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T20:45:45.967 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T20:45:45.967 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T20:45:45.967 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T20:45:45.967 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T20:45:45.967 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T20:45:45.967 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T20:45:45.968 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T20:45:45.968 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T20:45:45.968 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T20:45:45.968 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T20:45:45.968 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T20:45:45.968 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T20:45:45.968 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T20:45:45.968 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T20:45:45.968 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T20:45:45.968 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T20:45:45.968 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T20:45:45.968 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T20:45:45.968 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T20:45:45.968 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T20:45:45.968 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T20:45:45.969 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T20:45:45.969 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T20:45:45.969 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T20:45:45.969 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T20:45:45.969 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T20:45:45.969 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T20:45:45.969 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T20:45:45.969 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T20:45:45.969 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T20:45:45.969 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T20:45:45.969 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T20:45:45.969 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T20:45:45.969 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T20:45:45.969 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T20:45:45.969 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T20:45:45.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T20:45:45.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T20:45:45.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T20:45:45.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T20:45:45.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T20:45:45.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T20:45:45.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T20:45:45.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T20:45:45.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T20:45:45.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T20:45:45.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T20:45:45.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T20:45:45.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T20:45:45.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T20:45:45.970 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T20:45:45.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T20:45:45.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T20:45:45.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T20:45:45.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T20:45:45.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T20:45:45.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T20:45:45.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T20:45:45.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T20:45:45.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T20:45:45.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T20:45:45.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T20:45:45.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T20:45:45.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T20:45:45.971 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T20:45:45.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T20:45:45.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T20:45:45.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T20:45:45.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T20:45:45.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T20:45:45.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T20:45:45.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T20:45:45.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T20:45:45.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T20:45:45.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T20:45:45.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T20:45:45.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T20:45:45.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T20:45:45.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T20:45:45.972 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T20:45:45.973 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T20:45:45.973 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T20:45:45.973 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T20:45:45.973 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T20:45:45.973 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T20:45:45.973 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T20:45:45.973 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T20:45:45.973 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T20:45:45.973 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T20:45:45.973 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T20:45:45.973 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T20:45:45.973 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T20:45:45.973 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T20:45:45.973 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T20:45:45.973 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T20:45:45.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T20:45:45.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T20:45:45.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T20:45:45.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T20:45:45.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T20:45:45.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T20:45:45.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T20:45:45.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T20:45:45.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T20:45:45.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T20:45:45.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T20:45:45.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T20:45:45.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T20:45:45.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T20:45:45.974 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T20:45:45.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T20:45:45.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T20:45:45.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T20:45:45.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T20:45:45.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T20:45:45.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T20:45:45.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T20:45:45.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T20:45:45.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T20:45:45.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T20:45:45.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T20:45:45.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T20:45:45.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T20:45:45.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T20:45:45.975 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T20:45:45.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T20:45:45.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T20:45:45.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T20:45:45.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T20:45:45.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T20:45:45.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T20:45:45.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T20:45:45.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T20:45:45.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T20:45:45.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T20:45:45.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T20:45:45.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T20:45:45.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T20:45:45.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T20:45:45.976 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T20:45:45.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T20:45:45.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T20:45:45.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T20:45:45.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T20:45:45.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T20:45:45.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T20:45:45.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T20:45:45.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T20:45:45.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T20:45:45.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T20:45:45.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T20:45:45.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T20:45:45.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T20:45:45.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T20:45:45.977 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T20:45:45.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T20:45:45.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T20:45:45.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T20:45:45.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T20:45:45.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T20:45:45.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T20:45:45.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T20:45:45.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T20:45:45.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T20:45:45.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T20:45:45.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T20:45:45.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T20:45:45.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T20:45:45.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T20:45:45.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T20:45:45.978 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T20:45:45.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T20:45:45.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T20:45:45.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T20:45:45.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T20:45:45.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T20:45:45.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T20:45:45.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T20:45:45.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T20:45:45.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T20:45:45.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T20:45:45.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T20:45:45.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T20:45:45.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T20:45:45.979 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T20:45:45.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T20:45:45.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T20:45:45.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T20:45:45.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T20:45:45.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T20:45:45.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T20:45:45.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T20:45:45.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T20:45:45.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T20:45:45.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T20:45:45.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T20:45:45.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T20:45:45.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T20:45:45.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T20:45:45.980 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T20:45:45.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T20:45:45.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T20:45:45.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T20:45:45.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T20:45:45.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T20:45:45.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T20:45:45.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T20:45:45.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T20:45:45.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T20:45:45.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T20:45:45.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T20:45:45.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T20:45:45.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T20:45:45.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T20:45:45.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T20:45:45.981 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T20:45:45.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T20:45:45.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T20:45:45.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T20:45:45.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T20:45:45.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T20:45:45.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T20:45:45.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T20:45:45.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T20:45:45.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T20:45:45.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T20:45:45.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T20:45:45.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T20:45:45.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T20:45:45.982 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T20:45:45.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T20:45:45.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T20:45:45.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T20:45:45.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T20:45:45.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T20:45:45.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T20:45:45.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T20:45:45.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T20:45:45.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T20:45:45.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T20:45:45.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T20:45:45.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T20:45:45.983 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T20:45:45.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T20:45:45.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T20:45:45.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T20:45:45.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T20:45:45.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T20:45:45.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T20:45:45.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T20:45:45.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T20:45:45.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T20:45:45.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T20:45:45.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T20:45:45.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T20:45:45.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T20:45:45.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T20:45:45.984 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T20:45:45.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T20:45:45.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T20:45:45.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T20:45:45.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T20:45:45.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T20:45:45.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T20:45:45.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T20:45:45.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T20:45:45.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T20:45:45.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T20:45:45.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T20:45:45.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T20:45:45.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T20:45:45.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T20:45:45.985 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T20:45:45.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T20:45:45.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T20:45:45.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T20:45:45.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T20:45:45.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T20:45:45.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T20:45:45.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T20:45:45.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T20:45:45.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T20:45:45.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T20:45:45.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T20:45:45.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T20:45:45.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T20:45:45.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T20:45:45.986 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T20:45:45.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T20:45:45.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T20:45:45.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T20:45:45.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T20:45:45.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T20:45:45.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T20:45:45.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T20:45:45.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T20:45:45.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T20:45:45.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T20:45:45.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T20:45:45.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T20:45:45.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T20:45:45.987 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T20:45:45.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T20:45:45.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T20:45:45.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T20:45:45.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T20:45:45.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T20:45:45.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T20:45:45.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T20:45:45.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T20:45:45.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T20:45:45.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T20:45:45.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T20:45:45.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T20:45:45.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T20:45:45.988 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T20:45:45.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T20:45:45.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T20:45:45.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T20:45:45.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T20:45:45.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T20:45:45.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T20:45:45.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T20:45:45.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T20:45:45.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T20:45:45.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T20:45:45.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T20:45:45.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T20:45:45.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T20:45:45.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T20:45:45.989 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T20:45:45.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T20:45:45.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T20:45:45.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T20:45:45.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T20:45:45.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T20:45:45.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T20:45:45.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T20:45:45.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T20:45:45.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T20:45:45.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T20:45:45.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T20:45:45.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T20:45:45.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T20:45:45.990 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T20:45:45.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T20:45:45.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T20:45:45.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T20:45:45.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T20:45:45.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T20:45:45.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T20:45:45.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T20:45:45.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T20:45:45.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T20:45:45.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T20:45:45.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T20:45:45.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T20:45:45.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T20:45:45.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T20:45:45.991 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T20:45:45.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T20:45:45.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T20:45:45.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T20:45:45.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T20:45:45.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T20:45:45.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T20:45:45.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T20:45:45.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T20:45:45.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T20:45:45.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T20:45:45.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T20:45:45.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T20:45:45.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T20:45:45.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T20:45:45.992 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T20:45:45.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T20:45:45.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T20:45:45.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T20:45:45.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T20:45:45.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T20:45:45.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T20:45:45.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T20:45:45.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T20:45:45.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T20:45:45.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T20:45:45.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T20:45:45.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T20:45:45.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T20:45:45.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T20:45:45.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T20:45:45.993 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T20:45:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T20:45:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T20:45:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T20:45:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T20:45:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T20:45:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T20:45:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T20:45:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T20:45:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T20:45:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T20:45:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T20:45:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T20:45:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T20:45:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T20:45:45.994 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T20:45:45.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T20:45:45.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T20:45:45.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T20:45:45.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T20:45:45.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T20:45:45.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T20:45:45.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T20:45:45.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T20:45:45.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T20:45:45.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T20:45:45.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T20:45:45.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T20:45:45.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T20:45:45.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T20:45:45.995 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T20:45:45.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T20:45:45.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T20:45:45.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T20:45:45.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T20:45:45.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T20:45:45.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T20:45:45.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T20:45:45.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T20:45:45.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T20:45:45.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T20:45:45.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T20:45:45.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T20:45:45.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T20:45:45.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T20:45:45.996 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T20:45:45.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T20:45:45.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T20:45:45.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T20:45:45.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T20:45:45.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T20:45:45.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T20:45:45.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T20:45:45.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T20:45:45.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T20:45:45.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T20:45:45.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T20:45:45.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T20:45:45.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T20:45:45.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T20:45:45.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T20:45:45.997 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T20:45:45.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T20:45:45.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T20:45:45.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T20:45:45.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T20:45:45.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T20:45:45.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T20:45:45.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T20:45:45.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T20:45:45.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T20:45:45.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T20:45:45.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T20:45:45.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T20:45:45.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T20:45:45.998 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T20:45:45.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T20:45:45.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T20:45:45.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T20:45:45.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T20:45:45.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T20:45:45.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T20:45:45.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T20:45:45.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T20:45:45.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T20:45:45.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T20:45:45.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T20:45:45.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T20:45:45.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T20:45:45.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T20:45:45.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T20:45:45.999 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T20:45:46.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T20:45:46.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T20:45:46.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T20:45:46.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T20:45:46.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T20:45:46.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T20:45:46.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T20:45:46.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T20:45:46.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T20:45:46.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T20:45:46.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T20:45:46.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T20:45:46.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T20:45:46.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T20:45:46.000 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T20:45:46.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T20:45:46.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T20:45:46.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T20:45:46.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T20:45:46.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T20:45:46.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T20:45:46.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T20:45:46.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T20:45:46.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T20:45:46.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T20:45:46.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T20:45:46.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T20:45:46.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T20:45:46.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T20:45:46.001 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T20:45:46.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T20:45:46.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T20:45:46.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T20:45:46.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T20:45:46.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T20:45:46.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T20:45:46.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T20:45:46.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T20:45:46.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T20:45:46.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T20:45:46.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T20:45:46.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T20:45:46.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T20:45:46.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T20:45:46.002 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T20:45:46.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T20:45:46.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T20:45:46.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T20:45:46.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T20:45:46.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T20:45:46.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T20:45:46.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T20:45:46.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T20:45:46.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T20:45:46.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T20:45:46.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T20:45:46.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T20:45:46.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T20:45:46.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T20:45:46.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T20:45:46.003 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T20:45:46.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T20:45:46.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T20:45:46.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T20:45:46.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T20:45:46.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T20:45:46.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T20:45:46.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T20:45:46.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T20:45:46.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T20:45:46.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T20:45:46.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T20:45:46.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T20:45:46.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T20:45:46.004 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T20:45:46.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T20:45:46.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T20:45:46.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T20:45:46.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T20:45:46.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T20:45:46.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T20:45:46.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T20:45:46.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T20:45:46.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T20:45:46.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T20:45:46.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T20:45:46.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T20:45:46.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T20:45:46.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T20:45:46.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T20:45:46.005 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T20:45:46.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T20:45:46.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T20:45:46.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T20:45:46.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T20:45:46.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T20:45:46.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T20:45:46.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T20:45:46.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T20:45:46.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T20:45:46.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T20:45:46.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T20:45:46.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T20:45:46.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T20:45:46.006 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T20:45:46.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T20:45:46.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T20:45:46.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T20:45:46.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T20:45:46.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T20:45:46.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T20:45:46.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T20:45:46.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T20:45:46.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T20:45:46.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T20:45:46.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T20:45:46.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T20:45:46.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T20:45:46.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T20:45:46.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T20:45:46.007 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T20:45:46.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T20:45:46.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T20:45:46.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T20:45:46.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T20:45:46.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T20:45:46.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T20:45:46.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T20:45:46.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T20:45:46.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T20:45:46.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T20:45:46.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T20:45:46.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T20:45:46.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T20:45:46.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T20:45:46.008 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T20:45:46.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T20:45:46.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T20:45:46.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T20:45:46.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T20:45:46.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T20:45:46.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T20:45:46.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T20:45:46.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T20:45:46.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T20:45:46.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T20:45:46.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T20:45:46.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T20:45:46.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T20:45:46.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T20:45:46.009 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T20:45:46.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T20:45:46.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T20:45:46.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T20:45:46.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T20:45:46.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T20:45:46.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T20:45:46.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T20:45:46.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T20:45:46.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T20:45:46.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T20:45:46.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T20:45:46.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T20:45:46.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T20:45:46.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T20:45:46.010 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T20:45:46.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T20:45:46.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T20:45:46.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T20:45:46.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T20:45:46.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T20:45:46.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T20:45:46.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T20:45:46.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T20:45:46.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T20:45:46.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T20:45:46.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T20:45:46.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T20:45:46.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T20:45:46.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T20:45:46.011 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T20:45:46.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T20:45:46.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T20:45:46.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T20:45:46.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T20:45:46.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T20:45:46.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T20:45:46.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T20:45:46.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T20:45:46.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T20:45:46.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T20:45:46.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T20:45:46.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T20:45:46.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T20:45:46.012 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T20:45:46.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T20:45:46.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T20:45:46.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T20:45:46.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T20:45:46.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T20:45:46.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T20:45:46.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T20:45:46.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T20:45:46.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T20:45:46.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T20:45:46.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T20:45:46.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T20:45:46.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T20:45:46.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T20:45:46.013 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T20:45:46.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T20:45:46.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T20:45:46.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T20:45:46.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T20:45:46.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T20:45:46.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T20:45:46.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T20:45:46.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T20:45:46.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T20:45:46.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T20:45:46.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T20:45:46.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T20:45:46.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T20:45:46.014 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T20:45:46.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T20:45:46.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T20:45:46.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T20:45:46.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T20:45:46.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T20:45:46.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T20:45:46.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T20:45:46.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T20:45:46.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T20:45:46.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T20:45:46.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T20:45:46.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T20:45:46.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T20:45:46.015 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T20:45:46.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T20:45:46.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T20:45:46.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T20:45:46.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T20:45:46.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T20:45:46.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T20:45:46.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T20:45:46.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T20:45:46.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T20:45:46.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T20:45:46.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T20:45:46.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T20:45:46.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T20:45:46.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T20:45:46.016 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T20:45:46.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T20:45:46.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T20:45:46.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T20:45:46.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T20:45:46.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T20:45:46.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T20:45:46.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T20:45:46.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T20:45:46.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T20:45:46.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T20:45:46.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T20:45:46.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T20:45:46.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T20:45:46.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T20:45:46.017 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T20:45:46.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T20:45:46.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T20:45:46.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T20:45:46.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T20:45:46.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T20:45:46.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T20:45:46.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T20:45:46.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T20:45:46.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T20:45:46.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T20:45:46.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T20:45:46.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T20:45:46.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T20:45:46.018 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T20:45:46.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T20:45:46.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T20:45:46.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T20:45:46.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T20:45:46.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T20:45:46.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T20:45:46.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T20:45:46.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T20:45:46.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T20:45:46.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T20:45:46.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T20:45:46.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T20:45:46.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T20:45:46.019 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T20:45:46.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T20:45:46.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T20:45:46.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T20:45:46.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T20:45:46.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T20:45:46.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T20:45:46.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T20:45:46.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T20:45:46.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T20:45:46.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T20:45:46.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T20:45:46.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T20:45:46.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T20:45:46.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T20:45:46.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T20:45:46.020 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T20:45:46.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T20:45:46.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T20:45:46.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T20:45:46.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T20:45:46.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T20:45:46.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T20:45:46.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T20:45:46.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T20:45:46.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T20:45:46.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T20:45:46.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T20:45:46.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T20:45:46.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T20:45:46.021 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T20:45:46.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T20:45:46.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T20:45:46.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T20:45:46.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T20:45:46.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T20:45:46.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T20:45:46.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T20:45:46.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T20:45:46.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T20:45:46.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T20:45:46.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T20:45:46.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T20:45:46.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T20:45:46.022 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T20:45:46.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T20:45:46.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T20:45:46.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T20:45:46.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T20:45:46.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T20:45:46.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T20:45:46.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T20:45:46.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T20:45:46.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T20:45:46.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T20:45:46.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T20:45:46.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T20:45:46.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T20:45:46.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T20:45:46.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T20:45:46.023 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T20:45:46.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T20:45:46.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T20:45:46.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T20:45:46.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T20:45:46.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T20:45:46.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T20:45:46.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T20:45:46.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T20:45:46.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T20:45:46.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T20:45:46.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T20:45:46.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T20:45:46.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T20:45:46.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T20:45:46.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T20:45:46.024 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T20:45:46.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T20:45:46.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T20:45:46.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T20:45:46.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T20:45:46.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T20:45:46.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T20:45:46.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T20:45:46.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T20:45:46.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T20:45:46.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T20:45:46.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T20:45:46.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T20:45:46.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T20:45:46.025 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T20:45:46.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T20:45:46.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T20:45:46.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T20:45:46.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T20:45:46.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T20:45:46.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T20:45:46.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T20:45:46.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T20:45:46.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T20:45:46.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T20:45:46.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T20:45:46.026 INFO:teuthology.orchestra.run.smithi028.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T20:45:46.031 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T20:45:46.031 DEBUG:teuthology.orchestra.run.smithi028:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T20:45:46.056 INFO:teuthology.orchestra.run.smithi028.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T20:45:46.056 INFO:teuthology.orchestra.run.smithi028.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T20:45:46.056 INFO:teuthology.orchestra.run.smithi028.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T20:45:46.056 INFO:teuthology.orchestra.run.smithi028.stdout:fi 2024-04-26T20:45:46.056 INFO:teuthology.orchestra.run.smithi028.stdout:for file in vmlinuz System.map config; do 2024-04-26T20:45:46.056 INFO:teuthology.orchestra.run.smithi028.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T20:45:46.056 INFO:teuthology.orchestra.run.smithi028.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:45:46.057 INFO:teuthology.orchestra.run.smithi028.stdout: fi 2024-04-26T20:45:46.057 INFO:teuthology.orchestra.run.smithi028.stdout:done 2024-04-26T20:45:46.057 INFO:teuthology.orchestra.run.smithi028.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T20:45:46.057 INFO:teuthology.orchestra.run.smithi028.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T20:45:46.057 INFO:teuthology.orchestra.run.smithi028.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:45:46.057 INFO:teuthology.orchestra.run.smithi028.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:45:46.057 INFO:teuthology.orchestra.run.smithi028.stdout:fi 2024-04-26T20:45:46.057 INFO:teuthology.orchestra.run.smithi028.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T20:45:46.057 INFO:teuthology.orchestra.run.smithi028.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T20:45:46.057 INFO:teuthology.orchestra.run.smithi028.stdout:fi 2024-04-26T20:45:46.057 INFO:teuthology.orchestra.run.smithi028.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T20:45:46.057 INFO:teuthology.orchestra.run.smithi028.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T20:45:46.057 INFO:teuthology.orchestra.run.smithi028.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T20:45:46.057 INFO:teuthology.orchestra.run.smithi028.stdout:fi 2024-04-26T20:45:46.058 DEBUG:teuthology.orchestra.run.smithi028:> sudo rpm -qi grub2-tools 2024-04-26T20:45:46.144 INFO:teuthology.orchestra.run.smithi028.stdout:Name : grub2-tools 2024-04-26T20:45:46.145 INFO:teuthology.orchestra.run.smithi028.stdout:Epoch : 1 2024-04-26T20:45:46.145 INFO:teuthology.orchestra.run.smithi028.stdout:Version : 2.06 2024-04-26T20:45:46.145 INFO:teuthology.orchestra.run.smithi028.stdout:Release : 68.el9 2024-04-26T20:45:46.145 INFO:teuthology.orchestra.run.smithi028.stdout:Architecture: x86_64 2024-04-26T20:45:46.145 INFO:teuthology.orchestra.run.smithi028.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T20:45:46.145 INFO:teuthology.orchestra.run.smithi028.stdout:Group : Unspecified 2024-04-26T20:45:46.145 INFO:teuthology.orchestra.run.smithi028.stdout:Size : 8264254 2024-04-26T20:45:46.145 INFO:teuthology.orchestra.run.smithi028.stdout:License : GPLv3+ 2024-04-26T20:45:46.145 INFO:teuthology.orchestra.run.smithi028.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T20:45:46.145 INFO:teuthology.orchestra.run.smithi028.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T20:45:46.145 INFO:teuthology.orchestra.run.smithi028.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T20:45:46.146 INFO:teuthology.orchestra.run.smithi028.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T20:45:46.146 INFO:teuthology.orchestra.run.smithi028.stdout:Packager : builder@centos.org 2024-04-26T20:45:46.146 INFO:teuthology.orchestra.run.smithi028.stdout:Vendor : CentOS 2024-04-26T20:45:46.146 INFO:teuthology.orchestra.run.smithi028.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T20:45:46.146 INFO:teuthology.orchestra.run.smithi028.stdout:Summary : Support tools for GRUB. 2024-04-26T20:45:46.146 INFO:teuthology.orchestra.run.smithi028.stdout:Description : 2024-04-26T20:45:46.146 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:45:46.146 INFO:teuthology.orchestra.run.smithi028.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T20:45:46.146 INFO:teuthology.orchestra.run.smithi028.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T20:45:46.146 INFO:teuthology.orchestra.run.smithi028.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T20:45:46.146 INFO:teuthology.orchestra.run.smithi028.stdout:hardware devices. 2024-04-26T20:45:46.146 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:45:46.146 INFO:teuthology.orchestra.run.smithi028.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T20:45:46.147 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T20:45:46.147 INFO:teuthology.task.kernel:Updating grub on smithi028 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T20:45:46.148 DEBUG:teuthology.orchestra.run.smithi028:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T20:45:46.664 INFO:teuthology.orchestra.run.smithi028.stderr:Generating grub configuration file ... 2024-04-26T20:45:48.086 INFO:teuthology.orchestra.run.smithi028.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T20:45:48.115 INFO:teuthology.orchestra.run.smithi028.stderr:done 2024-04-26T20:45:48.118 DEBUG:teuthology.orchestra.run.smithi028:> mktemp 2024-04-26T20:45:48.147 INFO:teuthology.orchestra.run.smithi028.stdout:/tmp/tmp.BhzKhOlXCZ 2024-04-26T20:45:48.148 DEBUG:teuthology.orchestra.run.smithi028:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.BhzKhOlXCZ 2024-04-26T20:45:48.225 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 0666 /tmp/tmp.BhzKhOlXCZ 2024-04-26T20:45:48.372 DEBUG:teuthology.orchestra.remote:smithi028:/tmp/tmp.BhzKhOlXCZ is 6KB 2024-04-26T20:45:48.420 DEBUG:teuthology.orchestra.run.smithi028:> rm -fr /tmp/tmp.BhzKhOlXCZ 2024-04-26T20:45:48.435 DEBUG:teuthology.orchestra.run.smithi028:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T20:45:48.498 INFO:teuthology.orchestra.run.smithi028.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T20:45:48.498 INFO:teuthology.orchestra.run.smithi028.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T20:45:48.498 INFO:teuthology.orchestra.run.smithi028.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T20:45:48.498 INFO:teuthology.orchestra.run.smithi028.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T20:45:48.499 DEBUG:teuthology.orchestra.run.smithi028:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T20:45:48.651 DEBUG:teuthology.orchestra.run.smithi028:> sudo shutdown -r now 2024-04-26T20:45:51.613 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi103.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T20:45:51.613 DEBUG:teuthology.orchestra.run.smithi103:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T20:45:51.703 INFO:teuthology.orchestra.run.smithi103.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T20:45:51.703 INFO:teuthology.orchestra.run.smithi103.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T20:45:51.703 INFO:teuthology.orchestra.run.smithi103.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T20:45:51.703 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T20:45:51.703 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T20:45:51.704 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T20:45:51.704 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T20:45:51.704 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T20:45:51.704 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T20:45:51.704 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T20:45:51.704 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T20:45:51.704 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T20:45:51.704 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T20:45:51.704 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T20:45:51.704 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T20:45:51.704 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T20:45:51.704 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T20:45:51.704 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T20:45:51.704 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T20:45:51.704 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T20:45:51.705 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T20:45:51.705 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T20:45:51.705 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T20:45:51.705 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T20:45:51.705 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T20:45:51.705 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T20:45:51.705 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T20:45:51.705 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T20:45:51.705 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T20:45:51.705 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T20:45:51.705 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T20:45:51.705 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T20:45:51.705 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T20:45:51.705 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T20:45:51.706 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T20:45:51.706 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T20:45:51.706 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T20:45:51.706 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T20:45:51.706 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T20:45:51.706 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T20:45:51.706 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T20:45:51.706 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T20:45:51.706 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T20:45:51.706 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T20:45:51.706 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T20:45:51.706 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T20:45:51.706 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T20:45:51.706 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T20:45:51.707 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T20:45:51.707 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T20:45:51.707 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T20:45:51.707 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T20:45:51.707 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T20:45:51.707 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T20:45:51.707 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T20:45:51.707 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T20:45:51.707 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T20:45:51.707 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T20:45:51.707 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T20:45:51.707 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T20:45:51.707 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T20:45:51.707 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T20:45:51.708 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T20:45:51.708 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T20:45:51.708 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T20:45:51.708 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T20:45:51.708 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T20:45:51.708 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T20:45:51.708 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T20:45:51.708 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T20:45:51.708 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T20:45:51.708 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T20:45:51.708 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T20:45:51.708 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T20:45:51.708 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T20:45:51.708 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T20:45:51.708 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T20:45:51.709 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T20:45:51.709 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T20:45:51.709 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T20:45:51.709 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T20:45:51.709 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T20:45:51.709 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T20:45:51.709 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T20:45:51.709 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T20:45:51.709 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T20:45:51.709 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T20:45:51.709 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T20:45:51.709 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T20:45:51.709 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T20:45:51.710 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T20:45:51.710 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T20:45:51.710 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T20:45:51.710 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T20:45:51.710 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T20:45:51.710 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T20:45:51.710 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T20:45:51.710 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T20:45:51.710 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T20:45:51.710 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T20:45:51.710 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T20:45:51.710 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T20:45:51.710 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T20:45:51.710 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T20:45:51.711 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T20:45:51.711 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T20:45:51.711 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T20:45:51.711 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T20:45:51.711 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T20:45:51.711 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T20:45:51.711 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T20:45:51.711 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T20:45:51.711 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T20:45:51.711 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T20:45:51.711 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T20:45:51.711 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T20:45:51.711 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T20:45:51.711 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T20:45:51.712 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T20:45:51.712 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T20:45:51.712 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T20:45:51.712 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T20:45:51.712 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T20:45:51.712 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T20:45:51.712 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T20:45:51.712 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T20:45:51.712 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T20:45:51.712 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T20:45:51.712 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T20:45:51.712 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T20:45:51.712 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T20:45:51.712 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T20:45:51.713 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T20:45:51.713 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T20:45:51.713 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T20:45:51.713 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T20:45:51.713 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T20:45:51.713 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T20:45:51.713 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T20:45:51.713 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T20:45:51.713 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T20:45:51.713 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T20:45:51.713 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T20:45:51.713 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T20:45:51.713 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T20:45:51.713 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T20:45:51.714 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T20:45:51.714 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T20:45:51.714 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T20:45:51.714 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T20:45:51.714 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T20:45:51.714 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T20:45:51.714 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T20:45:51.714 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T20:45:51.714 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T20:45:51.714 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T20:45:51.714 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T20:45:51.714 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T20:45:51.714 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T20:45:51.714 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T20:45:51.715 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T20:45:51.715 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T20:45:51.715 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T20:45:51.715 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T20:45:51.715 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T20:45:51.715 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T20:45:51.715 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T20:45:51.715 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T20:45:51.715 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T20:45:51.715 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T20:45:51.715 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T20:45:51.715 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T20:45:51.715 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T20:45:51.716 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T20:45:51.716 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T20:45:51.716 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T20:45:51.716 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T20:45:51.716 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T20:45:51.716 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T20:45:51.716 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T20:45:51.716 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T20:45:51.716 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T20:45:51.716 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T20:45:51.716 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T20:45:51.716 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T20:45:51.716 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T20:45:51.716 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T20:45:51.717 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T20:45:51.717 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T20:45:51.717 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T20:45:51.717 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T20:45:51.717 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T20:45:51.717 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T20:45:51.717 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T20:45:51.717 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T20:45:51.717 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T20:45:51.717 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T20:45:51.717 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T20:45:51.717 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T20:45:51.717 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T20:45:51.717 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T20:45:51.718 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T20:45:51.718 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T20:45:51.718 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T20:45:51.718 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T20:45:51.718 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T20:45:51.718 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T20:45:51.718 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T20:45:51.718 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T20:45:51.718 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T20:45:51.718 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T20:45:51.718 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T20:45:51.718 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T20:45:51.718 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T20:45:51.718 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T20:45:51.719 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T20:45:51.719 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T20:45:51.719 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T20:45:51.719 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T20:45:51.719 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T20:45:51.719 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T20:45:51.719 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T20:45:51.719 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T20:45:51.719 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T20:45:51.719 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T20:45:51.719 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T20:45:51.719 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T20:45:51.719 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T20:45:51.719 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T20:45:51.719 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T20:45:51.720 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T20:45:51.720 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T20:45:51.720 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T20:45:51.720 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T20:45:51.720 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T20:45:51.720 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T20:45:51.720 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T20:45:51.720 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T20:45:51.720 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T20:45:51.720 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T20:45:51.720 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T20:45:51.720 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T20:45:51.720 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T20:45:51.720 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T20:45:51.721 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T20:45:51.721 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T20:45:51.721 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T20:45:51.721 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T20:45:51.721 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T20:45:51.721 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T20:45:51.721 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T20:45:51.721 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T20:45:51.721 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T20:45:51.721 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T20:45:51.721 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T20:45:51.721 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T20:45:51.721 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T20:45:51.721 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T20:45:51.722 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T20:45:51.722 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T20:45:51.722 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T20:45:51.722 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T20:45:51.722 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T20:45:51.722 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T20:45:51.722 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T20:45:51.722 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T20:45:51.722 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T20:45:51.722 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T20:45:51.722 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T20:45:51.722 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T20:45:51.722 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T20:45:51.722 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T20:45:51.723 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T20:45:51.723 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T20:45:51.723 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T20:45:51.723 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T20:45:51.723 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T20:45:51.723 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T20:45:51.723 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T20:45:51.723 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T20:45:51.723 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T20:45:51.723 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T20:45:51.723 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T20:45:51.723 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T20:45:51.723 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T20:45:51.723 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T20:45:51.723 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T20:45:51.724 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T20:45:51.724 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T20:45:51.724 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T20:45:51.724 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T20:45:51.724 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T20:45:51.724 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T20:45:51.724 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T20:45:51.724 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T20:45:51.724 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T20:45:51.724 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T20:45:51.724 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T20:45:51.724 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T20:45:51.724 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T20:45:51.724 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T20:45:51.724 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T20:45:51.725 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T20:45:51.725 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T20:45:51.725 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T20:45:51.725 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T20:45:51.725 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T20:45:51.725 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T20:45:51.725 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T20:45:51.725 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T20:45:51.725 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T20:45:51.725 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T20:45:51.725 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T20:45:51.725 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T20:45:51.725 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T20:45:51.725 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T20:45:51.725 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T20:45:51.726 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T20:45:51.726 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T20:45:51.726 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T20:45:51.726 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T20:45:51.726 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T20:45:51.726 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T20:45:51.726 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T20:45:51.726 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T20:45:51.726 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T20:45:51.726 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T20:45:51.726 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T20:45:51.726 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T20:45:51.726 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T20:45:51.726 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T20:45:51.727 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T20:45:51.727 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T20:45:51.727 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T20:45:51.727 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T20:45:51.727 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T20:45:51.727 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T20:45:51.727 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T20:45:51.727 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T20:45:51.727 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T20:45:51.727 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T20:45:51.727 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T20:45:51.727 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T20:45:51.727 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T20:45:51.727 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T20:45:51.727 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T20:45:51.728 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T20:45:51.728 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T20:45:51.728 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T20:45:51.728 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T20:45:51.728 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T20:45:51.728 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T20:45:51.728 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T20:45:51.728 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T20:45:51.728 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T20:45:51.728 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T20:45:51.728 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T20:45:51.728 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T20:45:51.728 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T20:45:51.728 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T20:45:51.729 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T20:45:51.729 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T20:45:51.729 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T20:45:51.729 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T20:45:51.729 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T20:45:51.729 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T20:45:51.729 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T20:45:51.729 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T20:45:51.729 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T20:45:51.729 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T20:45:51.729 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T20:45:51.729 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T20:45:51.729 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T20:45:51.729 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T20:45:51.730 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T20:45:51.730 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T20:45:51.730 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T20:45:51.730 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T20:45:51.730 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T20:45:51.730 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T20:45:51.730 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T20:45:51.730 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T20:45:51.730 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T20:45:51.730 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T20:45:51.730 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T20:45:51.730 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T20:45:51.730 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T20:45:51.730 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T20:45:51.731 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T20:45:51.731 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T20:45:51.731 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T20:45:51.731 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T20:45:51.731 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T20:45:51.731 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T20:45:51.731 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T20:45:51.731 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T20:45:51.731 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T20:45:51.731 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T20:45:51.731 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T20:45:51.731 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T20:45:51.731 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T20:45:51.731 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T20:45:51.731 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T20:45:51.732 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T20:45:51.732 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T20:45:51.732 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T20:45:51.732 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T20:45:51.732 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T20:45:51.732 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T20:45:51.732 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T20:45:51.732 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T20:45:51.732 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T20:45:51.732 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T20:45:51.732 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T20:45:51.732 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T20:45:51.732 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T20:45:51.732 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T20:45:51.732 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T20:45:51.733 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T20:45:51.733 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T20:45:51.733 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T20:45:51.733 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T20:45:51.733 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T20:45:51.733 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T20:45:51.733 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T20:45:51.733 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T20:45:51.733 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T20:45:51.733 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T20:45:51.733 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T20:45:51.733 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T20:45:51.733 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T20:45:51.733 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T20:45:51.733 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T20:45:51.733 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T20:45:51.734 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T20:45:51.734 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T20:45:51.734 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T20:45:51.734 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T20:45:51.734 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T20:45:51.734 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T20:45:51.734 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T20:45:51.734 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T20:45:51.734 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T20:45:51.734 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T20:45:51.734 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T20:45:51.734 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T20:45:51.734 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T20:45:51.734 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T20:45:51.734 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T20:45:51.735 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T20:45:51.735 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T20:45:51.735 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T20:45:51.735 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T20:45:51.735 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T20:45:51.735 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T20:45:51.735 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T20:45:51.735 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T20:45:51.735 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T20:45:51.735 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T20:45:51.735 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T20:45:51.735 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T20:45:51.735 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T20:45:51.735 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T20:45:51.735 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T20:45:51.736 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T20:45:51.736 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T20:45:51.736 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T20:45:51.736 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T20:45:51.736 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T20:45:51.736 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T20:45:51.736 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T20:45:51.736 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T20:45:51.736 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T20:45:51.736 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T20:45:51.736 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T20:45:51.736 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T20:45:51.736 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T20:45:51.736 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T20:45:51.737 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T20:45:51.737 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T20:45:51.737 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T20:45:51.737 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T20:45:51.737 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T20:45:51.737 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T20:45:51.737 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T20:45:51.737 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T20:45:51.737 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T20:45:51.737 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T20:45:51.737 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T20:45:51.737 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T20:45:51.737 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T20:45:51.737 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T20:45:51.737 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T20:45:51.737 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T20:45:51.738 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T20:45:51.738 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T20:45:51.738 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T20:45:51.738 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T20:45:51.738 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T20:45:51.738 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T20:45:51.738 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T20:45:51.738 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T20:45:51.738 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T20:45:51.738 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T20:45:51.738 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T20:45:51.738 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T20:45:51.738 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T20:45:51.738 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T20:45:51.738 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T20:45:51.738 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T20:45:51.739 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T20:45:51.739 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T20:45:51.739 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T20:45:51.739 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T20:45:51.739 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T20:45:51.739 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T20:45:51.739 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T20:45:51.739 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T20:45:51.739 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T20:45:51.739 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T20:45:51.739 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T20:45:51.739 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T20:45:51.739 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T20:45:51.740 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T20:45:51.740 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T20:45:51.740 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T20:45:51.740 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T20:45:51.740 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T20:45:51.740 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T20:45:51.740 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T20:45:51.740 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T20:45:51.740 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T20:45:51.740 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T20:45:51.740 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T20:45:51.740 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T20:45:51.740 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T20:45:51.740 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T20:45:51.740 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T20:45:51.741 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T20:45:51.741 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T20:45:51.741 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T20:45:51.741 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T20:45:51.741 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T20:45:51.741 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T20:45:51.741 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T20:45:51.741 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T20:45:51.741 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T20:45:51.741 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T20:45:51.741 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T20:45:51.741 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T20:45:51.741 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T20:45:51.742 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T20:45:51.742 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T20:45:51.742 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T20:45:51.742 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T20:45:51.742 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T20:45:51.742 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T20:45:51.742 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T20:45:51.742 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T20:45:51.742 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T20:45:51.742 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T20:45:51.742 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T20:45:51.742 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T20:45:51.742 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T20:45:51.742 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T20:45:51.743 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T20:45:51.743 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T20:45:51.743 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T20:45:51.743 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T20:45:51.743 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T20:45:51.743 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T20:45:51.743 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T20:45:51.743 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T20:45:51.743 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T20:45:51.743 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T20:45:51.743 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T20:45:51.743 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T20:45:51.743 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T20:45:51.743 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T20:45:51.743 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T20:45:51.744 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T20:45:51.744 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T20:45:51.744 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T20:45:51.744 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T20:45:51.744 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T20:45:51.744 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T20:45:51.744 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T20:45:51.744 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T20:45:51.744 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T20:45:51.744 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T20:45:51.744 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T20:45:51.744 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T20:45:51.744 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T20:45:51.745 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T20:45:51.745 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T20:45:51.745 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T20:45:51.745 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T20:45:51.745 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T20:45:51.745 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T20:45:51.745 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T20:45:51.745 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T20:45:51.745 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T20:45:51.745 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T20:45:51.745 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T20:45:51.745 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T20:45:51.745 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T20:45:51.745 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T20:45:51.745 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T20:45:51.746 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T20:45:51.746 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T20:45:51.746 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T20:45:51.746 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T20:45:51.746 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T20:45:51.746 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T20:45:51.746 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T20:45:51.746 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T20:45:51.746 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T20:45:51.746 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T20:45:51.746 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T20:45:51.746 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T20:45:51.746 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T20:45:51.747 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T20:45:51.747 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T20:45:51.747 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T20:45:51.747 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T20:45:51.747 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T20:45:51.747 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T20:45:51.747 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T20:45:51.747 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T20:45:51.747 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T20:45:51.747 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T20:45:51.747 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T20:45:51.747 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T20:45:51.747 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T20:45:51.748 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T20:45:51.748 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T20:45:51.748 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T20:45:51.748 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T20:45:51.748 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T20:45:51.748 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T20:45:51.748 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T20:45:51.748 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T20:45:51.748 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T20:45:51.748 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T20:45:51.748 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T20:45:51.748 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T20:45:51.748 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T20:45:51.749 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T20:45:51.749 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T20:45:51.749 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T20:45:51.749 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T20:45:51.749 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T20:45:51.749 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T20:45:51.749 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T20:45:51.749 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T20:45:51.749 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T20:45:51.749 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T20:45:51.749 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T20:45:51.749 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T20:45:51.749 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T20:45:51.749 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T20:45:51.750 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T20:45:51.750 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T20:45:51.750 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T20:45:51.750 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T20:45:51.750 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T20:45:51.750 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T20:45:51.750 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T20:45:51.750 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T20:45:51.750 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T20:45:51.750 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T20:45:51.750 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T20:45:51.750 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T20:45:51.751 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T20:45:51.751 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T20:45:51.751 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T20:45:51.751 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T20:45:51.751 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T20:45:51.751 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T20:45:51.751 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T20:45:51.751 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T20:45:51.751 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T20:45:51.751 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T20:45:51.751 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T20:45:51.751 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T20:45:51.751 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T20:45:51.751 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T20:45:51.752 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T20:45:51.752 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T20:45:51.752 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T20:45:51.752 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T20:45:51.752 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T20:45:51.752 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T20:45:51.752 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T20:45:51.752 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T20:45:51.752 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T20:45:51.752 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T20:45:51.752 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T20:45:51.752 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T20:45:51.752 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T20:45:51.752 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T20:45:51.753 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T20:45:51.753 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T20:45:51.753 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T20:45:51.753 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T20:45:51.753 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T20:45:51.753 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T20:45:51.753 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T20:45:51.753 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T20:45:51.753 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T20:45:51.753 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T20:45:51.753 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T20:45:51.753 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T20:45:51.753 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T20:45:51.753 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T20:45:51.754 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T20:45:51.754 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T20:45:51.754 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T20:45:51.754 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T20:45:51.754 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T20:45:51.754 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T20:45:51.754 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T20:45:51.754 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T20:45:51.754 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T20:45:51.754 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T20:45:51.754 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T20:45:51.754 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T20:45:51.754 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T20:45:51.754 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T20:45:51.755 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T20:45:51.755 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T20:45:51.755 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T20:45:51.755 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T20:45:51.755 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T20:45:51.755 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T20:45:51.755 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T20:45:51.755 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T20:45:51.755 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T20:45:51.755 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T20:45:51.755 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T20:45:51.755 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T20:45:51.755 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T20:45:51.755 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T20:45:51.755 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T20:45:51.756 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T20:45:51.756 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T20:45:51.756 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T20:45:51.756 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T20:45:51.756 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T20:45:51.756 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T20:45:51.756 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T20:45:51.756 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T20:45:51.756 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T20:45:51.756 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T20:45:51.756 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T20:45:51.756 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T20:45:51.756 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T20:45:51.756 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T20:45:51.757 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T20:45:51.757 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T20:45:51.757 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T20:45:51.757 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T20:45:51.757 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T20:45:51.757 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T20:45:51.757 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T20:45:51.757 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T20:45:51.757 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T20:45:51.757 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T20:45:51.757 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T20:45:51.757 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T20:45:51.757 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T20:45:51.757 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T20:45:51.758 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T20:45:51.758 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T20:45:51.758 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T20:45:51.758 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T20:45:51.758 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T20:45:51.758 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T20:45:51.758 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T20:45:51.758 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T20:45:51.758 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T20:45:51.758 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T20:45:51.758 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T20:45:51.758 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T20:45:51.758 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T20:45:51.759 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T20:45:51.759 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T20:45:51.759 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T20:45:51.759 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T20:45:51.759 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T20:45:51.759 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T20:45:51.759 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T20:45:51.759 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T20:45:51.759 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T20:45:51.759 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T20:45:51.759 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T20:45:51.759 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T20:45:51.759 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T20:45:51.759 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T20:45:51.760 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T20:45:51.760 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T20:45:51.760 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T20:45:51.760 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T20:45:51.760 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T20:45:51.760 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T20:45:51.760 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T20:45:51.760 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T20:45:51.760 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T20:45:51.760 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T20:45:51.760 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T20:45:51.760 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T20:45:51.760 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T20:45:51.760 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T20:45:51.760 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T20:45:51.761 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T20:45:51.761 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T20:45:51.761 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T20:45:51.761 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T20:45:51.761 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T20:45:51.761 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T20:45:51.761 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T20:45:51.761 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T20:45:51.761 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T20:45:51.761 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T20:45:51.761 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T20:45:51.761 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T20:45:51.761 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T20:45:51.762 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T20:45:51.762 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T20:45:51.762 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T20:45:51.762 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T20:45:51.762 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T20:45:51.762 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T20:45:51.762 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T20:45:51.762 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T20:45:51.762 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T20:45:51.762 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T20:45:51.762 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T20:45:51.762 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T20:45:51.762 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T20:45:51.763 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T20:45:51.763 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T20:45:51.763 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T20:45:51.763 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T20:45:51.763 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T20:45:51.763 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T20:45:51.763 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T20:45:51.763 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T20:45:51.763 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T20:45:51.763 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T20:45:51.763 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T20:45:51.763 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T20:45:51.763 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T20:45:51.763 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T20:45:51.763 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T20:45:51.764 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T20:45:51.764 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T20:45:51.764 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T20:45:51.764 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T20:45:51.764 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T20:45:51.764 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T20:45:51.764 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T20:45:51.764 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T20:45:51.764 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T20:45:51.764 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T20:45:51.764 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T20:45:51.764 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T20:45:51.764 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T20:45:51.764 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T20:45:51.764 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T20:45:51.765 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T20:45:51.765 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T20:45:51.765 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T20:45:51.765 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T20:45:51.765 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T20:45:51.765 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T20:45:51.765 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T20:45:51.765 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T20:45:51.765 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T20:45:51.765 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T20:45:51.765 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T20:45:51.765 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T20:45:51.765 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T20:45:51.765 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T20:45:51.765 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T20:45:51.766 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T20:45:51.766 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T20:45:51.766 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T20:45:51.766 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T20:45:51.766 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T20:45:51.766 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T20:45:51.766 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T20:45:51.766 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T20:45:51.766 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T20:45:51.766 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T20:45:51.766 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T20:45:51.766 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T20:45:51.766 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T20:45:51.766 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T20:45:51.767 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T20:45:51.767 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T20:45:51.767 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T20:45:51.767 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T20:45:51.767 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T20:45:51.767 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T20:45:51.767 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T20:45:51.767 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T20:45:51.767 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T20:45:51.767 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T20:45:51.767 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T20:45:51.767 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T20:45:51.767 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T20:45:51.767 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T20:45:51.768 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T20:45:51.768 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T20:45:51.768 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T20:45:51.768 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T20:45:51.768 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T20:45:51.768 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T20:45:51.768 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T20:45:51.768 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T20:45:51.768 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T20:45:51.768 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T20:45:51.768 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T20:45:51.768 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T20:45:51.768 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T20:45:51.769 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T20:45:51.769 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T20:45:51.769 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T20:45:51.769 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T20:45:51.769 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T20:45:51.769 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T20:45:51.769 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T20:45:51.769 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T20:45:51.769 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T20:45:51.769 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T20:45:51.769 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T20:45:51.769 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T20:45:51.769 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T20:45:51.770 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T20:45:51.770 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T20:45:51.770 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T20:45:51.770 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T20:45:51.770 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T20:45:51.770 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T20:45:51.770 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T20:45:51.770 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T20:45:51.770 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T20:45:51.770 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T20:45:51.770 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T20:45:51.770 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T20:45:51.770 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T20:45:51.770 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T20:45:51.770 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T20:45:51.771 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T20:45:51.771 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T20:45:51.771 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T20:45:51.771 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T20:45:51.771 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T20:45:51.771 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T20:45:51.771 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T20:45:51.771 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T20:45:51.771 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T20:45:51.771 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T20:45:51.771 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T20:45:51.771 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T20:45:51.771 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T20:45:51.771 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T20:45:51.771 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T20:45:51.772 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T20:45:51.772 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T20:45:51.772 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T20:45:51.772 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T20:45:51.772 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T20:45:51.772 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T20:45:51.772 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T20:45:51.772 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T20:45:51.772 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T20:45:51.772 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T20:45:51.772 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T20:45:51.772 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T20:45:51.772 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T20:45:51.772 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T20:45:51.772 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T20:45:51.773 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T20:45:51.773 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T20:45:51.773 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T20:45:51.773 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T20:45:51.773 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T20:45:51.773 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T20:45:51.773 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T20:45:51.773 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T20:45:51.773 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T20:45:51.773 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T20:45:51.773 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T20:45:51.773 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T20:45:51.773 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T20:45:51.773 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T20:45:51.773 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T20:45:51.773 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T20:45:51.774 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T20:45:51.774 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T20:45:51.774 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T20:45:51.774 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T20:45:51.774 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T20:45:51.774 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T20:45:51.774 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T20:45:51.774 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T20:45:51.774 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T20:45:51.774 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T20:45:51.774 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T20:45:51.774 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T20:45:51.774 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T20:45:51.774 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T20:45:51.775 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T20:45:51.775 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T20:45:51.775 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T20:45:51.775 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T20:45:51.775 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T20:45:51.775 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T20:45:51.775 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T20:45:51.775 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T20:45:51.775 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T20:45:51.775 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T20:45:51.775 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T20:45:51.775 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T20:45:51.775 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T20:45:51.775 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T20:45:51.776 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T20:45:51.776 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T20:45:51.776 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T20:45:51.776 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T20:45:51.776 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T20:45:51.776 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T20:45:51.776 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T20:45:51.776 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T20:45:51.776 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T20:45:51.776 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T20:45:51.776 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T20:45:51.776 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T20:45:51.776 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T20:45:51.776 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T20:45:51.776 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T20:45:51.776 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T20:45:51.777 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T20:45:51.777 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T20:45:51.777 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T20:45:51.777 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T20:45:51.777 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T20:45:51.777 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T20:45:51.777 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T20:45:51.777 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T20:45:51.777 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T20:45:51.777 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T20:45:51.777 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T20:45:51.777 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T20:45:51.777 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T20:45:51.777 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T20:45:51.777 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T20:45:51.777 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T20:45:51.778 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T20:45:51.778 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T20:45:51.778 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T20:45:51.778 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T20:45:51.778 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T20:45:51.778 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T20:45:51.778 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T20:45:51.778 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T20:45:51.778 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T20:45:51.778 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T20:45:51.778 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T20:45:51.778 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T20:45:51.778 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T20:45:51.778 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T20:45:51.778 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T20:45:51.779 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T20:45:51.779 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T20:45:51.779 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T20:45:51.779 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T20:45:51.779 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T20:45:51.779 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T20:45:51.779 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T20:45:51.779 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T20:45:51.779 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T20:45:51.779 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T20:45:51.779 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T20:45:51.779 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T20:45:51.779 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T20:45:51.779 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T20:45:51.779 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T20:45:51.779 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T20:45:51.780 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T20:45:51.780 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T20:45:51.780 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T20:45:51.780 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T20:45:51.780 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T20:45:51.780 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T20:45:51.780 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T20:45:51.780 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T20:45:51.780 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T20:45:51.780 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T20:45:51.780 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T20:45:51.780 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T20:45:51.780 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T20:45:51.780 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T20:45:51.780 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T20:45:51.780 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T20:45:51.781 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T20:45:51.781 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T20:45:51.781 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T20:45:51.781 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T20:45:51.781 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T20:45:51.781 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T20:45:51.781 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T20:45:51.781 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T20:45:51.781 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T20:45:51.781 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T20:45:51.781 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T20:45:51.781 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T20:45:51.781 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T20:45:51.781 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T20:45:51.781 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T20:45:51.781 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T20:45:51.781 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T20:45:51.782 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T20:45:51.782 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T20:45:51.782 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T20:45:51.782 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T20:45:51.782 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T20:45:51.782 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T20:45:51.782 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T20:45:51.782 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T20:45:51.782 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T20:45:51.782 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T20:45:51.782 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T20:45:51.782 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T20:45:51.782 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T20:45:51.782 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T20:45:51.782 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T20:45:51.783 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T20:45:51.783 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T20:45:51.783 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T20:45:51.783 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T20:45:51.783 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T20:45:51.783 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T20:45:51.783 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T20:45:51.783 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T20:45:51.783 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T20:45:51.783 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T20:45:51.783 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T20:45:51.783 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T20:45:51.783 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T20:45:51.783 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T20:45:51.783 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T20:45:51.783 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T20:45:51.783 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T20:45:51.784 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T20:45:51.784 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T20:45:51.784 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T20:45:51.784 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T20:45:51.784 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T20:45:51.784 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T20:45:51.784 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T20:45:51.784 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T20:45:51.784 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T20:45:51.784 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T20:45:51.784 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T20:45:51.784 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T20:45:51.784 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T20:45:51.784 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T20:45:51.784 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T20:45:51.784 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T20:45:51.784 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T20:45:51.785 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T20:45:51.785 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T20:45:51.785 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T20:45:51.785 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T20:45:51.785 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T20:45:51.785 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T20:45:51.785 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T20:45:51.785 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T20:45:51.785 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T20:45:51.785 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T20:45:51.785 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T20:45:51.785 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T20:45:51.785 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T20:45:51.785 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T20:45:51.785 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T20:45:51.785 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T20:45:51.786 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T20:45:51.786 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T20:45:51.786 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T20:45:51.786 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T20:45:51.786 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T20:45:51.786 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T20:45:51.786 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T20:45:51.786 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T20:45:51.786 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T20:45:51.786 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T20:45:51.786 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T20:45:51.786 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T20:45:51.786 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T20:45:51.786 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T20:45:51.786 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T20:45:51.786 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T20:45:51.787 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T20:45:51.787 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T20:45:51.787 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T20:45:51.787 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T20:45:51.787 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T20:45:51.787 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T20:45:51.787 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T20:45:51.787 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T20:45:51.787 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T20:45:51.787 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T20:45:51.787 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T20:45:51.787 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T20:45:51.787 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T20:45:51.787 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T20:45:51.787 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T20:45:51.788 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T20:45:51.788 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T20:45:51.788 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T20:45:51.788 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T20:45:51.788 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T20:45:51.788 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T20:45:51.788 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T20:45:51.788 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T20:45:51.788 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T20:45:51.788 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T20:45:51.788 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T20:45:51.788 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T20:45:51.788 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T20:45:51.788 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T20:45:51.788 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T20:45:51.789 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T20:45:51.789 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T20:45:51.789 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T20:45:51.789 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T20:45:51.789 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T20:45:51.789 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T20:45:51.789 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T20:45:51.789 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T20:45:51.789 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T20:45:51.789 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T20:45:51.789 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T20:45:51.789 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T20:45:51.789 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T20:45:51.789 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T20:45:51.790 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T20:45:51.790 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T20:45:51.790 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T20:45:51.790 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T20:45:51.790 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T20:45:51.790 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T20:45:51.790 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T20:45:51.790 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T20:45:51.790 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T20:45:51.790 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T20:45:51.790 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T20:45:51.790 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T20:45:51.790 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T20:45:51.790 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T20:45:51.790 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T20:45:51.791 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T20:45:51.791 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T20:45:51.791 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T20:45:51.791 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T20:45:51.791 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T20:45:51.791 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T20:45:51.791 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T20:45:51.791 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T20:45:51.791 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T20:45:51.791 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T20:45:51.791 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T20:45:51.791 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T20:45:51.791 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T20:45:51.792 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T20:45:51.792 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T20:45:51.792 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T20:45:51.792 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T20:45:51.792 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T20:45:51.792 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T20:45:51.792 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T20:45:51.792 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T20:45:51.792 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T20:45:51.792 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T20:45:51.792 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T20:45:51.792 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T20:45:51.792 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T20:45:51.793 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T20:45:51.793 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T20:45:51.793 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T20:45:51.793 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T20:45:51.793 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T20:45:51.793 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T20:45:51.793 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T20:45:51.793 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T20:45:51.793 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T20:45:51.793 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T20:45:51.793 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T20:45:51.793 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T20:45:51.793 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T20:45:51.793 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T20:45:51.794 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T20:45:51.794 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T20:45:51.794 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T20:45:51.794 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T20:45:51.794 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T20:45:51.794 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T20:45:51.794 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T20:45:51.794 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T20:45:51.794 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T20:45:51.794 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T20:45:51.794 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T20:45:51.794 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T20:45:51.794 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T20:45:51.794 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T20:45:51.795 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T20:45:51.795 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T20:45:51.795 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T20:45:51.795 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T20:45:51.795 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T20:45:51.795 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T20:45:51.795 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T20:45:51.795 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T20:45:51.795 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T20:45:51.795 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T20:45:51.795 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T20:45:51.795 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T20:45:51.795 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T20:45:51.795 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T20:45:51.796 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T20:45:51.796 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T20:45:51.796 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T20:45:51.796 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T20:45:51.796 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T20:45:51.796 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T20:45:51.796 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T20:45:51.796 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T20:45:51.796 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T20:45:51.796 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T20:45:51.796 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T20:45:51.796 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T20:45:51.796 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T20:45:51.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T20:45:51.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T20:45:51.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T20:45:51.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T20:45:51.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T20:45:51.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T20:45:51.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T20:45:51.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T20:45:51.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T20:45:51.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T20:45:51.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T20:45:51.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T20:45:51.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T20:45:51.797 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T20:45:51.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T20:45:51.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T20:45:51.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T20:45:51.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T20:45:51.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T20:45:51.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T20:45:51.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T20:45:51.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T20:45:51.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T20:45:51.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T20:45:51.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T20:45:51.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T20:45:51.798 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T20:45:51.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T20:45:51.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T20:45:51.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T20:45:51.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T20:45:51.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T20:45:51.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T20:45:51.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T20:45:51.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T20:45:51.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T20:45:51.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T20:45:51.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T20:45:51.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T20:45:51.799 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T20:45:51.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T20:45:51.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T20:45:51.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T20:45:51.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T20:45:51.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T20:45:51.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T20:45:51.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T20:45:51.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T20:45:51.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T20:45:51.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T20:45:51.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T20:45:51.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T20:45:51.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T20:45:51.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T20:45:51.800 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T20:45:51.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T20:45:51.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T20:45:51.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T20:45:51.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T20:45:51.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T20:45:51.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T20:45:51.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T20:45:51.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T20:45:51.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T20:45:51.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T20:45:51.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T20:45:51.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T20:45:51.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T20:45:51.801 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T20:45:51.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T20:45:51.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T20:45:51.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T20:45:51.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T20:45:51.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T20:45:51.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T20:45:51.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T20:45:51.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T20:45:51.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T20:45:51.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T20:45:51.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T20:45:51.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T20:45:51.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T20:45:51.802 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T20:45:51.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T20:45:51.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T20:45:51.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T20:45:51.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T20:45:51.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T20:45:51.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T20:45:51.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T20:45:51.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T20:45:51.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T20:45:51.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T20:45:51.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T20:45:51.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T20:45:51.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T20:45:51.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T20:45:51.803 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T20:45:51.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T20:45:51.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T20:45:51.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T20:45:51.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T20:45:51.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T20:45:51.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T20:45:51.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T20:45:51.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T20:45:51.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T20:45:51.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T20:45:51.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T20:45:51.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T20:45:51.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T20:45:51.804 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T20:45:51.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T20:45:51.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T20:45:51.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T20:45:51.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T20:45:51.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T20:45:51.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T20:45:51.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T20:45:51.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T20:45:51.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T20:45:51.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T20:45:51.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T20:45:51.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T20:45:51.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T20:45:51.805 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T20:45:51.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T20:45:51.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T20:45:51.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T20:45:51.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T20:45:51.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T20:45:51.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T20:45:51.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T20:45:51.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T20:45:51.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T20:45:51.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T20:45:51.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T20:45:51.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T20:45:51.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T20:45:51.806 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T20:45:51.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T20:45:51.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T20:45:51.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T20:45:51.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T20:45:51.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T20:45:51.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T20:45:51.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T20:45:51.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T20:45:51.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T20:45:51.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T20:45:51.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T20:45:51.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T20:45:51.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T20:45:51.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T20:45:51.807 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T20:45:51.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T20:45:51.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T20:45:51.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T20:45:51.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T20:45:51.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T20:45:51.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T20:45:51.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T20:45:51.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T20:45:51.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T20:45:51.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T20:45:51.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T20:45:51.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T20:45:51.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T20:45:51.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T20:45:51.808 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T20:45:51.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T20:45:51.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T20:45:51.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T20:45:51.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T20:45:51.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T20:45:51.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T20:45:51.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T20:45:51.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T20:45:51.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T20:45:51.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T20:45:51.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T20:45:51.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T20:45:51.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T20:45:51.809 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T20:45:51.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T20:45:51.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T20:45:51.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T20:45:51.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T20:45:51.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T20:45:51.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T20:45:51.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T20:45:51.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T20:45:51.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T20:45:51.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T20:45:51.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T20:45:51.810 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T20:45:51.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T20:45:51.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T20:45:51.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T20:45:51.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T20:45:51.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T20:45:51.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T20:45:51.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T20:45:51.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T20:45:51.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T20:45:51.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T20:45:51.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T20:45:51.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T20:45:51.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T20:45:51.811 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T20:45:51.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T20:45:51.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T20:45:51.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T20:45:51.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T20:45:51.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T20:45:51.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T20:45:51.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T20:45:51.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T20:45:51.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T20:45:51.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T20:45:51.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T20:45:51.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T20:45:51.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T20:45:51.812 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T20:45:51.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T20:45:51.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T20:45:51.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T20:45:51.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T20:45:51.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T20:45:51.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T20:45:51.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T20:45:51.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T20:45:51.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T20:45:51.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T20:45:51.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T20:45:51.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T20:45:51.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T20:45:51.813 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T20:45:51.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T20:45:51.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T20:45:51.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T20:45:51.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T20:45:51.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T20:45:51.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T20:45:51.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T20:45:51.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T20:45:51.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T20:45:51.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T20:45:51.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T20:45:51.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T20:45:51.814 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T20:45:51.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T20:45:51.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T20:45:51.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T20:45:51.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T20:45:51.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T20:45:51.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T20:45:51.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T20:45:51.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T20:45:51.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T20:45:51.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T20:45:51.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T20:45:51.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T20:45:51.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T20:45:51.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T20:45:51.815 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T20:45:51.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T20:45:51.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T20:45:51.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T20:45:51.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T20:45:51.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T20:45:51.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T20:45:51.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T20:45:51.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T20:45:51.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T20:45:51.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T20:45:51.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T20:45:51.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T20:45:51.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T20:45:51.816 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T20:45:51.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T20:45:51.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T20:45:51.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T20:45:51.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T20:45:51.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T20:45:51.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T20:45:51.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T20:45:51.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T20:45:51.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T20:45:51.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T20:45:51.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T20:45:51.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T20:45:51.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T20:45:51.817 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T20:45:51.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T20:45:51.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T20:45:51.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T20:45:51.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T20:45:51.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T20:45:51.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T20:45:51.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T20:45:51.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T20:45:51.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T20:45:51.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T20:45:51.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T20:45:51.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T20:45:51.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T20:45:51.818 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T20:45:51.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T20:45:51.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T20:45:51.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T20:45:51.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T20:45:51.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T20:45:51.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T20:45:51.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T20:45:51.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T20:45:51.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T20:45:51.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T20:45:51.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T20:45:51.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T20:45:51.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T20:45:51.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T20:45:51.819 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T20:45:51.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T20:45:51.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T20:45:51.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T20:45:51.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T20:45:51.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T20:45:51.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T20:45:51.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T20:45:51.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T20:45:51.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T20:45:51.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T20:45:51.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T20:45:51.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T20:45:51.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T20:45:51.820 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T20:45:51.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T20:45:51.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T20:45:51.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T20:45:51.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T20:45:51.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T20:45:51.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T20:45:51.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T20:45:51.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T20:45:51.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T20:45:51.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T20:45:51.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T20:45:51.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T20:45:51.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T20:45:51.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T20:45:51.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T20:45:51.821 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T20:45:51.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T20:45:51.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T20:45:51.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T20:45:51.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T20:45:51.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T20:45:51.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T20:45:51.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T20:45:51.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T20:45:51.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T20:45:51.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T20:45:51.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T20:45:51.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T20:45:51.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T20:45:51.822 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T20:45:51.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T20:45:51.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T20:45:51.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T20:45:51.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T20:45:51.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T20:45:51.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T20:45:51.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T20:45:51.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T20:45:51.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T20:45:51.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T20:45:51.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T20:45:51.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T20:45:51.823 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T20:45:51.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T20:45:51.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T20:45:51.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T20:45:51.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T20:45:51.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T20:45:51.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T20:45:51.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T20:45:51.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T20:45:51.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T20:45:51.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T20:45:51.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T20:45:51.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T20:45:51.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T20:45:51.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T20:45:51.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T20:45:51.824 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T20:45:51.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T20:45:51.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T20:45:51.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T20:45:51.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T20:45:51.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T20:45:51.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T20:45:51.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T20:45:51.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T20:45:51.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T20:45:51.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T20:45:51.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T20:45:51.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T20:45:51.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T20:45:51.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T20:45:51.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T20:45:51.825 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T20:45:51.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T20:45:51.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T20:45:51.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T20:45:51.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T20:45:51.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T20:45:51.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T20:45:51.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T20:45:51.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T20:45:51.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T20:45:51.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T20:45:51.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T20:45:51.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T20:45:51.826 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T20:45:51.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T20:45:51.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T20:45:51.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T20:45:51.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T20:45:51.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T20:45:51.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T20:45:51.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T20:45:51.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T20:45:51.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T20:45:51.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T20:45:51.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T20:45:51.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T20:45:51.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T20:45:51.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T20:45:51.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T20:45:51.827 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T20:45:51.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T20:45:51.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T20:45:51.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T20:45:51.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T20:45:51.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T20:45:51.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T20:45:51.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T20:45:51.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T20:45:51.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T20:45:51.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T20:45:51.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T20:45:51.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T20:45:51.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T20:45:51.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T20:45:51.828 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T20:45:51.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T20:45:51.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T20:45:51.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T20:45:51.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T20:45:51.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T20:45:51.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T20:45:51.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T20:45:51.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T20:45:51.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T20:45:51.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T20:45:51.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T20:45:51.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T20:45:51.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T20:45:51.829 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T20:45:51.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T20:45:51.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T20:45:51.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T20:45:51.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T20:45:51.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T20:45:51.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T20:45:51.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T20:45:51.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T20:45:51.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T20:45:51.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T20:45:51.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T20:45:51.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T20:45:51.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T20:45:51.830 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T20:45:51.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T20:45:51.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T20:45:51.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T20:45:51.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T20:45:51.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T20:45:51.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T20:45:51.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T20:45:51.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T20:45:51.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T20:45:51.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T20:45:51.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T20:45:51.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T20:45:51.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T20:45:51.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T20:45:51.831 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T20:45:51.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T20:45:51.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T20:45:51.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T20:45:51.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T20:45:51.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T20:45:51.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T20:45:51.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T20:45:51.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T20:45:51.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T20:45:51.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T20:45:51.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T20:45:51.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T20:45:51.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T20:45:51.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T20:45:51.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T20:45:51.832 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T20:45:51.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T20:45:51.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T20:45:51.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T20:45:51.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T20:45:51.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T20:45:51.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T20:45:51.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T20:45:51.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T20:45:51.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T20:45:51.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T20:45:51.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T20:45:51.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T20:45:51.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T20:45:51.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T20:45:51.833 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T20:45:51.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T20:45:51.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T20:45:51.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T20:45:51.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T20:45:51.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T20:45:51.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T20:45:51.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T20:45:51.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T20:45:51.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T20:45:51.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T20:45:51.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T20:45:51.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T20:45:51.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T20:45:51.834 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T20:45:51.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T20:45:51.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T20:45:51.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T20:45:51.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T20:45:51.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T20:45:51.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T20:45:51.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T20:45:51.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T20:45:51.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T20:45:51.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T20:45:51.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T20:45:51.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T20:45:51.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T20:45:51.835 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T20:45:51.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T20:45:51.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T20:45:51.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T20:45:51.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T20:45:51.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T20:45:51.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T20:45:51.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T20:45:51.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T20:45:51.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T20:45:51.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T20:45:51.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T20:45:51.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T20:45:51.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T20:45:51.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T20:45:51.836 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T20:45:51.837 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T20:45:51.837 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T20:45:51.837 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T20:45:51.837 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T20:45:51.837 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T20:45:51.837 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T20:45:51.837 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T20:45:51.837 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T20:45:51.837 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T20:45:51.837 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T20:45:51.837 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T20:45:51.837 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T20:45:51.837 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T20:45:51.837 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T20:45:51.838 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T20:45:51.838 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T20:45:51.838 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T20:45:51.838 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T20:45:51.838 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T20:45:51.838 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T20:45:51.838 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T20:45:51.838 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T20:45:51.838 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T20:45:51.838 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T20:45:51.838 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T20:45:51.838 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T20:45:51.838 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T20:45:51.838 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T20:45:51.838 INFO:teuthology.orchestra.run.smithi103.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T20:45:51.842 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T20:45:51.842 DEBUG:teuthology.orchestra.run.smithi103:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T20:45:51.865 INFO:teuthology.orchestra.run.smithi103.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T20:45:51.865 INFO:teuthology.orchestra.run.smithi103.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T20:45:51.865 INFO:teuthology.orchestra.run.smithi103.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T20:45:51.865 INFO:teuthology.orchestra.run.smithi103.stdout:fi 2024-04-26T20:45:51.865 INFO:teuthology.orchestra.run.smithi103.stdout:for file in vmlinuz System.map config; do 2024-04-26T20:45:51.865 INFO:teuthology.orchestra.run.smithi103.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T20:45:51.865 INFO:teuthology.orchestra.run.smithi103.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:45:51.865 INFO:teuthology.orchestra.run.smithi103.stdout: fi 2024-04-26T20:45:51.865 INFO:teuthology.orchestra.run.smithi103.stdout:done 2024-04-26T20:45:51.865 INFO:teuthology.orchestra.run.smithi103.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T20:45:51.865 INFO:teuthology.orchestra.run.smithi103.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T20:45:51.865 INFO:teuthology.orchestra.run.smithi103.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:45:51.865 INFO:teuthology.orchestra.run.smithi103.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T20:45:51.866 INFO:teuthology.orchestra.run.smithi103.stdout:fi 2024-04-26T20:45:51.866 INFO:teuthology.orchestra.run.smithi103.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T20:45:51.866 INFO:teuthology.orchestra.run.smithi103.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T20:45:51.866 INFO:teuthology.orchestra.run.smithi103.stdout:fi 2024-04-26T20:45:51.866 INFO:teuthology.orchestra.run.smithi103.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T20:45:51.866 INFO:teuthology.orchestra.run.smithi103.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T20:45:51.866 INFO:teuthology.orchestra.run.smithi103.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T20:45:51.866 INFO:teuthology.orchestra.run.smithi103.stdout:fi 2024-04-26T20:45:51.867 DEBUG:teuthology.orchestra.run.smithi103:> sudo rpm -qi grub2-tools 2024-04-26T20:45:51.950 INFO:teuthology.orchestra.run.smithi103.stdout:Name : grub2-tools 2024-04-26T20:45:51.950 INFO:teuthology.orchestra.run.smithi103.stdout:Epoch : 1 2024-04-26T20:45:51.951 INFO:teuthology.orchestra.run.smithi103.stdout:Version : 2.06 2024-04-26T20:45:51.951 INFO:teuthology.orchestra.run.smithi103.stdout:Release : 68.el9 2024-04-26T20:45:51.951 INFO:teuthology.orchestra.run.smithi103.stdout:Architecture: x86_64 2024-04-26T20:45:51.951 INFO:teuthology.orchestra.run.smithi103.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T20:45:51.951 INFO:teuthology.orchestra.run.smithi103.stdout:Group : Unspecified 2024-04-26T20:45:51.951 INFO:teuthology.orchestra.run.smithi103.stdout:Size : 8264254 2024-04-26T20:45:51.951 INFO:teuthology.orchestra.run.smithi103.stdout:License : GPLv3+ 2024-04-26T20:45:51.951 INFO:teuthology.orchestra.run.smithi103.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T20:45:51.951 INFO:teuthology.orchestra.run.smithi103.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T20:45:51.951 INFO:teuthology.orchestra.run.smithi103.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T20:45:51.951 INFO:teuthology.orchestra.run.smithi103.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T20:45:51.951 INFO:teuthology.orchestra.run.smithi103.stdout:Packager : builder@centos.org 2024-04-26T20:45:51.951 INFO:teuthology.orchestra.run.smithi103.stdout:Vendor : CentOS 2024-04-26T20:45:51.952 INFO:teuthology.orchestra.run.smithi103.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T20:45:51.952 INFO:teuthology.orchestra.run.smithi103.stdout:Summary : Support tools for GRUB. 2024-04-26T20:45:51.952 INFO:teuthology.orchestra.run.smithi103.stdout:Description : 2024-04-26T20:45:51.952 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:45:51.952 INFO:teuthology.orchestra.run.smithi103.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T20:45:51.952 INFO:teuthology.orchestra.run.smithi103.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T20:45:51.952 INFO:teuthology.orchestra.run.smithi103.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T20:45:51.952 INFO:teuthology.orchestra.run.smithi103.stdout:hardware devices. 2024-04-26T20:45:51.952 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:45:51.952 INFO:teuthology.orchestra.run.smithi103.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T20:45:51.953 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T20:45:51.953 INFO:teuthology.task.kernel:Updating grub on smithi103 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T20:45:51.953 DEBUG:teuthology.orchestra.run.smithi103:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T20:45:52.545 INFO:teuthology.orchestra.run.smithi103.stderr:Generating grub configuration file ... 2024-04-26T20:45:53.908 INFO:teuthology.orchestra.run.smithi103.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T20:45:53.938 INFO:teuthology.orchestra.run.smithi103.stderr:done 2024-04-26T20:45:53.943 DEBUG:teuthology.orchestra.run.smithi103:> mktemp 2024-04-26T20:45:53.958 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/tmp.3ljTSa2ojO 2024-04-26T20:45:53.958 DEBUG:teuthology.orchestra.run.smithi103:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.3ljTSa2ojO 2024-04-26T20:45:54.040 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0666 /tmp/tmp.3ljTSa2ojO 2024-04-26T20:45:54.182 DEBUG:teuthology.orchestra.remote:smithi103:/tmp/tmp.3ljTSa2ojO is 6KB 2024-04-26T20:45:54.230 DEBUG:teuthology.orchestra.run.smithi103:> rm -fr /tmp/tmp.3ljTSa2ojO 2024-04-26T20:45:54.244 DEBUG:teuthology.orchestra.run.smithi103:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T20:45:54.306 INFO:teuthology.orchestra.run.smithi103.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T20:45:54.306 INFO:teuthology.orchestra.run.smithi103.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T20:45:54.306 INFO:teuthology.orchestra.run.smithi103.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T20:45:54.306 INFO:teuthology.orchestra.run.smithi103.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T20:45:54.307 DEBUG:teuthology.orchestra.run.smithi103:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T20:45:54.463 DEBUG:teuthology.orchestra.run.smithi103:> sudo shutdown -r now 2024-04-26T20:46:18.678 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-26T20:46:18.679 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-04-26T20:46:18.680 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:46:24.471 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-26T20:46:24.471 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-04-26T20:46:24.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:46:59.104 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.103 2024-04-26T20:47:08.114 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-04-26T20:47:08.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:47:18.692 DEBUG:teuthology.orchestra.remote:timed out 2024-04-26T20:47:27.702 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-04-26T20:47:27.703 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:47:28.070 DEBUG:teuthology.orchestra.run.smithi028:> true 2024-04-26T20:47:28.711 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-04-26T20:47:28.711 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T20:47:28.711 DEBUG:teuthology.orchestra.run.smithi028:> uname -r 2024-04-26T20:47:28.757 INFO:teuthology.orchestra.run.smithi028.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T20:47:28.757 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T20:47:28.757 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T20:47:28.757 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T20:47:29.758 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-26T20:47:29.758 DEBUG:teuthology.orchestra.run.smithi028:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T20:47:29.895 INFO:teuthology.orchestra.run.smithi028.stdout:ttyS1 2024-04-26T20:47:29.948 DEBUG:teuthology.parallel:result is None 2024-04-26T20:48:08.118 DEBUG:teuthology.orchestra.remote:timed out 2024-04-26T20:48:20.119 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-04-26T20:48:20.120 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:48:20.521 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-04-26T20:48:21.162 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-04-26T20:48:21.162 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T20:48:21.162 DEBUG:teuthology.orchestra.run.smithi103:> uname -r 2024-04-26T20:48:21.216 INFO:teuthology.orchestra.run.smithi103.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T20:48:21.216 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T20:48:21.216 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T20:48:21.216 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T20:48:22.218 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-26T20:48:22.218 DEBUG:teuthology.orchestra.run.smithi103:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T20:48:22.367 INFO:teuthology.orchestra.run.smithi103.stdout:ttyS1 2024-04-26T20:48:22.425 DEBUG:teuthology.parallel:result is None 2024-04-26T20:48:22.426 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T20:48:22.435 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T20:48:22.435 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T20:48:22.443 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T20:48:22.501 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T20:48:22.513 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T20:48:22.543 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T20:48:22.543 DEBUG:teuthology.orchestra.run.smithi028:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T20:48:22.551 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T20:48:22.619 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T20:48:22.631 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T20:48:22.631 DEBUG:teuthology.orchestra.run.smithi028:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T20:48:22.639 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T20:48:22.726 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:48:22.728 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:48:22.758 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:48:22.758 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T20:48:22.763 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T20:48:22.774 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T20:48:22.775 DEBUG:teuthology.orchestra.run.smithi028:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T20:48:22.780 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T20:48:22.867 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T20:48:22.877 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T20:48:22.877 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T20:48:22.883 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T20:48:22.937 DEBUG:teuthology.orchestra.run.smithi028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:48:23.025 DEBUG:teuthology.orchestra.run.smithi028:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:48:23.128 DEBUG:teuthology.orchestra.run.smithi028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:48:23.215 DEBUG:teuthology.orchestra.run.smithi028:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:48:23.305 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:48:23.305 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T20:48:23.395 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:48:23.480 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T20:48:23.587 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:48:23.669 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T20:48:23.752 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:48:23.752 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T20:48:23.829 DEBUG:teuthology.orchestra.run.smithi028:> sudo service rsyslog restart 2024-04-26T20:48:23.837 DEBUG:teuthology.orchestra.run.smithi103:> sudo service rsyslog restart 2024-04-26T20:48:23.966 INFO:teuthology.orchestra.run.smithi028.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:48:23.999 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T20:48:24.438 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T20:48:24.449 INFO:teuthology.task.internal:Starting timer... 2024-04-26T20:48:24.449 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T20:48:24.465 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T20:48:24.480 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-26T20:48:24.480 DEBUG:teuthology.orchestra.run.smithi028:> sudo service auditd rotate 2024-04-26T20:48:24.603 INFO:teuthology.orchestra.run.smithi028.stdout:Rotating logs: 2024-04-26T20:48:24.607 DEBUG:teuthology.orchestra.run.smithi103:> sudo service auditd rotate 2024-04-26T20:48:24.716 INFO:teuthology.orchestra.run.smithi103.stdout:Rotating logs: 2024-04-26T20:48:24.720 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T20:48:24.720 DEBUG:teuthology.orchestra.run.smithi028:> /usr/sbin/getenforce 2024-04-26T20:48:24.794 INFO:teuthology.orchestra.run.smithi028.stdout:Permissive 2024-04-26T20:48:24.794 DEBUG:teuthology.orchestra.run.smithi103:> /usr/sbin/getenforce 2024-04-26T20:48:24.849 INFO:teuthology.orchestra.run.smithi103.stdout:Permissive 2024-04-26T20:48:24.849 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi028.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi103.front.sepia.ceph.com': 'permissive'} 2024-04-26T20:48:24.849 DEBUG:teuthology.orchestra.run.smithi028:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:48:24.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:48:24.917 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T20:48:24.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:48:24.983 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T20:48:24.983 DEBUG:teuthology.orchestra.run.smithi028:> sudo /usr/sbin/setenforce permissive 2024-04-26T20:48:25.072 DEBUG:teuthology.orchestra.run.smithi103:> sudo /usr/sbin/setenforce permissive 2024-04-26T20:48:25.146 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T20:48:25.157 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-26T20:48:25.162 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-26T20:48:25.293 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T20:48:25.310 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T20:48:25.312 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi028.front.sepia.ceph.com,smithi103.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T20:53:38.958 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi028.front.sepia.ceph.com'), Remote(name='ubuntu@smithi103.front.sepia.ceph.com')] 2024-04-26T20:53:38.959 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-04-26T20:53:38.961 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:53:39.068 DEBUG:teuthology.orchestra.run.smithi028:> true 2024-04-26T20:53:39.227 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi028.front.sepia.ceph.com' 2024-04-26T20:53:39.228 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-04-26T20:53:39.228 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T20:53:39.340 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-04-26T20:53:39.499 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-04-26T20:53:39.500 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T20:53:39.512 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T20:53:39.512 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T20:53:39.512 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:53:39.520 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T20:53:39.520 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T20:53:39.609 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T20:53:39.611 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T20:53:39.652 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T20:53:39.653 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T20:53:39.724 INFO:teuthology.orchestra.run.smithi028.stderr:sudo: ntpd: command not found 2024-04-26T20:53:39.725 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: ntpd: command not found 2024-04-26T20:53:39.756 INFO:teuthology.orchestra.run.smithi028.stdout:506 Cannot talk to daemon 2024-04-26T20:53:39.758 INFO:teuthology.orchestra.run.smithi103.stdout:506 Cannot talk to daemon 2024-04-26T20:53:39.794 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T20:53:39.798 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T20:53:39.838 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T20:53:39.838 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T20:53:39.975 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:53:39.982 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:53:39.982 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-04-26T20:53:39.982 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:53:39.982 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:53:39.982 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:53:39.982 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:53:39.988 INFO:teuthology.orchestra.run.smithi028.stderr:bash: line 1: ntpq: command not found 2024-04-26T20:53:39.996 INFO:teuthology.orchestra.run.smithi028.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T20:53:39.996 INFO:teuthology.orchestra.run.smithi028.stdout:=============================================================================== 2024-04-26T20:53:39.997 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:53:39.997 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:53:39.997 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:53:39.997 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T20:53:39.998 INFO:teuthology.run_tasks:Running task install... 2024-04-26T20:53:40.006 DEBUG:teuthology.task.install:project ceph 2024-04-26T20:53:40.006 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T20:53:40.007 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T20:53:40.007 INFO:teuthology.task.install:Using flavor: default 2024-04-26T20:53:40.012 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T20:53:40.012 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T20:53:40.013 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T20:53:40.013 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:53:40.015 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T20:53:40.015 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:53:40.237 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T20:53:40.238 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T20:53:40.252 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T20:53:40.252 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T20:53:40.369 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T20:53:40.370 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:53:40.370 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T20:53:40.442 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T20:53:40.443 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:53:40.443 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T20:53:40.452 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T20:53:40.452 DEBUG:teuthology.orchestra.run.smithi103:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T20:53:40.540 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T20:53:40.540 DEBUG:teuthology.orchestra.run.smithi028:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T20:53:40.582 DEBUG:teuthology.orchestra.run.smithi103:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T20:53:40.678 DEBUG:teuthology.orchestra.run.smithi028:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T20:53:40.693 DEBUG:teuthology.orchestra.run.smithi103:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T20:53:40.772 INFO:teuthology.orchestra.run.smithi103.stdout:check_obsoletes = 1 2024-04-26T20:53:40.776 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum clean all 2024-04-26T20:53:40.803 DEBUG:teuthology.orchestra.run.smithi028:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T20:53:40.887 INFO:teuthology.orchestra.run.smithi028.stdout:check_obsoletes = 1 2024-04-26T20:53:40.892 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum clean all 2024-04-26T20:53:41.103 INFO:teuthology.orchestra.run.smithi103.stdout:48 files removed 2024-04-26T20:53:41.142 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T20:53:41.226 INFO:teuthology.orchestra.run.smithi028.stdout:48 files removed 2024-04-26T20:53:41.266 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T20:53:41.968 INFO:teuthology.orchestra.run.smithi103.stdout:ceph packages for x86_64 214 kB/s | 84 kB 00:00 2024-04-26T20:53:42.176 INFO:teuthology.orchestra.run.smithi028.stdout:ceph packages for x86_64 180 kB/s | 84 kB 00:00 2024-04-26T20:53:42.485 INFO:teuthology.orchestra.run.smithi103.stdout:ceph noarch packages 28 kB/s | 12 kB 00:00 2024-04-26T20:53:42.751 INFO:teuthology.orchestra.run.smithi028.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-04-26T20:53:42.994 INFO:teuthology.orchestra.run.smithi103.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-04-26T20:53:43.218 INFO:teuthology.orchestra.run.smithi028.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-26T20:53:43.875 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-26T20:53:46.106 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-04-26T20:53:51.441 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 6.2 MB 00:01 2024-04-26T20:53:51.609 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 962 kB/s | 8.0 MB 00:08 2024-04-26T20:53:53.053 INFO:teuthology.orchestra.run.smithi028.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 16 kB 00:00 2024-04-26T20:53:54.047 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-26T20:53:56.837 INFO:teuthology.orchestra.run.smithi028.stdout:Extra Packages for Enterprise Linux 5.8 MB/s | 21 MB 00:03 2024-04-26T20:53:58.959 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 6.2 MB 00:01 2024-04-26T20:54:01.545 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-04-26T20:54:03.062 INFO:teuthology.orchestra.run.smithi028.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-26T20:54:04.184 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 8.5 MB/s | 21 MB 00:02 2024-04-26T20:54:05.527 INFO:teuthology.orchestra.run.smithi028.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:54:05.528 INFO:teuthology.orchestra.run.smithi028.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:54:05.548 INFO:teuthology.orchestra.run.smithi028.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T20:54:05.549 INFO:teuthology.orchestra.run.smithi028.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T20:54:05.551 INFO:teuthology.orchestra.run.smithi028.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T20:54:05.552 INFO:teuthology.orchestra.run.smithi028.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T20:54:05.554 INFO:teuthology.orchestra.run.smithi028.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T20:54:05.558 INFO:teuthology.orchestra.run.smithi028.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T20:54:05.559 INFO:teuthology.orchestra.run.smithi028.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T20:54:05.633 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2024-04-26T20:54:05.653 INFO:teuthology.orchestra.run.smithi028.stdout:======================================================================================= 2024-04-26T20:54:05.653 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2024-04-26T20:54:05.653 INFO:teuthology.orchestra.run.smithi028.stdout:======================================================================================= 2024-04-26T20:54:05.653 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2024-04-26T20:54:05.653 INFO:teuthology.orchestra.run.smithi028.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T20:54:05.653 INFO:teuthology.orchestra.run.smithi028.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T20:54:05.653 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:54:05.653 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T20:54:05.653 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T20:54:05.653 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T20:54:05.653 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T20:54:05.653 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T20:54:05.653 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T20:54:05.654 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T20:54:05.654 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T20:54:05.654 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T20:54:05.654 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T20:54:05.654 INFO:teuthology.orchestra.run.smithi028.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T20:54:05.654 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T20:54:05.654 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T20:54:05.654 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T20:54:05.654 INFO:teuthology.orchestra.run.smithi028.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T20:54:05.654 INFO:teuthology.orchestra.run.smithi028.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T20:54:05.654 INFO:teuthology.orchestra.run.smithi028.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T20:54:05.654 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T20:54:05.654 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T20:54:05.655 INFO:teuthology.orchestra.run.smithi028.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T20:54:05.655 INFO:teuthology.orchestra.run.smithi028.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T20:54:05.655 INFO:teuthology.orchestra.run.smithi028.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T20:54:05.655 INFO:teuthology.orchestra.run.smithi028.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T20:54:05.655 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T20:54:05.655 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T20:54:05.655 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T20:54:05.655 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T20:54:05.655 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T20:54:05.655 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T20:54:05.655 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T20:54:05.655 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T20:54:05.655 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:54:05.656 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T20:54:05.656 INFO:teuthology.orchestra.run.smithi028.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T20:54:05.656 INFO:teuthology.orchestra.run.smithi028.stdout:Upgrading: 2024-04-26T20:54:05.656 INFO:teuthology.orchestra.run.smithi028.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T20:54:05.656 INFO:teuthology.orchestra.run.smithi028.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:54:05.656 INFO:teuthology.orchestra.run.smithi028.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T20:54:05.656 INFO:teuthology.orchestra.run.smithi028.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T20:54:05.656 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2024-04-26T20:54:05.656 INFO:teuthology.orchestra.run.smithi028.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T20:54:05.656 INFO:teuthology.orchestra.run.smithi028.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T20:54:05.656 INFO:teuthology.orchestra.run.smithi028.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T20:54:05.656 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T20:54:05.656 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T20:54:05.656 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T20:54:05.656 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T20:54:05.657 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T20:54:05.657 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T20:54:05.657 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T20:54:05.657 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T20:54:05.657 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T20:54:05.657 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T20:54:05.657 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T20:54:05.657 INFO:teuthology.orchestra.run.smithi028.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T20:54:05.657 INFO:teuthology.orchestra.run.smithi028.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T20:54:05.657 INFO:teuthology.orchestra.run.smithi028.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T20:54:05.657 INFO:teuthology.orchestra.run.smithi028.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T20:54:05.657 INFO:teuthology.orchestra.run.smithi028.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T20:54:05.657 INFO:teuthology.orchestra.run.smithi028.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T20:54:05.657 INFO:teuthology.orchestra.run.smithi028.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T20:54:05.658 INFO:teuthology.orchestra.run.smithi028.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T20:54:05.658 INFO:teuthology.orchestra.run.smithi028.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T20:54:05.658 INFO:teuthology.orchestra.run.smithi028.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T20:54:05.658 INFO:teuthology.orchestra.run.smithi028.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T20:54:05.658 INFO:teuthology.orchestra.run.smithi028.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T20:54:05.658 INFO:teuthology.orchestra.run.smithi028.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T20:54:05.658 INFO:teuthology.orchestra.run.smithi028.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:54:05.658 INFO:teuthology.orchestra.run.smithi028.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T20:54:05.658 INFO:teuthology.orchestra.run.smithi028.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T20:54:05.658 INFO:teuthology.orchestra.run.smithi028.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T20:54:05.658 INFO:teuthology.orchestra.run.smithi028.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T20:54:05.658 INFO:teuthology.orchestra.run.smithi028.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T20:54:05.658 INFO:teuthology.orchestra.run.smithi028.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T20:54:05.658 INFO:teuthology.orchestra.run.smithi028.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T20:54:05.659 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T20:54:05.659 INFO:teuthology.orchestra.run.smithi028.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T20:54:05.659 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T20:54:05.659 INFO:teuthology.orchestra.run.smithi028.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T20:54:05.659 INFO:teuthology.orchestra.run.smithi028.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T20:54:05.659 INFO:teuthology.orchestra.run.smithi028.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T20:54:05.659 INFO:teuthology.orchestra.run.smithi028.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T20:54:05.659 INFO:teuthology.orchestra.run.smithi028.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T20:54:05.659 INFO:teuthology.orchestra.run.smithi028.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T20:54:05.659 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T20:54:05.659 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T20:54:05.659 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T20:54:05.659 INFO:teuthology.orchestra.run.smithi028.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T20:54:05.660 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T20:54:05.660 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T20:54:05.660 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T20:54:05.660 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T20:54:05.660 INFO:teuthology.orchestra.run.smithi028.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T20:54:05.660 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T20:54:05.660 INFO:teuthology.orchestra.run.smithi028.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T20:54:05.660 INFO:teuthology.orchestra.run.smithi028.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T20:54:05.660 INFO:teuthology.orchestra.run.smithi028.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T20:54:05.660 INFO:teuthology.orchestra.run.smithi028.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T20:54:05.660 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T20:54:05.660 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T20:54:05.660 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T20:54:05.660 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T20:54:05.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T20:54:05.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T20:54:05.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T20:54:05.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T20:54:05.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T20:54:05.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T20:54:05.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T20:54:05.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T20:54:05.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T20:54:05.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T20:54:05.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T20:54:05.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T20:54:05.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T20:54:05.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T20:54:05.661 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T20:54:05.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T20:54:05.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T20:54:05.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T20:54:05.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T20:54:05.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T20:54:05.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T20:54:05.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T20:54:05.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T20:54:05.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T20:54:05.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T20:54:05.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T20:54:05.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T20:54:05.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T20:54:05.662 INFO:teuthology.orchestra.run.smithi028.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T20:54:05.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T20:54:05.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T20:54:05.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T20:54:05.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T20:54:05.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T20:54:05.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T20:54:05.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T20:54:05.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T20:54:05.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T20:54:05.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T20:54:05.663 INFO:teuthology.orchestra.run.smithi028.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T20:54:05.663 INFO:teuthology.orchestra.run.smithi028.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T20:54:05.663 INFO:teuthology.orchestra.run.smithi028.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T20:54:05.663 INFO:teuthology.orchestra.run.smithi028.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T20:54:05.664 INFO:teuthology.orchestra.run.smithi028.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T20:54:05.664 INFO:teuthology.orchestra.run.smithi028.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T20:54:05.664 INFO:teuthology.orchestra.run.smithi028.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T20:54:05.664 INFO:teuthology.orchestra.run.smithi028.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T20:54:05.664 INFO:teuthology.orchestra.run.smithi028.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T20:54:05.664 INFO:teuthology.orchestra.run.smithi028.stdout:Installing weak dependencies: 2024-04-26T20:54:05.664 INFO:teuthology.orchestra.run.smithi028.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T20:54:05.664 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:54:05.664 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2024-04-26T20:54:05.664 INFO:teuthology.orchestra.run.smithi028.stdout:======================================================================================= 2024-04-26T20:54:05.664 INFO:teuthology.orchestra.run.smithi028.stdout:Install 148 Packages 2024-04-26T20:54:05.664 INFO:teuthology.orchestra.run.smithi028.stdout:Upgrade 4 Packages 2024-04-26T20:54:05.664 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:54:05.665 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 222 M 2024-04-26T20:54:05.665 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2024-04-26T20:54:06.939 INFO:teuthology.orchestra.run.smithi028.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-04-26T20:54:07.306 INFO:teuthology.orchestra.run.smithi028.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 2.4 MB/s | 889 kB 00:00 2024-04-26T20:54:07.373 INFO:teuthology.orchestra.run.smithi028.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 2.2 MB/s | 145 kB 00:00 2024-04-26T20:54:07.931 INFO:teuthology.orchestra.run.smithi028.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 3.8 MB/s | 5.4 MB 00:01 2024-04-26T20:54:07.998 INFO:teuthology.orchestra.run.smithi028.stdout:(5/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 3.8 MB/s | 2.4 MB 00:00 2024-04-26T20:54:08.199 INFO:teuthology.orchestra.run.smithi028.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 5.7 MB/s | 1.5 MB 00:00 2024-04-26T20:54:08.915 INFO:teuthology.orchestra.run.smithi028.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 5.1 MB/s | 4.7 MB 00:00 2024-04-26T20:54:10.130 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-26T20:54:10.349 INFO:teuthology.orchestra.run.smithi028.stdout:(8/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x86 7.8 MB/s | 17 MB 00:02 2024-04-26T20:54:10.433 INFO:teuthology.orchestra.run.smithi028.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 306 kB/s | 25 kB 00:00 2024-04-26T20:54:10.900 INFO:teuthology.orchestra.run.smithi028.stdout:(10/152): ceph-common-19.0.0-3306.gbc612ffb.el9 4.8 MB/s | 21 MB 00:04 2024-04-26T20:54:11.126 INFO:teuthology.orchestra.run.smithi028.stdout:(11/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 4.8 MB/s | 11 MB 00:02 2024-04-26T20:54:11.167 INFO:teuthology.orchestra.run.smithi028.stdout:(12/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 846 kB/s | 225 kB 00:00 2024-04-26T20:54:11.200 INFO:teuthology.orchestra.run.smithi028.stdout:(13/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 367 kB/s | 27 kB 00:00 2024-04-26T20:54:11.242 INFO:teuthology.orchestra.run.smithi028.stdout:(14/152): libcephfs-devel-19.0.0-3306.gbc612ffb 427 kB/s | 32 kB 00:00 2024-04-26T20:54:11.342 INFO:teuthology.orchestra.run.smithi028.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.6 MB/s | 163 kB 00:00 2024-04-26T20:54:11.443 INFO:teuthology.orchestra.run.smithi028.stdout:(16/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 3.0 MB/s | 747 kB 00:00 2024-04-26T20:54:11.476 INFO:teuthology.orchestra.run.smithi028.stdout:(17/152): librados-devel-19.0.0-3306.gbc612ffb. 955 kB/s | 127 kB 00:00 2024-04-26T20:54:11.560 INFO:teuthology.orchestra.run.smithi028.stdout:(18/152): libradosstriper1-19.0.0-3306.gbc612ff 4.2 MB/s | 501 kB 00:00 2024-04-26T20:54:11.660 INFO:teuthology.orchestra.run.smithi028.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 455 kB/s | 45 kB 00:00 2024-04-26T20:54:11.777 INFO:teuthology.orchestra.run.smithi028.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.1 MB/s | 135 kB 00:00 2024-04-26T20:54:11.903 INFO:teuthology.orchestra.run.smithi028.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 1.3 MB/s | 161 kB 00:00 2024-04-26T20:54:12.028 INFO:teuthology.orchestra.run.smithi028.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 2.5 MB/s | 321 kB 00:00 2024-04-26T20:54:12.153 INFO:teuthology.orchestra.run.smithi028.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 2.3 MB/s | 298 kB 00:00 2024-04-26T20:54:12.279 INFO:teuthology.orchestra.run.smithi028.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 801 kB/s | 99 kB 00:00 2024-04-26T20:54:12.445 INFO:teuthology.orchestra.run.smithi028.stdout:(25/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 5.5 MB/s | 5.4 MB 00:00 2024-04-26T20:54:12.463 INFO:teuthology.orchestra.run.smithi103.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:54:12.465 INFO:teuthology.orchestra.run.smithi103.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T20:54:12.470 INFO:teuthology.orchestra.run.smithi028.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 447 kB/s | 85 kB 00:00 2024-04-26T20:54:12.484 INFO:teuthology.orchestra.run.smithi103.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T20:54:12.486 INFO:teuthology.orchestra.run.smithi103.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T20:54:12.487 INFO:teuthology.orchestra.run.smithi103.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T20:54:12.489 INFO:teuthology.orchestra.run.smithi103.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T20:54:12.490 INFO:teuthology.orchestra.run.smithi103.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T20:54:12.494 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T20:54:12.495 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T20:54:12.565 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-04-26T20:54:12.587 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-04-26T20:54:12.587 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-04-26T20:54:12.587 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-04-26T20:54:12.587 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-04-26T20:54:12.587 INFO:teuthology.orchestra.run.smithi103.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T20:54:12.587 INFO:teuthology.orchestra.run.smithi103.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T20:54:12.587 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:54:12.587 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T20:54:12.587 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T20:54:12.587 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T20:54:12.587 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T20:54:12.587 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T20:54:12.587 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T20:54:12.588 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T20:54:12.588 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T20:54:12.588 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T20:54:12.588 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T20:54:12.588 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T20:54:12.588 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T20:54:12.588 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T20:54:12.588 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T20:54:12.588 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T20:54:12.588 INFO:teuthology.orchestra.run.smithi103.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T20:54:12.588 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T20:54:12.588 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T20:54:12.588 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T20:54:12.588 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T20:54:12.589 INFO:teuthology.orchestra.run.smithi103.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T20:54:12.589 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T20:54:12.589 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T20:54:12.589 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T20:54:12.589 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T20:54:12.589 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T20:54:12.589 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T20:54:12.589 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T20:54:12.589 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T20:54:12.589 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T20:54:12.589 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T20:54:12.589 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:54:12.589 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T20:54:12.590 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T20:54:12.590 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrading: 2024-04-26T20:54:12.590 INFO:teuthology.orchestra.run.smithi103.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T20:54:12.590 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T20:54:12.590 INFO:teuthology.orchestra.run.smithi103.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T20:54:12.590 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T20:54:12.590 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-04-26T20:54:12.590 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T20:54:12.590 INFO:teuthology.orchestra.run.smithi103.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T20:54:12.590 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T20:54:12.590 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T20:54:12.590 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T20:54:12.590 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T20:54:12.591 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T20:54:12.591 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T20:54:12.591 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T20:54:12.591 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T20:54:12.591 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T20:54:12.591 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T20:54:12.591 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T20:54:12.591 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T20:54:12.591 INFO:teuthology.orchestra.run.smithi103.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T20:54:12.591 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T20:54:12.591 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T20:54:12.591 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T20:54:12.591 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T20:54:12.592 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T20:54:12.592 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T20:54:12.592 INFO:teuthology.orchestra.run.smithi103.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T20:54:12.592 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T20:54:12.592 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T20:54:12.592 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T20:54:12.592 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T20:54:12.592 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T20:54:12.592 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T20:54:12.592 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T20:54:12.592 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T20:54:12.592 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T20:54:12.592 INFO:teuthology.orchestra.run.smithi103.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T20:54:12.593 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T20:54:12.593 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T20:54:12.593 INFO:teuthology.orchestra.run.smithi103.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T20:54:12.593 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T20:54:12.593 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T20:54:12.593 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T20:54:12.593 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T20:54:12.593 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T20:54:12.593 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T20:54:12.593 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T20:54:12.593 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T20:54:12.593 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T20:54:12.593 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T20:54:12.594 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T20:54:12.594 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T20:54:12.594 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T20:54:12.594 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T20:54:12.594 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T20:54:12.594 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T20:54:12.594 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T20:54:12.594 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T20:54:12.594 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T20:54:12.594 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T20:54:12.594 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T20:54:12.594 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T20:54:12.594 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T20:54:12.594 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T20:54:12.595 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T20:54:12.595 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T20:54:12.595 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T20:54:12.595 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T20:54:12.595 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T20:54:12.595 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T20:54:12.595 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T20:54:12.595 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T20:54:12.597 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T20:54:12.597 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T20:54:12.597 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T20:54:12.597 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T20:54:12.597 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T20:54:12.598 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T20:54:12.598 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T20:54:12.598 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T20:54:12.598 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T20:54:12.598 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T20:54:12.598 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T20:54:12.598 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T20:54:12.598 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T20:54:12.598 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T20:54:12.598 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T20:54:12.598 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T20:54:12.598 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T20:54:12.599 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T20:54:12.599 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T20:54:12.599 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T20:54:12.599 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T20:54:12.599 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T20:54:12.599 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T20:54:12.599 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T20:54:12.600 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T20:54:12.600 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T20:54:12.600 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T20:54:12.600 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T20:54:12.600 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T20:54:12.600 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T20:54:12.600 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T20:54:12.600 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T20:54:12.600 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T20:54:12.600 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T20:54:12.600 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T20:54:12.601 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T20:54:12.601 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T20:54:12.601 INFO:teuthology.orchestra.run.smithi103.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T20:54:12.601 INFO:teuthology.orchestra.run.smithi103.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T20:54:12.601 INFO:teuthology.orchestra.run.smithi103.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T20:54:12.601 INFO:teuthology.orchestra.run.smithi103.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T20:54:12.601 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T20:54:12.601 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T20:54:12.601 INFO:teuthology.orchestra.run.smithi103.stdout:Installing weak dependencies: 2024-04-26T20:54:12.601 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T20:54:12.601 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:54:12.601 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-04-26T20:54:12.601 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-04-26T20:54:12.601 INFO:teuthology.orchestra.run.smithi103.stdout:Install 148 Packages 2024-04-26T20:54:12.602 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrade 4 Packages 2024-04-26T20:54:12.602 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:54:12.602 INFO:teuthology.orchestra.run.smithi028.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 1.3 MB/s | 171 kB 00:00 2024-04-26T20:54:12.603 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 222 M 2024-04-26T20:54:12.603 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-04-26T20:54:12.696 INFO:teuthology.orchestra.run.smithi028.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 345 kB/s | 34 kB 00:00 2024-04-26T20:54:12.813 INFO:teuthology.orchestra.run.smithi028.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.2 MB/s | 142 kB 00:00 2024-04-26T20:54:12.972 INFO:teuthology.orchestra.run.smithi028.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 5.9 MB/s | 3.1 MB 00:00 2024-04-26T20:54:13.572 INFO:teuthology.orchestra.run.smithi028.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 4.7 MB/s | 3.5 MB 00:00 2024-04-26T20:54:13.689 INFO:teuthology.orchestra.run.smithi028.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.1 MB/s | 249 kB 00:00 2024-04-26T20:54:13.797 INFO:teuthology.orchestra.run.smithi028.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 460 kB/s | 49 kB 00:00 2024-04-26T20:54:13.890 INFO:teuthology.orchestra.run.smithi103.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 17 kB/s | 6.6 kB 00:00 2024-04-26T20:54:13.898 INFO:teuthology.orchestra.run.smithi028.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 166 kB/s | 16 kB 00:00 2024-04-26T20:54:14.048 INFO:teuthology.orchestra.run.smithi028.stdout:(35/152): ceph-mgr-diskprediction-local-19.0.0- 6.9 MB/s | 7.4 MB 00:01 2024-04-26T20:54:14.082 INFO:teuthology.orchestra.run.smithi028.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 1.5 MB/s | 273 kB 00:00 2024-04-26T20:54:14.182 INFO:teuthology.orchestra.run.smithi028.stdout:(37/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 183 kB/s | 18 kB 00:00 2024-04-26T20:54:14.224 INFO:teuthology.orchestra.run.smithi028.stdout:(38/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 4.3 MB/s | 762 kB 00:00 2024-04-26T20:54:14.424 INFO:teuthology.orchestra.run.smithi028.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 334 kB/s | 80 kB 00:00 2024-04-26T20:54:14.440 INFO:teuthology.orchestra.run.smithi103.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.6 MB/s | 889 kB 00:00 2024-04-26T20:54:14.500 INFO:teuthology.orchestra.run.smithi028.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-04-26T20:54:14.534 INFO:teuthology.orchestra.run.smithi028.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-04-26T20:54:14.566 INFO:teuthology.orchestra.run.smithi103.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 1.1 MB/s | 145 kB 00:00 2024-04-26T20:54:14.609 INFO:teuthology.orchestra.run.smithi028.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-04-26T20:54:14.676 INFO:teuthology.orchestra.run.smithi028.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-26T20:54:14.734 INFO:teuthology.orchestra.run.smithi028.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 6.1 MB/s | 1.2 MB 00:00 2024-04-26T20:54:14.784 INFO:teuthology.orchestra.run.smithi028.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-04-26T20:54:14.826 INFO:teuthology.orchestra.run.smithi028.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-04-26T20:54:14.877 INFO:teuthology.orchestra.run.smithi028.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-26T20:54:14.974 INFO:teuthology.orchestra.run.smithi103.stdout:(4/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 3.7 MB/s | 5.4 MB 00:01 2024-04-26T20:54:15.258 INFO:teuthology.orchestra.run.smithi103.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 5.4 MB/s | 1.5 MB 00:00 2024-04-26T20:54:15.500 INFO:teuthology.orchestra.run.smithi103.stdout:(6/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 2.5 MB/s | 2.4 MB 00:00 2024-04-26T20:54:15.602 INFO:teuthology.orchestra.run.smithi028.stdout:(48/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 9.5 MB/s | 49 MB 00:05 2024-04-26T20:54:15.652 INFO:teuthology.orchestra.run.smithi028.stdout:(49/152): automake-1.16.2-8.el9.noarch.rpm 836 kB/s | 689 kB 00:00 2024-04-26T20:54:15.727 INFO:teuthology.orchestra.run.smithi028.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-04-26T20:54:15.753 INFO:teuthology.orchestra.run.smithi028.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 216 kB/s | 21 kB 00:00 2024-04-26T20:54:15.853 INFO:teuthology.orchestra.run.smithi028.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 329 kB/s | 33 kB 00:00 2024-04-26T20:54:15.995 INFO:teuthology.orchestra.run.smithi028.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 1.1 MB/s | 313 kB 00:00 2024-04-26T20:54:16.008 INFO:teuthology.orchestra.run.smithi103.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 6.3 MB/s | 4.7 MB 00:00 2024-04-26T20:54:16.079 INFO:teuthology.orchestra.run.smithi028.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 220 kB/s | 18 kB 00:00 2024-04-26T20:54:16.138 INFO:teuthology.orchestra.run.smithi028.stdout:(55/152): boost-program-options-1.75.0-8.el9.x8 200 kB/s | 107 kB 00:00 2024-04-26T20:54:16.162 INFO:teuthology.orchestra.run.smithi028.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 134 kB/s | 11 kB 00:00 2024-04-26T20:54:16.313 INFO:teuthology.orchestra.run.smithi028.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 303 kB/s | 45 kB 00:00 2024-04-26T20:54:16.363 INFO:teuthology.orchestra.run.smithi028.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 714 kB/s | 160 kB 00:00 2024-04-26T20:54:16.692 INFO:teuthology.orchestra.run.smithi103.stdout:(8/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 6.7 MB/s | 21 MB 00:03 2024-04-26T20:54:16.742 INFO:teuthology.orchestra.run.smithi103.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 516 kB/s | 25 kB 00:00 2024-04-26T20:54:16.805 INFO:teuthology.orchestra.run.smithi028.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 1.3 MB/s | 582 kB 00:00 2024-04-26T20:54:16.864 INFO:teuthology.orchestra.run.smithi028.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 445 kB/s | 25 kB 00:00 2024-04-26T20:54:16.964 INFO:teuthology.orchestra.run.smithi028.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 479 kB/s | 47 kB 00:00 2024-04-26T20:54:17.156 INFO:teuthology.orchestra.run.smithi028.stdout:(62/152): librdkafka-1.6.1-102.el9.x86_64.rpm 787 kB/s | 662 kB 00:00 2024-04-26T20:54:17.223 INFO:teuthology.orchestra.run.smithi028.stdout:(63/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.2 MB/s | 3.0 MB 00:01 2024-04-26T20:54:17.323 INFO:teuthology.orchestra.run.smithi028.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-04-26T20:54:17.381 INFO:teuthology.orchestra.run.smithi028.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 224 kB/s | 35 kB 00:00 2024-04-26T20:54:17.384 INFO:teuthology.orchestra.run.smithi103.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 7.7 MB/s | 11 MB 00:01 2024-04-26T20:54:17.415 INFO:teuthology.orchestra.run.smithi028.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 649 kB/s | 292 kB 00:00 2024-04-26T20:54:17.467 INFO:teuthology.orchestra.run.smithi103.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 2.7 MB/s | 225 kB 00:00 2024-04-26T20:54:17.526 INFO:teuthology.orchestra.run.smithi103.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 470 kB/s | 27 kB 00:00 2024-04-26T20:54:17.585 INFO:teuthology.orchestra.run.smithi103.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 553 kB/s | 32 kB 00:00 2024-04-26T20:54:17.735 INFO:teuthology.orchestra.run.smithi103.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 4.9 MB/s | 747 kB 00:00 2024-04-26T20:54:17.827 INFO:teuthology.orchestra.run.smithi103.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.8 MB/s | 163 kB 00:00 2024-04-26T20:54:17.902 INFO:teuthology.orchestra.run.smithi103.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.7 MB/s | 127 kB 00:00 2024-04-26T20:54:17.994 INFO:teuthology.orchestra.run.smithi103.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 5.4 MB/s | 501 kB 00:00 2024-04-26T20:54:18.491 INFO:teuthology.orchestra.run.smithi028.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-04-26T20:54:18.694 INFO:teuthology.orchestra.run.smithi103.stdout:(18/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 7.7 MB/s | 5.4 MB 00:00 2024-04-26T20:54:18.937 INFO:teuthology.orchestra.run.smithi103.stdout:(19/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 4.9 MB/s | 17 MB 00:03 2024-04-26T20:54:18.961 INFO:teuthology.orchestra.run.smithi103.stdout:(20/152): python3-ceph-argparse-19.0.0-3306.gbc 169 kB/s | 45 kB 00:00 2024-04-26T20:54:18.995 INFO:teuthology.orchestra.run.smithi103.stdout:(21/152): python3-ceph-common-19.0.0-3306.gbc61 2.3 MB/s | 135 kB 00:00 2024-04-26T20:54:19.016 INFO:teuthology.orchestra.run.smithi028.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 1.7 MB/s | 911 kB 00:00 2024-04-26T20:54:19.029 INFO:teuthology.orchestra.run.smithi103.stdout:(22/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.4 MB/s | 161 kB 00:00 2024-04-26T20:54:19.079 INFO:teuthology.orchestra.run.smithi103.stdout:(23/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.8 MB/s | 321 kB 00:00 2024-04-26T20:54:19.113 INFO:teuthology.orchestra.run.smithi103.stdout:(24/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 3.5 MB/s | 298 kB 00:00 2024-04-26T20:54:19.125 INFO:teuthology.orchestra.run.smithi028.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-04-26T20:54:19.147 INFO:teuthology.orchestra.run.smithi103.stdout:(25/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.5 MB/s | 99 kB 00:00 2024-04-26T20:54:19.172 INFO:teuthology.orchestra.run.smithi103.stdout:(26/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-26T20:54:19.231 INFO:teuthology.orchestra.run.smithi103.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.9 MB/s | 171 kB 00:00 2024-04-26T20:54:19.290 INFO:teuthology.orchestra.run.smithi103.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 599 kB/s | 34 kB 00:00 2024-04-26T20:54:19.365 INFO:teuthology.orchestra.run.smithi103.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.9 MB/s | 142 kB 00:00 2024-04-26T20:54:19.391 INFO:teuthology.orchestra.run.smithi028.stdout:(70/152): openblas-openmp-0.3.21-2.el9.x86_64.r 2.3 MB/s | 4.7 MB 00:02 2024-04-26T20:54:19.665 INFO:teuthology.orchestra.run.smithi103.stdout:(30/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 6.0 MB/s | 3.1 MB 00:00 2024-04-26T20:54:19.725 INFO:teuthology.orchestra.run.smithi028.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 3.0 MB/s | 1.0 MB 00:00 2024-04-26T20:54:19.801 INFO:teuthology.orchestra.run.smithi028.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 87 kB 00:00 2024-04-26T20:54:19.824 INFO:teuthology.orchestra.run.smithi103.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 7.7 MB/s | 3.5 MB 00:00 2024-04-26T20:54:19.924 INFO:teuthology.orchestra.run.smithi103.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.5 MB/s | 249 kB 00:00 2024-04-26T20:54:19.999 INFO:teuthology.orchestra.run.smithi103.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 666 kB/s | 49 kB 00:00 2024-04-26T20:54:20.075 INFO:teuthology.orchestra.run.smithi103.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 222 kB/s | 16 kB 00:00 2024-04-26T20:54:20.376 INFO:teuthology.orchestra.run.smithi028.stdout:(73/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.5 MB/s | 4.4 MB 00:02 2024-04-26T20:54:20.501 INFO:teuthology.orchestra.run.smithi028.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.9 MB/s | 244 kB 00:00 2024-04-26T20:54:20.592 INFO:teuthology.orchestra.run.smithi103.stdout:(35/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 13 MB/s | 49 MB 00:03 2024-04-26T20:54:20.626 INFO:teuthology.orchestra.run.smithi028.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-04-26T20:54:20.633 INFO:teuthology.orchestra.run.smithi103.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 490 kB/s | 273 kB 00:00 2024-04-26T20:54:20.685 INFO:teuthology.orchestra.run.smithi028.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 443 kB/s | 26 kB 00:00 2024-04-26T20:54:20.709 INFO:teuthology.orchestra.run.smithi103.stdout:(37/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 6.4 MB/s | 762 kB 00:00 2024-04-26T20:54:20.734 INFO:teuthology.orchestra.run.smithi103.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 185 kB/s | 18 kB 00:00 2024-04-26T20:54:20.743 INFO:teuthology.orchestra.run.smithi028.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 332 kB/s | 19 kB 00:00 2024-04-26T20:54:20.835 INFO:teuthology.orchestra.run.smithi028.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-04-26T20:54:20.902 INFO:teuthology.orchestra.run.smithi028.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 526 kB/s | 35 kB 00:00 2024-04-26T20:54:20.909 INFO:teuthology.orchestra.run.smithi103.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 6.0 MB/s | 7.4 MB 00:01 2024-04-26T20:54:20.934 INFO:teuthology.orchestra.run.smithi103.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 403 kB/s | 80 kB 00:00 2024-04-26T20:54:21.101 INFO:teuthology.orchestra.run.smithi103.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 253 kB 00:00 2024-04-26T20:54:21.143 INFO:teuthology.orchestra.run.smithi103.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 840 kB/s | 195 kB 00:00 2024-04-26T20:54:21.219 INFO:teuthology.orchestra.run.smithi103.stdout:(43/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 803 kB 00:00 2024-04-26T20:54:21.244 INFO:teuthology.orchestra.run.smithi103.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-04-26T20:54:21.278 INFO:teuthology.orchestra.run.smithi103.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-26T20:54:21.311 INFO:teuthology.orchestra.run.smithi103.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-26T20:54:21.361 INFO:teuthology.orchestra.run.smithi103.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 4.7 MB/s | 1.2 MB 00:00 2024-04-26T20:54:21.387 INFO:teuthology.orchestra.run.smithi103.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.0 MB/s | 215 kB 00:00 2024-04-26T20:54:21.695 INFO:teuthology.orchestra.run.smithi103.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 347 kB/s | 107 kB 00:00 2024-04-26T20:54:21.762 INFO:teuthology.orchestra.run.smithi103.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 1.5 MB/s | 689 kB 00:00 2024-04-26T20:54:21.787 INFO:teuthology.orchestra.run.smithi103.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 238 kB/s | 21 kB 00:00 2024-04-26T20:54:21.832 INFO:teuthology.orchestra.run.smithi103.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 2.0 MB/s | 943 kB 00:00 2024-04-26T20:54:21.874 INFO:teuthology.orchestra.run.smithi103.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-04-26T20:54:21.901 INFO:teuthology.orchestra.run.smithi103.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 294 kB/s | 33 kB 00:00 2024-04-26T20:54:21.957 INFO:teuthology.orchestra.run.smithi103.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 220 kB/s | 18 kB 00:00 2024-04-26T20:54:21.983 INFO:teuthology.orchestra.run.smithi103.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 140 kB/s | 11 kB 00:00 2024-04-26T20:54:22.058 INFO:teuthology.orchestra.run.smithi103.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 617 kB/s | 45 kB 00:00 2024-04-26T20:54:22.083 INFO:teuthology.orchestra.run.smithi103.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-04-26T20:54:22.127 INFO:teuthology.orchestra.run.smithi028.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 1.9 MB/s | 5.8 MB 00:03 2024-04-26T20:54:22.184 INFO:teuthology.orchestra.run.smithi103.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.5 MB/s | 3.0 MB 00:00 2024-04-26T20:54:22.242 INFO:teuthology.orchestra.run.smithi103.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.5 MB/s | 662 kB 00:00 2024-04-26T20:54:22.276 INFO:teuthology.orchestra.run.smithi103.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 3.0 MB/s | 582 kB 00:00 2024-04-26T20:54:22.301 INFO:teuthology.orchestra.run.smithi103.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 218 kB/s | 25 kB 00:00 2024-04-26T20:54:22.311 INFO:teuthology.orchestra.run.smithi028.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 2.4 MB/s | 6.0 MB 00:02 2024-04-26T20:54:22.326 INFO:teuthology.orchestra.run.smithi103.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 572 kB/s | 47 kB 00:00 2024-04-26T20:54:22.344 INFO:teuthology.orchestra.run.smithi028.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 250 kB 00:00 2024-04-26T20:54:22.376 INFO:teuthology.orchestra.run.smithi103.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2024-04-26T20:54:22.410 INFO:teuthology.orchestra.run.smithi103.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-26T20:54:22.428 INFO:teuthology.orchestra.run.smithi028.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 935 kB/s | 77 kB 00:00 2024-04-26T20:54:22.435 INFO:teuthology.orchestra.run.smithi103.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 325 kB/s | 35 kB 00:00 2024-04-26T20:54:22.462 INFO:teuthology.orchestra.run.smithi028.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-04-26T20:54:22.512 INFO:teuthology.orchestra.run.smithi028.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 511 kB/s | 42 kB 00:00 2024-04-26T20:54:22.612 INFO:teuthology.orchestra.run.smithi028.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 1.8 MB/s | 269 kB 00:00 2024-04-26T20:54:22.627 INFO:teuthology.orchestra.run.smithi103.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-04-26T20:54:22.654 INFO:teuthology.orchestra.run.smithi028.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-04-26T20:54:22.746 INFO:teuthology.orchestra.run.smithi028.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 561 kB/s | 51 kB 00:00 2024-04-26T20:54:22.769 INFO:teuthology.orchestra.run.smithi103.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 6.3 MB/s | 911 kB 00:00 2024-04-26T20:54:22.788 INFO:teuthology.orchestra.run.smithi028.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-04-26T20:54:22.830 INFO:teuthology.orchestra.run.smithi028.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 648 kB/s | 54 kB 00:00 2024-04-26T20:54:22.870 INFO:teuthology.orchestra.run.smithi103.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.7 MB/s | 4.4 MB 00:00 2024-04-26T20:54:22.913 INFO:teuthology.orchestra.run.smithi028.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 504 kB/s | 42 kB 00:00 2024-04-26T20:54:22.970 INFO:teuthology.orchestra.run.smithi103.stdout:(70/152): openblas-openmp-0.3.21-2.el9.x86_64.r 8.0 MB/s | 4.7 MB 00:00 2024-04-26T20:54:23.003 INFO:teuthology.orchestra.run.smithi103.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 571 kB/s | 133 kB 00:00 2024-04-26T20:54:23.014 INFO:teuthology.orchestra.run.smithi028.stdout:(92/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.4 MB/s | 5.1 MB 00:02 2024-04-26T20:54:23.039 INFO:teuthology.orchestra.run.smithi028.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 355 kB/s | 44 kB 00:00 2024-04-26T20:54:23.145 INFO:teuthology.orchestra.run.smithi103.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 618 kB/s | 87 kB 00:00 2024-04-26T20:54:23.162 INFO:teuthology.orchestra.run.smithi028.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 294 kB/s | 36 kB 00:00 2024-04-26T20:54:23.195 INFO:teuthology.orchestra.run.smithi103.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.5 MB/s | 1.0 MB 00:00 2024-04-26T20:54:23.198 INFO:teuthology.orchestra.run.smithi028.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.2 MB/s | 221 kB 00:00 2024-04-26T20:54:23.226 INFO:teuthology.orchestra.run.smithi028.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 786 kB/s | 47 kB 00:00 2024-04-26T20:54:23.288 INFO:teuthology.orchestra.run.smithi103.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-04-26T20:54:23.307 INFO:teuthology.orchestra.run.smithi028.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 701 kB/s | 58 kB 00:00 2024-04-26T20:54:23.416 INFO:teuthology.orchestra.run.smithi028.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 305 kB 00:00 2024-04-26T20:54:23.421 INFO:teuthology.orchestra.run.smithi103.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-04-26T20:54:23.491 INFO:teuthology.orchestra.run.smithi028.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 858 kB/s | 64 kB 00:00 2024-04-26T20:54:23.538 INFO:teuthology.orchestra.run.smithi103.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 221 kB/s | 26 kB 00:00 2024-04-26T20:54:23.566 INFO:teuthology.orchestra.run.smithi028.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 604 kB/s | 45 kB 00:00 2024-04-26T20:54:23.625 INFO:teuthology.orchestra.run.smithi028.stdout:(101/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 296 kB/s | 94 kB 00:00 2024-04-26T20:54:23.690 INFO:teuthology.orchestra.run.smithi103.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-26T20:54:23.739 INFO:teuthology.orchestra.run.smithi103.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 96 kB/s | 19 kB 00:00 2024-04-26T20:54:23.856 INFO:teuthology.orchestra.run.smithi103.stdout:(79/152): postgresql-server-13.14-1.el9.x86_64. 5.9 MB/s | 5.8 MB 00:00 2024-04-26T20:54:23.889 INFO:teuthology.orchestra.run.smithi103.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 868 kB/s | 172 kB 00:00 2024-04-26T20:54:23.925 INFO:teuthology.orchestra.run.smithi028.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 62 kB/s | 22 kB 00:00 2024-04-26T20:54:23.948 INFO:teuthology.orchestra.run.smithi103.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 167 kB/s | 35 kB 00:00 2024-04-26T20:54:24.015 INFO:teuthology.orchestra.run.smithi103.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-04-26T20:54:24.049 INFO:teuthology.orchestra.run.smithi103.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-04-26T20:54:24.108 INFO:teuthology.orchestra.run.smithi103.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 846 kB/s | 77 kB 00:00 2024-04-26T20:54:24.149 INFO:teuthology.orchestra.run.smithi103.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 423 kB/s | 42 kB 00:00 2024-04-26T20:54:24.201 INFO:teuthology.orchestra.run.smithi028.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.0 MB/s | 548 kB 00:00 2024-04-26T20:54:24.208 INFO:teuthology.orchestra.run.smithi103.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-04-26T20:54:24.241 INFO:teuthology.orchestra.run.smithi103.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-04-26T20:54:24.300 INFO:teuthology.orchestra.run.smithi103.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-04-26T20:54:24.342 INFO:teuthology.orchestra.run.smithi103.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 511 kB/s | 51 kB 00:00 2024-04-26T20:54:24.393 INFO:teuthology.orchestra.run.smithi028.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.1 MB/s | 863 kB 00:00 2024-04-26T20:54:24.426 INFO:teuthology.orchestra.run.smithi028.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-04-26T20:54:24.443 INFO:teuthology.orchestra.run.smithi028.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 420 kB/s | 19 kB 00:00 2024-04-26T20:54:24.459 INFO:teuthology.orchestra.run.smithi103.stdout:(90/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.5 MB/s | 5.1 MB 00:00 2024-04-26T20:54:24.501 INFO:teuthology.orchestra.run.smithi103.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 268 kB/s | 54 kB 00:00 2024-04-26T20:54:24.568 INFO:teuthology.orchestra.run.smithi103.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 387 kB/s | 42 kB 00:00 2024-04-26T20:54:24.635 INFO:teuthology.orchestra.run.smithi103.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 333 kB/s | 44 kB 00:00 2024-04-26T20:54:24.694 INFO:teuthology.orchestra.run.smithi103.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-04-26T20:54:24.736 INFO:teuthology.orchestra.run.smithi103.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 361 kB/s | 36 kB 00:00 2024-04-26T20:54:24.760 INFO:teuthology.orchestra.run.smithi028.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 82 kB/s | 26 kB 00:00 2024-04-26T20:54:24.803 INFO:teuthology.orchestra.run.smithi103.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 439 kB/s | 47 kB 00:00 2024-04-26T20:54:24.853 INFO:teuthology.orchestra.run.smithi103.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2024-04-26T20:54:24.869 INFO:teuthology.orchestra.run.smithi028.stdout:(108/152): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-04-26T20:54:24.894 INFO:teuthology.orchestra.run.smithi028.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 402 kB/s | 49 kB 00:00 2024-04-26T20:54:24.895 INFO:teuthology.orchestra.run.smithi103.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 635 kB/s | 58 kB 00:00 2024-04-26T20:54:24.919 INFO:teuthology.orchestra.run.smithi028.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.4 MB/s | 67 kB 00:00 2024-04-26T20:54:24.945 INFO:teuthology.orchestra.run.smithi028.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-04-26T20:54:24.945 INFO:teuthology.orchestra.run.smithi103.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 94 kB 00:00 2024-04-26T20:54:24.987 INFO:teuthology.orchestra.run.smithi103.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 699 kB/s | 64 kB 00:00 2024-04-26T20:54:25.054 INFO:teuthology.orchestra.run.smithi103.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 414 kB/s | 45 kB 00:00 2024-04-26T20:54:25.096 INFO:teuthology.orchestra.run.smithi103.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 207 kB/s | 22 kB 00:00 2024-04-26T20:54:25.220 INFO:teuthology.orchestra.run.smithi028.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.8 MB/s | 839 kB 00:00 2024-04-26T20:54:25.238 INFO:teuthology.orchestra.run.smithi103.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-04-26T20:54:25.262 INFO:teuthology.orchestra.run.smithi028.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 1.7 MB/s | 548 kB 00:00 2024-04-26T20:54:25.287 INFO:teuthology.orchestra.run.smithi028.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 718 kB/s | 43 kB 00:00 2024-04-26T20:54:25.332 INFO:teuthology.orchestra.run.smithi028.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 470 kB/s | 32 kB 00:00 2024-04-26T20:54:25.363 INFO:teuthology.orchestra.run.smithi028.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 202 kB/s | 15 kB 00:00 2024-04-26T20:54:25.413 INFO:teuthology.orchestra.run.smithi028.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.1 MB/s | 172 kB 00:00 2024-04-26T20:54:25.447 INFO:teuthology.orchestra.run.smithi028.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-04-26T20:54:25.480 INFO:teuthology.orchestra.run.smithi028.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.7 MB/s | 303 kB 00:00 2024-04-26T20:54:25.506 INFO:teuthology.orchestra.run.smithi028.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 645 kB/s | 35 kB 00:00 2024-04-26T20:54:25.539 INFO:teuthology.orchestra.run.smithi028.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.5 MB/s | 206 kB 00:00 2024-04-26T20:54:25.597 INFO:teuthology.orchestra.run.smithi103.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 13 MB/s | 16 MB 00:01 2024-04-26T20:54:25.756 INFO:teuthology.orchestra.run.smithi103.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 833 kB/s | 548 kB 00:00 2024-04-26T20:54:25.856 INFO:teuthology.orchestra.run.smithi103.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 500 kB/s | 309 kB 00:00 2024-04-26T20:54:25.881 INFO:teuthology.orchestra.run.smithi028.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 5.5 MB/s | 2.0 MB 00:00 2024-04-26T20:54:25.915 INFO:teuthology.orchestra.run.smithi103.stdout:(107/152): grpc-data-1.46.7-10.el9.noarch.rpm 62 kB/s | 19 kB 00:00 2024-04-26T20:54:25.923 INFO:teuthology.orchestra.run.smithi028.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 377 kB/s | 144 kB 00:00 2024-04-26T20:54:25.949 INFO:teuthology.orchestra.run.smithi028.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 165 kB/s | 11 kB 00:00 2024-04-26T20:54:25.974 INFO:teuthology.orchestra.run.smithi103.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 223 kB/s | 26 kB 00:00 2024-04-26T20:54:25.974 INFO:teuthology.orchestra.run.smithi028.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 359 kB/s | 18 kB 00:00 2024-04-26T20:54:25.999 INFO:teuthology.orchestra.run.smithi028.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 472 kB/s | 23 kB 00:00 2024-04-26T20:54:26.024 INFO:teuthology.orchestra.run.smithi028.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 396 kB/s | 19 kB 00:00 2024-04-26T20:54:26.050 INFO:teuthology.orchestra.run.smithi028.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 400 kB/s | 20 kB 00:00 2024-04-26T20:54:26.099 INFO:teuthology.orchestra.run.smithi103.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 265 kB/s | 49 kB 00:00 2024-04-26T20:54:26.124 INFO:teuthology.orchestra.run.smithi103.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 451 kB/s | 67 kB 00:00 2024-04-26T20:54:26.241 INFO:teuthology.orchestra.run.smithi103.stdout:(111/152): libarrow-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-04-26T20:54:26.258 INFO:teuthology.orchestra.run.smithi028.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 4.4 MB/s | 1.0 MB 00:00 2024-04-26T20:54:26.275 INFO:teuthology.orchestra.run.smithi103.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 866 kB/s | 151 kB 00:00 2024-04-26T20:54:26.283 INFO:teuthology.orchestra.run.smithi028.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 199 kB/s | 46 kB 00:00 2024-04-26T20:54:26.334 INFO:teuthology.orchestra.run.smithi028.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.6 MB/s | 79 kB 00:00 2024-04-26T20:54:26.359 INFO:teuthology.orchestra.run.smithi028.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 781 kB/s | 58 kB 00:00 2024-04-26T20:54:26.367 INFO:teuthology.orchestra.run.smithi103.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-04-26T20:54:26.409 INFO:teuthology.orchestra.run.smithi028.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.6 MB/s | 272 kB 00:00 2024-04-26T20:54:26.417 INFO:teuthology.orchestra.run.smithi103.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 307 kB/s | 43 kB 00:00 2024-04-26T20:54:26.443 INFO:teuthology.orchestra.run.smithi028.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 205 kB/s | 16 kB 00:00 2024-04-26T20:54:26.468 INFO:teuthology.orchestra.run.smithi028.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-26T20:54:26.484 INFO:teuthology.orchestra.run.smithi103.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 277 kB/s | 32 kB 00:00 2024-04-26T20:54:26.502 INFO:teuthology.orchestra.run.smithi028.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 497 kB/s | 28 kB 00:00 2024-04-26T20:54:26.527 INFO:teuthology.orchestra.run.smithi028.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 532 kB/s | 31 kB 00:00 2024-04-26T20:54:26.561 INFO:teuthology.orchestra.run.smithi028.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-04-26T20:54:26.586 INFO:teuthology.orchestra.run.smithi028.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-26T20:54:26.618 INFO:teuthology.orchestra.run.smithi103.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 75 kB/s | 15 kB 00:00 2024-04-26T20:54:26.620 INFO:teuthology.orchestra.run.smithi028.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 619 kB/s | 36 kB 00:00 2024-04-26T20:54:26.645 INFO:teuthology.orchestra.run.smithi028.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 882 kB/s | 51 kB 00:00 2024-04-26T20:54:26.651 INFO:teuthology.orchestra.run.smithi103.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.0 MB/s | 172 kB 00:00 2024-04-26T20:54:26.685 INFO:teuthology.orchestra.run.smithi103.stdout:(118/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.5 MB/s | 839 kB 00:00 2024-04-26T20:54:26.744 INFO:teuthology.orchestra.run.smithi103.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-26T20:54:26.754 INFO:teuthology.orchestra.run.smithi028.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-04-26T20:54:26.779 INFO:teuthology.orchestra.run.smithi028.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 674 kB/s | 90 kB 00:00 2024-04-26T20:54:26.785 INFO:teuthology.orchestra.run.smithi103.stdout:(120/152): python3-cherrypy-18.6.1-2.el9.noarch 2.1 MB/s | 358 kB 00:00 2024-04-26T20:54:26.827 INFO:teuthology.orchestra.run.smithi103.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 249 kB/s | 35 kB 00:00 2024-04-26T20:54:26.846 INFO:teuthology.orchestra.run.smithi028.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.6 MB/s | 427 kB 00:00 2024-04-26T20:54:26.861 INFO:teuthology.orchestra.run.smithi103.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.7 MB/s | 206 kB 00:00 2024-04-26T20:54:26.871 INFO:teuthology.orchestra.run.smithi028.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 219 kB/s | 20 kB 00:00 2024-04-26T20:54:26.905 INFO:teuthology.orchestra.run.smithi028.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.3 MB/s | 192 kB 00:00 2024-04-26T20:54:26.978 INFO:teuthology.orchestra.run.smithi103.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-04-26T20:54:26.997 INFO:teuthology.orchestra.run.smithi028.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-26T20:54:27.012 INFO:teuthology.orchestra.run.smithi103.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 789 kB/s | 144 kB 00:00 2024-04-26T20:54:27.096 INFO:teuthology.orchestra.run.smithi103.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 152 kB/s | 18 kB 00:00 2024-04-26T20:54:27.171 INFO:teuthology.orchestra.run.smithi103.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 146 kB/s | 23 kB 00:00 2024-04-26T20:54:27.213 INFO:teuthology.orchestra.run.smithi103.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 167 kB/s | 19 kB 00:00 2024-04-26T20:54:27.263 INFO:teuthology.orchestra.run.smithi103.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 216 kB/s | 20 kB 00:00 2024-04-26T20:54:27.306 INFO:teuthology.orchestra.run.smithi028.stdout:(148/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 10 MB/s | 3.1 MB 00:00 2024-04-26T20:54:27.322 INFO:teuthology.orchestra.run.smithi103.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 3.8 MB/s | 2.0 MB 00:00 2024-04-26T20:54:27.380 INFO:teuthology.orchestra.run.smithi103.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-04-26T20:54:27.422 INFO:teuthology.orchestra.run.smithi103.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 292 kB/s | 46 kB 00:00 2024-04-26T20:54:27.447 INFO:teuthology.orchestra.run.smithi103.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 632 kB/s | 79 kB 00:00 2024-04-26T20:54:27.464 INFO:teuthology.orchestra.run.smithi028.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 8.6 MB/s | 1.3 MB 00:00 2024-04-26T20:54:27.498 INFO:teuthology.orchestra.run.smithi103.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 491 kB/s | 58 kB 00:00 2024-04-26T20:54:27.549 INFO:teuthology.orchestra.run.smithi103.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-26T20:54:27.573 INFO:teuthology.orchestra.run.smithi103.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-26T20:54:27.624 INFO:teuthology.orchestra.run.smithi103.stdout:(136/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-04-26T20:54:27.664 INFO:teuthology.orchestra.run.smithi028.stdout:(150/152): librados2-19.0.0-3306.gbc612ffb.el9. 4.5 MB/s | 3.4 MB 00:00 2024-04-26T20:54:27.666 INFO:teuthology.orchestra.run.smithi103.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 243 kB/s | 28 kB 00:00 2024-04-26T20:54:27.708 INFO:teuthology.orchestra.run.smithi103.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 231 kB/s | 31 kB 00:00 2024-04-26T20:54:27.775 INFO:teuthology.orchestra.run.smithi103.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-04-26T20:54:27.789 INFO:teuthology.orchestra.run.smithi028.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.4 MB/s | 2.4 MB 00:00 2024-04-26T20:54:27.825 INFO:teuthology.orchestra.run.smithi103.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 510 kB/s | 59 kB 00:00 2024-04-26T20:54:27.867 INFO:teuthology.orchestra.run.smithi103.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 227 kB/s | 36 kB 00:00 2024-04-26T20:54:27.909 INFO:teuthology.orchestra.run.smithi103.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 383 kB/s | 51 kB 00:00 2024-04-26T20:54:27.943 INFO:teuthology.orchestra.run.smithi103.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-04-26T20:54:27.968 INFO:teuthology.orchestra.run.smithi103.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 898 kB/s | 90 kB 00:00 2024-04-26T20:54:28.063 INFO:teuthology.orchestra.run.smithi103.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 167 kB/s | 20 kB 00:00 2024-04-26T20:54:28.101 INFO:teuthology.orchestra.run.smithi103.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-04-26T20:54:28.177 INFO:teuthology.orchestra.run.smithi103.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.6 MB/s | 427 kB 00:00 2024-04-26T20:54:28.235 INFO:teuthology.orchestra.run.smithi103.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 9.2 MB/s | 1.6 MB 00:00 2024-04-26T20:54:28.507 INFO:teuthology.orchestra.run.smithi028.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.8 MB/s | 16 MB 00:05 2024-04-26T20:54:28.519 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2024-04-26T20:54:28.519 INFO:teuthology.orchestra.run.smithi028.stdout:Total 9.7 MB/s | 222 MB 00:22 2024-04-26T20:54:28.544 INFO:teuthology.orchestra.run.smithi103.stdout:(149/152): librados2-19.0.0-3306.gbc612ffb.el9. 7.7 MB/s | 3.4 MB 00:00 2024-04-26T20:54:28.620 INFO:teuthology.orchestra.run.smithi103.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 4.3 MB/s | 1.3 MB 00:00 2024-04-26T20:54:28.961 INFO:teuthology.orchestra.run.smithi103.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.8 MB/s | 2.4 MB 00:00 2024-04-26T20:54:29.536 INFO:teuthology.orchestra.run.smithi103.stdout:(152/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 2.3 MB/s | 3.1 MB 00:01 2024-04-26T20:54:29.554 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-04-26T20:54:29.554 INFO:teuthology.orchestra.run.smithi103.stdout:Total 13 MB/s | 222 MB 00:16 2024-04-26T20:54:31.416 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2024-04-26T20:54:31.678 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2024-04-26T20:54:31.679 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2024-04-26T20:54:32.405 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-04-26T20:54:32.648 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-04-26T20:54:32.648 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-04-26T20:54:34.628 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2024-04-26T20:54:34.629 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2024-04-26T20:54:35.314 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-04-26T20:54:35.314 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-04-26T20:54:41.146 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2024-04-26T20:54:41.256 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T20:54:41.330 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T20:54:41.444 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T20:54:41.510 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T20:54:41.557 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T20:54:41.704 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T20:54:41.841 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-04-26T20:54:41.935 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T20:54:42.020 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T20:54:42.146 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T20:54:42.161 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T20:54:42.215 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T20:54:42.254 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T20:54:42.254 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:54:42.387 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T20:54:42.477 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:54:42.506 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:54:42.704 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:54:42.741 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:54:42.844 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T20:54:42.985 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:54:43.164 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T20:54:43.176 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:54:43.202 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:54:43.227 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T20:54:43.251 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:54:43.334 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:54:43.377 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:54:43.395 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:54:43.428 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:54:43.558 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:54:43.587 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:54:43.659 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:54:43.699 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T20:54:43.757 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T20:54:43.815 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:54:43.960 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:54:44.000 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:54:44.086 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T20:54:44.096 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:54:44.117 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:54:44.156 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:54:44.186 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T20:54:44.358 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T20:54:44.491 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T20:54:44.513 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:54:44.529 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T20:54:44.570 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T20:54:44.580 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T20:54:44.686 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T20:54:44.743 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T20:54:44.792 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T20:54:44.808 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T20:54:44.856 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T20:54:44.898 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T20:54:44.917 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T20:54:44.960 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T20:54:44.999 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T20:54:45.018 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T20:54:45.091 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T20:54:45.099 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T20:54:45.132 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T20:54:45.139 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T20:54:45.192 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T20:54:45.198 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T20:54:45.243 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T20:54:45.283 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T20:54:45.311 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T20:54:45.353 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T20:54:45.395 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T20:54:45.408 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T20:54:45.443 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T20:54:45.470 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T20:54:45.526 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T20:54:45.568 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T20:54:45.587 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T20:54:45.621 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T20:54:45.694 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T20:54:45.730 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T20:54:45.733 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T20:54:45.781 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T20:54:45.812 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T20:54:45.923 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T20:54:45.927 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T20:54:45.991 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T20:54:46.001 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T20:54:46.111 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T20:54:46.285 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T20:54:46.352 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T20:54:46.722 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T20:54:46.832 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T20:54:46.873 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T20:54:46.919 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T20:54:46.968 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T20:54:47.043 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T20:54:47.054 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T20:54:47.175 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T20:54:47.211 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T20:54:47.256 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T20:54:47.297 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T20:54:47.337 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T20:54:47.364 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T20:54:47.411 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T20:54:47.497 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T20:54:47.602 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T20:54:47.643 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T20:54:47.651 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T20:54:47.674 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T20:54:47.741 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T20:54:47.748 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T20:54:47.782 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T20:54:47.832 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T20:54:47.843 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T20:54:47.901 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T20:54:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T20:54:47.972 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T20:54:48.035 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T20:54:48.104 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T20:54:48.117 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T20:54:48.162 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T20:54:48.163 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T20:54:48.200 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T20:54:48.224 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T20:54:48.284 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T20:54:48.290 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T20:54:48.428 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T20:54:48.466 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T20:54:48.491 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T20:54:48.516 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T20:54:48.548 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T20:54:48.551 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T20:54:48.581 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T20:54:48.588 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T20:54:48.645 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T20:54:48.784 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T20:54:48.790 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T20:54:48.830 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T20:54:48.878 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T20:54:48.937 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T20:54:48.971 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T20:54:48.977 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T20:54:49.021 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T20:54:49.060 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T20:54:49.107 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T20:54:49.167 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T20:54:49.225 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T20:54:49.267 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T20:54:49.307 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T20:54:49.320 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T20:54:49.371 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T20:54:49.464 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T20:54:49.512 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T20:54:49.546 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T20:54:49.593 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T20:54:49.631 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T20:54:49.660 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T20:54:49.680 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T20:54:49.719 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T20:54:49.746 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T20:54:49.769 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T20:54:49.807 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T20:54:49.844 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T20:54:49.856 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T20:54:49.903 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T20:54:49.952 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T20:54:50.002 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T20:54:50.028 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T20:54:50.073 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T20:54:50.074 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T20:54:50.219 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T20:54:50.278 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T20:54:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T20:54:50.392 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T20:54:50.550 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T20:54:50.620 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T20:54:51.105 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T20:54:51.163 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T20:54:51.196 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T20:54:51.441 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T20:54:51.511 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T20:54:51.607 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T20:54:51.633 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T20:54:51.661 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T20:54:51.674 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T20:54:51.694 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T20:54:51.814 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T20:54:51.881 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T20:54:51.913 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T20:54:51.945 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T20:54:51.985 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T20:54:52.029 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T20:54:52.053 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T20:54:52.264 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T20:54:52.332 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T20:54:52.451 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T20:54:52.929 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T20:54:53.081 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T20:54:53.277 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T20:54:53.417 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T20:54:54.770 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T20:54:54.876 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T20:54:54.921 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T20:54:54.963 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T20:54:55.023 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T20:54:55.118 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T20:54:55.157 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T20:54:55.201 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T20:54:55.367 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T20:54:55.405 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:54:55.528 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:54:55.562 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T20:54:55.575 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T20:54:55.617 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T20:54:55.621 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:54:55.750 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T20:54:55.795 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T20:54:55.854 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T20:54:56.209 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T20:54:56.258 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:54:56.398 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T20:54:56.399 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:54:56.424 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:54:56.427 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T20:54:56.516 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T20:54:56.543 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T20:54:59.273 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:54:59.315 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:54:59.546 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:54:59.581 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:54:59.968 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:55:00.019 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:55:00.244 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:55:00.244 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T20:55:00.245 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T20:55:00.245 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:00.263 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T20:55:00.286 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:55:00.319 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:55:00.503 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T20:55:00.503 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T20:55:00.503 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T20:55:00.503 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:00.544 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:55:13.654 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:55:13.654 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /sys 2024-04-26T20:55:13.654 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /proc 2024-04-26T20:55:13.654 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /mnt 2024-04-26T20:55:13.654 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /var/tmp 2024-04-26T20:55:13.654 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /home 2024-04-26T20:55:13.654 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /root 2024-04-26T20:55:13.654 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /tmp 2024-04-26T20:55:13.655 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:13.879 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:55:13.923 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:55:13.924 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:55:13.924 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T20:55:13.924 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:13.981 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T20:55:13.981 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /sys 2024-04-26T20:55:13.981 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /proc 2024-04-26T20:55:13.981 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /mnt 2024-04-26T20:55:13.981 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /var/tmp 2024-04-26T20:55:13.982 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /home 2024-04-26T20:55:13.982 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /root 2024-04-26T20:55:13.982 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /tmp 2024-04-26T20:55:13.982 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:14.208 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:55:14.258 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T20:55:14.258 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:55:14.258 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T20:55:14.258 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:15.547 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:55:15.624 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:55:15.624 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:55:15.624 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T20:55:15.624 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:55:15.624 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:55:15.624 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:15.872 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:55:15.939 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:55:15.939 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:55:15.939 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T20:55:15.939 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:55:15.939 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:55:15.939 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:15.944 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:55:16.025 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T20:55:16.025 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:55:16.025 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T20:55:16.025 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:55:16.026 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T20:55:16.026 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:16.286 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:55:16.357 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T20:55:16.357 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:55:16.357 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T20:55:16.357 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:55:16.357 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T20:55:16.357 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:16.423 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:55:16.501 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:55:16.501 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:55:16.501 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T20:55:16.501 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:55:16.501 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:55:16.502 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:16.632 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T20:55:16.691 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T20:55:16.761 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T20:55:16.878 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:55:16.948 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T20:55:16.948 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:55:16.948 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T20:55:16.948 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:55:16.948 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T20:55:16.948 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:17.002 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T20:55:17.061 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T20:55:17.080 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T20:55:17.142 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T20:55:17.147 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T20:55:17.217 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T20:55:17.257 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:55:17.434 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:55:17.475 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T20:55:17.491 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T20:55:17.535 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T20:55:17.542 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T20:55:17.568 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T20:55:17.632 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T20:55:17.742 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:55:17.915 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:55:17.991 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T20:55:18.033 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T20:55:18.064 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T20:55:18.397 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:55:18.447 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:55:18.504 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T20:55:18.623 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T20:55:18.680 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:55:18.756 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:55:18.802 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T20:55:18.862 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T20:55:18.974 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T20:55:19.017 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:55:19.966 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:55:20.009 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:55:20.391 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T20:55:20.430 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:55:21.111 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:55:21.142 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:55:21.288 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:55:21.592 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T20:55:21.638 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:55:21.693 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T20:55:21.728 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:55:21.798 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T20:55:21.799 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:55:21.799 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:55:21.799 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T20:55:21.799 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:55:21.799 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:55:21.799 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:21.862 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:55:21.899 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:55:21.977 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T20:55:22.007 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:55:22.037 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T20:55:22.106 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T20:55:22.106 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:55:22.106 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T20:55:22.106 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:55:22.106 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T20:55:22.106 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:22.131 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T20:55:22.158 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:55:22.196 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T20:55:22.311 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T20:55:22.390 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T20:55:23.195 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T20:55:23.231 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:55:23.304 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:55:23.304 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:55:23.304 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T20:55:23.304 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:55:23.305 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:55:23.305 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:23.366 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:55:23.435 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:55:23.435 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:55:23.435 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T20:55:23.435 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:23.447 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T20:55:23.508 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:55:23.528 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:55:23.599 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:55:23.599 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:55:23.599 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T20:55:23.600 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:55:23.600 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:55:23.600 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:23.627 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T20:55:23.627 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:55:23.627 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T20:55:23.627 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:55:23.627 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T20:55:23.627 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:23.678 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:55:23.769 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T20:55:23.769 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:55:23.769 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T20:55:23.769 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:23.853 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:55:23.959 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T20:55:23.959 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:55:23.960 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T20:55:23.960 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:55:23.960 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T20:55:23.960 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:24.017 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:55:24.092 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:55:24.092 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:55:24.093 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T20:55:24.093 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:55:24.093 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:55:24.093 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:24.354 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:55:24.435 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T20:55:24.435 INFO:teuthology.orchestra.run.smithi028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T20:55:24.435 INFO:teuthology.orchestra.run.smithi028.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T20:55:24.435 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:55:24.435 INFO:teuthology.orchestra.run.smithi028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T20:55:24.435 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:29.141 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T20:55:29.185 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T20:55:29.266 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T20:55:29.310 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T20:55:29.359 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T20:55:29.674 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T20:55:29.727 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T20:55:29.729 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T20:55:29.757 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T20:55:29.770 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T20:55:29.846 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T20:55:29.888 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T20:55:29.943 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T20:55:30.247 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T20:55:30.299 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T20:55:30.326 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T20:55:30.656 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:55:30.706 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:55:30.760 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:55:31.247 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:55:31.299 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:55:31.359 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T20:55:32.140 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T20:55:32.273 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T20:55:32.356 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T20:55:32.521 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T20:55:32.562 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T20:55:32.562 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:55:32.688 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:55:32.735 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:55:32.735 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:55:32.764 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T20:55:32.872 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T20:55:32.876 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:55:32.966 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T20:55:33.017 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:55:33.147 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T20:55:33.212 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T20:55:33.212 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:55:33.329 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:55:33.330 INFO:teuthology.orchestra.run.smithi103.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T20:55:33.330 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:33.381 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T20:55:33.428 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:55:33.428 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:55:33.610 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T20:55:33.745 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:55:34.050 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:55:34.050 INFO:teuthology.orchestra.run.smithi028.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T20:55:34.050 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:39.095 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T20:55:39.095 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T20:55:39.095 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T20:55:39.095 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T20:55:39.095 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T20:55:39.096 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T20:55:39.096 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T20:55:39.096 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:55:39.096 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:55:39.096 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:55:39.096 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T20:55:39.096 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T20:55:39.096 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:55:39.096 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T20:55:39.096 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T20:55:39.096 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T20:55:39.096 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T20:55:39.096 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T20:55:39.096 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T20:55:39.097 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T20:55:39.097 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T20:55:39.097 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T20:55:39.097 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T20:55:39.097 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T20:55:39.097 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T20:55:39.097 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T20:55:39.097 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T20:55:39.109 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T20:55:39.109 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T20:55:39.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T20:55:39.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T20:55:39.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T20:55:39.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T20:55:39.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T20:55:39.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T20:55:39.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T20:55:39.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T20:55:39.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T20:55:39.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T20:55:39.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T20:55:39.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T20:55:39.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T20:55:39.110 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T20:55:39.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T20:55:39.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T20:55:39.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T20:55:39.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T20:55:39.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T20:55:39.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T20:55:39.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T20:55:39.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T20:55:39.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T20:55:39.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T20:55:39.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T20:55:39.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T20:55:39.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T20:55:39.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T20:55:39.111 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T20:55:39.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T20:55:39.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T20:55:39.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T20:55:39.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T20:55:39.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T20:55:39.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T20:55:39.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T20:55:39.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T20:55:39.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T20:55:39.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T20:55:39.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T20:55:39.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T20:55:39.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T20:55:39.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T20:55:39.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T20:55:39.112 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T20:55:39.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T20:55:39.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T20:55:39.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T20:55:39.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T20:55:39.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T20:55:39.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T20:55:39.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T20:55:39.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T20:55:39.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T20:55:39.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T20:55:39.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T20:55:39.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T20:55:39.113 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T20:55:39.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T20:55:39.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T20:55:39.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T20:55:39.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T20:55:39.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T20:55:39.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T20:55:39.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T20:55:39.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T20:55:39.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T20:55:39.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T20:55:39.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T20:55:39.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T20:55:39.114 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T20:55:39.115 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T20:55:39.115 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T20:55:39.115 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T20:55:39.115 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T20:55:39.115 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T20:55:39.115 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T20:55:39.115 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T20:55:39.115 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T20:55:39.115 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T20:55:39.115 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T20:55:39.115 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T20:55:39.115 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T20:55:39.115 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T20:55:39.115 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T20:55:39.116 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T20:55:39.116 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T20:55:39.116 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T20:55:39.116 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:55:39.116 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:55:39.116 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T20:55:39.116 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T20:55:39.116 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T20:55:39.116 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T20:55:39.116 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T20:55:39.116 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T20:55:39.116 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T20:55:39.116 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T20:55:39.116 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T20:55:39.116 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T20:55:39.117 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T20:55:39.117 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T20:55:39.117 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T20:55:39.117 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T20:55:39.117 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T20:55:39.117 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T20:55:39.117 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T20:55:39.117 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T20:55:39.117 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T20:55:39.117 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T20:55:39.117 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T20:55:39.117 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T20:55:39.117 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T20:55:39.117 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T20:55:39.117 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T20:55:39.118 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T20:55:39.118 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T20:55:39.118 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T20:55:39.118 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T20:55:39.118 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T20:55:39.118 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T20:55:39.118 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T20:55:39.118 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T20:55:39.118 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T20:55:39.118 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:55:39.118 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T20:55:39.516 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T20:55:39.516 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T20:55:39.516 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T20:55:39.516 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T20:55:39.516 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T20:55:39.516 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T20:55:39.516 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T20:55:39.516 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T20:55:39.517 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T20:55:39.517 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T20:55:39.517 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T20:55:39.517 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T20:55:39.517 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T20:55:39.517 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T20:55:39.517 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T20:55:39.517 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T20:55:39.517 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T20:55:39.517 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T20:55:39.517 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T20:55:39.517 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T20:55:39.517 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T20:55:39.518 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T20:55:39.518 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T20:55:39.518 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T20:55:39.518 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T20:55:39.529 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T20:55:39.530 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T20:55:39.530 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T20:55:39.530 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T20:55:39.530 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T20:55:39.530 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T20:55:39.530 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T20:55:39.530 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T20:55:39.530 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T20:55:39.530 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T20:55:39.530 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T20:55:39.530 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T20:55:39.530 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T20:55:39.531 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T20:55:39.531 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T20:55:39.531 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T20:55:39.531 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T20:55:39.531 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T20:55:39.531 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T20:55:39.531 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T20:55:39.531 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T20:55:39.531 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T20:55:39.531 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T20:55:39.531 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T20:55:39.531 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T20:55:39.531 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T20:55:39.531 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T20:55:39.532 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T20:55:39.532 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T20:55:39.532 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T20:55:39.532 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T20:55:39.532 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T20:55:39.532 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T20:55:39.532 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T20:55:39.532 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T20:55:39.532 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T20:55:39.532 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T20:55:39.532 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T20:55:39.532 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T20:55:39.532 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T20:55:39.532 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T20:55:39.533 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T20:55:39.533 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T20:55:39.533 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T20:55:39.533 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T20:55:39.533 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T20:55:39.533 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T20:55:39.533 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T20:55:39.533 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T20:55:39.533 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T20:55:39.533 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T20:55:39.533 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T20:55:39.533 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T20:55:39.533 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T20:55:39.534 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T20:55:39.534 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T20:55:39.534 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T20:55:39.534 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T20:55:39.534 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T20:55:39.534 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T20:55:39.534 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T20:55:39.534 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T20:55:39.534 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T20:55:39.534 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T20:55:39.534 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T20:55:39.534 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T20:55:39.534 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T20:55:39.534 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T20:55:39.535 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T20:55:39.535 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T20:55:39.535 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T20:55:39.535 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T20:55:39.535 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T20:55:39.535 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T20:55:39.535 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T20:55:39.535 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T20:55:39.535 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T20:55:39.535 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T20:55:39.535 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T20:55:39.535 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T20:55:39.535 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T20:55:39.536 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T20:55:39.536 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T20:55:39.536 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T20:55:39.536 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T20:55:39.536 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T20:55:39.536 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T20:55:39.536 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T20:55:39.536 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T20:55:39.536 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T20:55:39.536 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T20:55:39.536 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T20:55:39.536 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T20:55:39.536 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T20:55:39.536 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T20:55:39.536 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T20:55:39.536 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T20:55:39.537 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T20:55:39.537 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T20:55:39.537 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T20:55:39.537 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T20:55:39.537 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T20:55:39.537 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T20:55:39.537 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T20:55:39.537 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T20:55:39.537 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T20:55:39.537 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T20:55:39.537 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T20:55:39.537 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T20:55:39.537 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T20:55:39.538 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T20:55:39.538 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T20:55:39.538 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T20:55:39.538 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T20:55:39.538 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T20:55:39.538 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T20:55:39.538 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T20:55:39.538 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T20:55:39.538 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T20:55:39.538 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T20:55:39.538 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T20:55:39.538 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T20:55:39.538 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T20:55:39.539 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T20:55:39.539 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T20:55:39.539 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T20:55:39.539 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T20:55:39.539 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T20:55:39.539 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T20:55:39.539 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T20:55:41.510 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:55:41.510 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:41.510 INFO:teuthology.orchestra.run.smithi103.stdout:Upgraded: 2024-04-26T20:55:41.510 INFO:teuthology.orchestra.run.smithi103.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.510 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.510 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T20:55:41.510 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T20:55:41.510 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-04-26T20:55:41.510 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T20:55:41.510 INFO:teuthology.orchestra.run.smithi103.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T20:55:41.510 INFO:teuthology.orchestra.run.smithi103.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T20:55:41.511 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T20:55:41.511 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.511 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.511 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.511 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.511 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:55:41.511 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.511 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.511 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.511 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:55:41.511 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:55:41.511 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:55:41.511 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:55:41.511 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:55:41.512 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.512 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.512 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:55:41.512 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.512 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.512 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.512 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:55:41.512 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:55:41.512 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.512 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.512 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:55:41.512 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T20:55:41.512 INFO:teuthology.orchestra.run.smithi103.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T20:55:41.513 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T20:55:41.513 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T20:55:41.513 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T20:55:41.513 INFO:teuthology.orchestra.run.smithi103.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T20:55:41.513 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T20:55:41.513 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T20:55:41.513 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T20:55:41.513 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T20:55:41.513 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T20:55:41.513 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.513 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.513 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.513 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T20:55:41.513 INFO:teuthology.orchestra.run.smithi103.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T20:55:41.514 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T20:55:41.514 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T20:55:41.514 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T20:55:41.514 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.514 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.514 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T20:55:41.514 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.514 INFO:teuthology.orchestra.run.smithi103.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T20:55:41.514 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T20:55:41.514 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T20:55:41.514 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T20:55:41.514 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T20:55:41.514 INFO:teuthology.orchestra.run.smithi103.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T20:55:41.515 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T20:55:41.515 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T20:55:41.515 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T20:55:41.515 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T20:55:41.515 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T20:55:41.515 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T20:55:41.515 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T20:55:41.515 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T20:55:41.515 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T20:55:41.515 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T20:55:41.515 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T20:55:41.515 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T20:55:41.515 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T20:55:41.515 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T20:55:41.515 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T20:55:41.515 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T20:55:41.516 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T20:55:41.516 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.516 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.516 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.516 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T20:55:41.516 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T20:55:41.516 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T20:55:41.516 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T20:55:41.516 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T20:55:41.516 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T20:55:41.516 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T20:55:41.516 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T20:55:41.516 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T20:55:41.516 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T20:55:41.516 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T20:55:41.517 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T20:55:41.517 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T20:55:41.517 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T20:55:41.517 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T20:55:41.517 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T20:55:41.517 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T20:55:41.517 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T20:55:41.517 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T20:55:41.517 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T20:55:41.517 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T20:55:41.517 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T20:55:41.517 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T20:55:41.517 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T20:55:41.518 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T20:55:41.518 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T20:55:41.518 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T20:55:41.518 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T20:55:41.518 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T20:55:41.518 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T20:55:41.518 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T20:55:41.518 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T20:55:41.518 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T20:55:41.518 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T20:55:41.518 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T20:55:41.518 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T20:55:41.518 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T20:55:41.518 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T20:55:41.519 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T20:55:41.519 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T20:55:41.519 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.519 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.519 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T20:55:41.519 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T20:55:41.519 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T20:55:41.519 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.519 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T20:55:41.519 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T20:55:41.519 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T20:55:41.519 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T20:55:41.519 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T20:55:41.519 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T20:55:41.519 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T20:55:41.519 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T20:55:41.520 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T20:55:41.520 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T20:55:41.520 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T20:55:41.520 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T20:55:41.520 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T20:55:41.520 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T20:55:41.520 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T20:55:41.520 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.520 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.520 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:41.520 INFO:teuthology.orchestra.run.smithi103.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T20:55:41.520 INFO:teuthology.orchestra.run.smithi103.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T20:55:41.520 INFO:teuthology.orchestra.run.smithi103.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T20:55:41.521 INFO:teuthology.orchestra.run.smithi103.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T20:55:41.521 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T20:55:41.521 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T20:55:41.521 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T20:55:41.521 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:41.521 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-04-26T20:55:42.149 DEBUG:teuthology.parallel:result is None 2024-04-26T20:55:42.163 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T20:55:42.163 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:42.163 INFO:teuthology.orchestra.run.smithi028.stdout:Upgraded: 2024-04-26T20:55:42.163 INFO:teuthology.orchestra.run.smithi028.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.163 INFO:teuthology.orchestra.run.smithi028.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.163 INFO:teuthology.orchestra.run.smithi028.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T20:55:42.163 INFO:teuthology.orchestra.run.smithi028.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T20:55:42.164 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2024-04-26T20:55:42.164 INFO:teuthology.orchestra.run.smithi028.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T20:55:42.164 INFO:teuthology.orchestra.run.smithi028.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T20:55:42.164 INFO:teuthology.orchestra.run.smithi028.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T20:55:42.164 INFO:teuthology.orchestra.run.smithi028.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T20:55:42.164 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.164 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.164 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.164 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.164 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:55:42.164 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.164 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.164 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.164 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:55:42.165 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:55:42.165 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:55:42.165 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:55:42.165 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:55:42.165 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.165 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.165 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:55:42.165 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.165 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.165 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.165 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:55:42.165 INFO:teuthology.orchestra.run.smithi028.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:55:42.165 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.165 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.165 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T20:55:42.166 INFO:teuthology.orchestra.run.smithi028.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T20:55:42.166 INFO:teuthology.orchestra.run.smithi028.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T20:55:42.166 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T20:55:42.166 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T20:55:42.166 INFO:teuthology.orchestra.run.smithi028.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T20:55:42.166 INFO:teuthology.orchestra.run.smithi028.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T20:55:42.166 INFO:teuthology.orchestra.run.smithi028.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T20:55:42.166 INFO:teuthology.orchestra.run.smithi028.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T20:55:42.166 INFO:teuthology.orchestra.run.smithi028.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T20:55:42.166 INFO:teuthology.orchestra.run.smithi028.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T20:55:42.166 INFO:teuthology.orchestra.run.smithi028.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T20:55:42.166 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.166 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.166 INFO:teuthology.orchestra.run.smithi028.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.167 INFO:teuthology.orchestra.run.smithi028.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T20:55:42.167 INFO:teuthology.orchestra.run.smithi028.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T20:55:42.167 INFO:teuthology.orchestra.run.smithi028.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T20:55:42.167 INFO:teuthology.orchestra.run.smithi028.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T20:55:42.167 INFO:teuthology.orchestra.run.smithi028.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T20:55:42.167 INFO:teuthology.orchestra.run.smithi028.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.167 INFO:teuthology.orchestra.run.smithi028.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.167 INFO:teuthology.orchestra.run.smithi028.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T20:55:42.167 INFO:teuthology.orchestra.run.smithi028.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.167 INFO:teuthology.orchestra.run.smithi028.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T20:55:42.167 INFO:teuthology.orchestra.run.smithi028.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T20:55:42.167 INFO:teuthology.orchestra.run.smithi028.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T20:55:42.167 INFO:teuthology.orchestra.run.smithi028.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T20:55:42.167 INFO:teuthology.orchestra.run.smithi028.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T20:55:42.168 INFO:teuthology.orchestra.run.smithi028.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T20:55:42.168 INFO:teuthology.orchestra.run.smithi028.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T20:55:42.168 INFO:teuthology.orchestra.run.smithi028.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T20:55:42.168 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T20:55:42.168 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T20:55:42.168 INFO:teuthology.orchestra.run.smithi028.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T20:55:42.168 INFO:teuthology.orchestra.run.smithi028.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T20:55:42.168 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T20:55:42.168 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T20:55:42.168 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T20:55:42.168 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T20:55:42.168 INFO:teuthology.orchestra.run.smithi028.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T20:55:42.168 INFO:teuthology.orchestra.run.smithi028.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T20:55:42.168 INFO:teuthology.orchestra.run.smithi028.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T20:55:42.168 INFO:teuthology.orchestra.run.smithi028.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T20:55:42.169 INFO:teuthology.orchestra.run.smithi028.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T20:55:42.169 INFO:teuthology.orchestra.run.smithi028.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T20:55:42.169 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T20:55:42.169 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.169 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.169 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.169 INFO:teuthology.orchestra.run.smithi028.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T20:55:42.169 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T20:55:42.169 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T20:55:42.169 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T20:55:42.169 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T20:55:42.169 INFO:teuthology.orchestra.run.smithi028.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T20:55:42.169 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T20:55:42.169 INFO:teuthology.orchestra.run.smithi028.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T20:55:42.169 INFO:teuthology.orchestra.run.smithi028.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T20:55:42.169 INFO:teuthology.orchestra.run.smithi028.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T20:55:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T20:55:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T20:55:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T20:55:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T20:55:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T20:55:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T20:55:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T20:55:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T20:55:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T20:55:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T20:55:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T20:55:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T20:55:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T20:55:42.170 INFO:teuthology.orchestra.run.smithi028.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T20:55:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T20:55:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T20:55:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T20:55:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T20:55:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T20:55:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T20:55:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T20:55:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T20:55:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T20:55:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T20:55:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T20:55:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T20:55:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T20:55:42.171 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T20:55:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T20:55:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T20:55:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T20:55:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T20:55:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T20:55:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T20:55:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T20:55:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T20:55:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T20:55:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T20:55:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T20:55:42.172 INFO:teuthology.orchestra.run.smithi028.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T20:55:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T20:55:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T20:55:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T20:55:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T20:55:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T20:55:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T20:55:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T20:55:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T20:55:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T20:55:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T20:55:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T20:55:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T20:55:42.173 INFO:teuthology.orchestra.run.smithi028.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T20:55:42.174 INFO:teuthology.orchestra.run.smithi028.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T20:55:42.174 INFO:teuthology.orchestra.run.smithi028.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T20:55:42.174 INFO:teuthology.orchestra.run.smithi028.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T20:55:42.174 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:42.174 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2024-04-26T20:55:42.754 DEBUG:teuthology.parallel:result is None 2024-04-26T20:55:42.755 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:55:42.972 DEBUG:teuthology.orchestra.run.smithi028:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T20:55:43.034 INFO:teuthology.orchestra.run.smithi028.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T20:55:43.035 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T20:55:43.035 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T20:55:43.037 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T20:55:43.179 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T20:55:43.236 INFO:teuthology.orchestra.run.smithi103.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T20:55:43.237 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T20:55:43.237 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T20:55:43.238 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T20:55:43.239 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:55:43.239 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T20:55:43.308 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:55:43.308 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T20:55:43.377 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T20:55:43.378 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:55:43.378 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T20:55:43.455 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T20:55:43.525 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:55:43.525 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T20:55:43.594 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T20:55:43.703 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T20:55:43.703 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:55:43.703 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T20:55:43.766 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T20:55:43.867 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:55:43.867 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T20:55:43.932 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T20:55:44.040 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T20:55:44.040 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:55:44.040 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T20:55:44.106 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T20:55:44.210 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:55:44.210 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T20:55:44.277 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T20:55:44.382 INFO:teuthology.run_tasks:Running task ceph... 2024-04-26T20:55:44.471 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-26T20:55:44.471 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 777 /var/log/ceph 2024-04-26T20:55:44.475 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 777 /var/log/ceph 2024-04-26T20:55:44.542 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-26T20:55:44.542 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T20:55:44.546 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T20:55:44.609 INFO:tasks.ceph:Creating extra log directories... 2024-04-26T20:55:44.610 DEBUG:teuthology.orchestra.run.smithi028:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T20:55:44.613 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T20:55:44.678 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-26T20:55:44.678 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-26T20:55:44.678 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-26T20:55:44.678 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:55:44.679 DEBUG:teuthology.orchestra.run.smithi028:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T20:55:44.717 DEBUG:teuthology.orchestra.run.smithi028:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T20:55:44.895 DEBUG:teuthology.orchestra.run.smithi028:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T20:55:44.957 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:55:44.957 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T20:55:44.996 DEBUG:teuthology.orchestra.run.smithi103:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T20:55:45.170 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T20:55:45.234 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-26T20:55:45.234 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-26T20:55:45.234 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675182', 'branch': 'wip-rishabh-testing-20240426.111959', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/asok_dump_tree}', 'email': None, 'first_in_suite': False, 'job_id': '7675182', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240426.111959'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240426.111959', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 816, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'sleep_before_teardown': 0, 'subset': '166/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240426.111959', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi028.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIK+zg/JWra1uD612P625cNOww2OHY02pxIF8/9BwvxAKT3XK7tUAMyzrzCJUhku57hQKvrn2sqmYng7u9kbJ64=', 'smithi103.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEEKkQDpg0Qr3CZiGw53Rqbpec5xVz6/xjXT8sCdl24BbF/tT3cu5+b/frYYHVCRg/0ymyHAxlTesvOs1Y95tMc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_dump_tree']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-26_19:30:57', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-26T20:55:45.235 DEBUG:teuthology.orchestra.run.smithi028:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T20:55:45.240 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T20:55:45.276 DEBUG:teuthology.orchestra.run.smithi028:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T20:55:45.320 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T20:55:45.382 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:55:45.382 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T20:55:45.426 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T20:55:45.426 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_1 2024-04-26T20:55:45.521 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T20:55:45.521 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:55:45.521 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-26T20:55:45.521 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:55:45.521 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:55:45.521 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-04-26 20:55:34.537886161 +0000 2024-04-26T20:55:45.521 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-04-26 20:53:11.132785780 +0000 2024-04-26T20:55:45.521 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-04-26 20:53:11.132785780 +0000 2024-04-26T20:55:45.521 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-04-26 20:53:11.132785780 +0000 2024-04-26T20:55:45.521 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T20:55:45.592 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-04-26T20:55:45.592 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-04-26T20:55:45.592 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.000306445 s, 1.7 MB/s 2024-04-26T20:55:45.595 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T20:55:45.638 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_2 2024-04-26T20:55:45.716 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T20:55:45.716 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:55:45.717 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-26T20:55:45.717 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:55:45.717 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:55:45.717 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-04-26 20:55:34.537886161 +0000 2024-04-26T20:55:45.717 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-04-26 20:53:11.747768187 +0000 2024-04-26T20:55:45.717 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-04-26 20:53:11.747768187 +0000 2024-04-26T20:55:45.717 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-04-26 20:53:11.747768187 +0000 2024-04-26T20:55:45.717 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T20:55:45.816 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-04-26T20:55:45.816 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-04-26T20:55:45.816 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.000190164 s, 2.7 MB/s 2024-04-26T20:55:45.819 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T20:55:45.858 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_3 2024-04-26T20:55:45.937 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T20:55:45.937 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:55:45.937 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-26T20:55:45.937 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:55:45.937 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:55:45.937 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-04-26 20:55:34.538886132 +0000 2024-04-26T20:55:45.937 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-04-26 20:53:12.366750480 +0000 2024-04-26T20:55:45.937 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-04-26 20:53:12.366750480 +0000 2024-04-26T20:55:45.938 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-04-26 20:53:12.366750480 +0000 2024-04-26T20:55:45.938 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T20:55:46.036 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-04-26T20:55:46.036 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-04-26T20:55:46.036 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.000258887 s, 2.0 MB/s 2024-04-26T20:55:46.039 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T20:55:46.080 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_4 2024-04-26T20:55:46.158 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T20:55:46.159 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:55:46.159 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-04-26T20:55:46.159 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:55:46.159 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:55:46.159 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2024-04-26 20:55:34.539886103 +0000 2024-04-26T20:55:46.159 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2024-04-26 20:53:12.998732402 +0000 2024-04-26T20:55:46.159 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2024-04-26 20:53:12.998732402 +0000 2024-04-26T20:55:46.159 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: 2024-04-26 20:53:12.998732402 +0000 2024-04-26T20:55:46.159 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T20:55:46.258 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2024-04-26T20:55:46.258 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2024-04-26T20:55:46.258 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.000195619 s, 2.6 MB/s 2024-04-26T20:55:46.261 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T20:55:46.302 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T20:55:46.302 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:55:46.302 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T20:55:46.343 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T20:55:46.343 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_1 2024-04-26T20:55:46.424 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T20:55:46.424 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:55:46.424 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-26T20:55:46.424 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:55:46.424 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:55:46.424 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-04-26 20:55:33.779412284 +0000 2024-04-26T20:55:46.424 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-04-26 20:53:07.827324626 +0000 2024-04-26T20:55:46.424 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-04-26 20:53:07.827324626 +0000 2024-04-26T20:55:46.424 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-04-26 20:53:07.827324626 +0000 2024-04-26T20:55:46.424 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T20:55:46.522 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-04-26T20:55:46.522 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-04-26T20:55:46.522 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000332744 s, 1.5 MB/s 2024-04-26T20:55:46.525 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T20:55:46.564 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_2 2024-04-26T20:55:46.645 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T20:55:46.646 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:55:46.646 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 653 Links: 1 2024-04-26T20:55:46.646 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:55:46.646 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:55:46.646 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-04-26 20:55:33.780412265 +0000 2024-04-26T20:55:46.646 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-04-26 20:53:08.419312939 +0000 2024-04-26T20:55:46.646 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-04-26 20:53:08.419312939 +0000 2024-04-26T20:55:46.646 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-04-26 20:53:08.419312939 +0000 2024-04-26T20:55:46.646 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T20:55:46.746 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-04-26T20:55:46.746 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-04-26T20:55:46.746 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000279252 s, 1.8 MB/s 2024-04-26T20:55:46.748 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T20:55:46.788 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_3 2024-04-26T20:55:46.866 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T20:55:46.866 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:55:46.866 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-04-26T20:55:46.866 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:55:46.866 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:55:46.866 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-04-26 20:55:33.781412245 +0000 2024-04-26T20:55:46.866 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-04-26 20:53:09.016301153 +0000 2024-04-26T20:55:46.866 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-04-26 20:53:09.016301153 +0000 2024-04-26T20:55:46.866 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-04-26 20:53:09.016301153 +0000 2024-04-26T20:55:46.867 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T20:55:46.964 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-04-26T20:55:46.965 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-04-26T20:55:46.965 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000340282 s, 1.5 MB/s 2024-04-26T20:55:46.967 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T20:55:47.006 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_4 2024-04-26T20:55:47.085 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T20:55:47.085 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T20:55:47.085 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-26T20:55:47.085 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T20:55:47.085 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T20:55:47.085 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-04-26 20:55:33.781412245 +0000 2024-04-26T20:55:47.085 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-04-26 20:53:09.605289526 +0000 2024-04-26T20:55:47.085 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-04-26 20:53:09.605289526 +0000 2024-04-26T20:55:47.085 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-04-26 20:53:09.605289526 +0000 2024-04-26T20:55:47.085 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T20:55:47.188 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-04-26T20:55:47.188 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-04-26T20:55:47.188 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000287951 s, 1.8 MB/s 2024-04-26T20:55:47.192 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T20:55:47.233 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T20:55:47.233 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi028.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi103.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T20:55:47.233 INFO:tasks.ceph:Generating config... 2024-04-26T20:55:47.234 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-26T20:55:47.235 INFO:tasks.ceph:[client] debug client = 20 2024-04-26T20:55:47.235 INFO:tasks.ceph:[client] debug ms = 1 2024-04-26T20:55:47.235 INFO:tasks.ceph:[client] pid file = 2024-04-26T20:55:47.235 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-26T20:55:47.235 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-26T20:55:47.235 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-26T20:55:47.235 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-26T20:55:47.235 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-26T20:55:47.235 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-26T20:55:47.235 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-26T20:55:47.235 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-26T20:55:47.235 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-26T20:55:47.235 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-26T20:55:47.236 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-26T20:55:47.236 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-26T20:55:47.236 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-26T20:55:47.236 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-26T20:55:47.236 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-26T20:55:47.236 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-26T20:55:47.236 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-26T20:55:47.236 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-26T20:55:47.236 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-26T20:55:47.236 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-26T20:55:47.236 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-26T20:55:47.236 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-26T20:55:47.236 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-26T20:55:47.236 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-26T20:55:47.236 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-26T20:55:47.237 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-26T20:55:47.237 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-26T20:55:47.237 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-26T20:55:47.237 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-26T20:55:47.237 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-26T20:55:47.237 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-26T20:55:47.237 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-26T20:55:47.237 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-26T20:55:47.237 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-26T20:55:47.237 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-26T20:55:47.237 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-26T20:55:47.237 INFO:tasks.ceph:Setting up mon.a... 2024-04-26T20:55:47.238 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-26T20:55:47.369 INFO:teuthology.orchestra.run.smithi028.stdout:creating /etc/ceph/ceph.keyring 2024-04-26T20:55:47.376 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-26T20:55:47.473 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T20:55:47.543 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.28'), ('mon.b', '172.21.15.103'), ('mon.c', '[v2:172.21.15.103:3301,v1:172.21.15.103:6790]')] 2024-04-26T20:55:47.544 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.28,172.21.15.103,[v2:172.21.15.103:3301,v1:172.21.15.103:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-26T20:55:47.545 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:55:47.545 DEBUG:teuthology.orchestra.run.smithi028:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T20:55:47.586 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.28 --add b 172.21.15.103 --addv c '[v2:172.21.15.103:3301,v1:172.21.15.103:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:55:47.713 INFO:teuthology.orchestra.run.smithi028.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:55:47.714 INFO:teuthology.orchestra.run.smithi028.stdout:monmaptool: generated fsid 79845bd7-2c94-4fa6-8082-292285d530c9 2024-04-26T20:55:47.714 INFO:teuthology.orchestra.run.smithi028.stdout:setting min_mon_release = quincy 2024-04-26T20:55:47.714 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 0 2024-04-26T20:55:47.714 INFO:teuthology.orchestra.run.smithi028.stdout:fsid 79845bd7-2c94-4fa6-8082-292285d530c9 2024-04-26T20:55:47.714 INFO:teuthology.orchestra.run.smithi028.stdout:last_changed 2024-04-26T20:55:47.713921+0000 2024-04-26T20:55:47.714 INFO:teuthology.orchestra.run.smithi028.stdout:created 2024-04-26T20:55:47.713921+0000 2024-04-26T20:55:47.714 INFO:teuthology.orchestra.run.smithi028.stdout:min_mon_release 17 (quincy) 2024-04-26T20:55:47.714 INFO:teuthology.orchestra.run.smithi028.stdout:election_strategy: 1 2024-04-26T20:55:47.714 INFO:teuthology.orchestra.run.smithi028.stdout:0: [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon.a 2024-04-26T20:55:47.714 INFO:teuthology.orchestra.run.smithi028.stdout:1: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.b 2024-04-26T20:55:47.714 INFO:teuthology.orchestra.run.smithi028.stdout:2: [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] mon.c 2024-04-26T20:55:47.714 INFO:teuthology.orchestra.run.smithi028.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-26T20:55:47.717 DEBUG:teuthology.orchestra.run.smithi028:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T20:55:47.759 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 79845bd7-2c94-4fa6-8082-292285d530c9... 2024-04-26T20:55:47.760 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T20:55:47.805 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T20:55:47.918 INFO:teuthology.orchestra.run.smithi028.stdout:[global] 2024-04-26T20:55:47.918 INFO:teuthology.orchestra.run.smithi028.stdout: chdir = "" 2024-04-26T20:55:47.918 INFO:teuthology.orchestra.run.smithi028.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T20:55:47.918 INFO:teuthology.orchestra.run.smithi028.stdout: auth supported = cephx 2024-04-26T20:55:47.918 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.918 INFO:teuthology.orchestra.run.smithi028.stdout: filestore xattr use omap = true 2024-04-26T20:55:47.918 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.919 INFO:teuthology.orchestra.run.smithi028.stdout: mon clock drift allowed = 1.000 2024-04-26T20:55:47.919 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.919 INFO:teuthology.orchestra.run.smithi028.stdout: osd crush chooseleaf type = 0 2024-04-26T20:55:47.919 INFO:teuthology.orchestra.run.smithi028.stdout: auth debug = true 2024-04-26T20:55:47.919 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.919 INFO:teuthology.orchestra.run.smithi028.stdout: ms die on old message = true 2024-04-26T20:55:47.919 INFO:teuthology.orchestra.run.smithi028.stdout: ms die on bug = true 2024-04-26T20:55:47.919 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.919 INFO:teuthology.orchestra.run.smithi028.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T20:55:47.919 INFO:teuthology.orchestra.run.smithi028.stdout: mon pg warn max object skew = 0 2024-04-26T20:55:47.919 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.919 INFO:teuthology.orchestra.run.smithi028.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T20:55:47.919 INFO:teuthology.orchestra.run.smithi028.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T20:55:47.920 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.920 INFO:teuthology.orchestra.run.smithi028.stdout: osd pool default size = 2 2024-04-26T20:55:47.920 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.920 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd allow primary affinity = true 2024-04-26T20:55:47.920 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd allow pg remap = true 2024-04-26T20:55:47.920 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on legacy crush tunables = false 2024-04-26T20:55:47.920 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on crush straw calc version zero = false 2024-04-26T20:55:47.920 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on no sortbitwise = false 2024-04-26T20:55:47.920 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on osd down out interval zero = false 2024-04-26T20:55:47.920 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on too few osds = false 2024-04-26T20:55:47.920 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T20:55:47.920 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T20:55:47.920 INFO:teuthology.orchestra.run.smithi028.stdout: mon_allow_pool_size_one = true 2024-04-26T20:55:47.920 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.920 INFO:teuthology.orchestra.run.smithi028.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T20:55:47.921 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.921 INFO:teuthology.orchestra.run.smithi028.stdout: osd default data pool replay window = 5 2024-04-26T20:55:47.921 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.921 INFO:teuthology.orchestra.run.smithi028.stdout: mon allow pool delete = true 2024-04-26T20:55:47.921 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.921 INFO:teuthology.orchestra.run.smithi028.stdout: mon cluster log file level = debug 2024-04-26T20:55:47.921 INFO:teuthology.orchestra.run.smithi028.stdout: debug asserts on shutdown = true 2024-04-26T20:55:47.921 INFO:teuthology.orchestra.run.smithi028.stdout: mon health detail to clog = false 2024-04-26T20:55:47.921 INFO:teuthology.orchestra.run.smithi028.stdout: mon host = "172.21.15.28,172.21.15.103,[v2:172.21.15.103:3301,v1:172.21.15.103:6790]" 2024-04-26T20:55:47.921 INFO:teuthology.orchestra.run.smithi028.stdout: fsid = 79845bd7-2c94-4fa6-8082-292285d530c9 2024-04-26T20:55:47.921 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.921 INFO:teuthology.orchestra.run.smithi028.stdout:[osd] 2024-04-26T20:55:47.921 INFO:teuthology.orchestra.run.smithi028.stdout: osd journal size = 100 2024-04-26T20:55:47.921 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.921 INFO:teuthology.orchestra.run.smithi028.stdout: osd scrub load threshold = 5.0 2024-04-26T20:55:47.922 INFO:teuthology.orchestra.run.smithi028.stdout: osd scrub max interval = 600 2024-04-26T20:55:47.922 INFO:teuthology.orchestra.run.smithi028.stdout: osd mclock profile = high_recovery_ops 2024-04-26T20:55:47.922 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.922 INFO:teuthology.orchestra.run.smithi028.stdout: osd recover clone overlap = true 2024-04-26T20:55:47.922 INFO:teuthology.orchestra.run.smithi028.stdout: osd recovery max chunk = 1048576 2024-04-26T20:55:47.922 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.922 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug shutdown = true 2024-04-26T20:55:47.922 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug op order = true 2024-04-26T20:55:47.922 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug verify stray on activate = true 2024-04-26T20:55:47.922 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug trim objects = true 2024-04-26T20:55:47.922 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.922 INFO:teuthology.orchestra.run.smithi028.stdout: osd open classes on start = true 2024-04-26T20:55:47.922 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug pg log writeout = true 2024-04-26T20:55:47.922 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.922 INFO:teuthology.orchestra.run.smithi028.stdout: osd deep scrub update digest min age = 30 2024-04-26T20:55:47.922 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.923 INFO:teuthology.orchestra.run.smithi028.stdout: osd map max advance = 10 2024-04-26T20:55:47.923 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.923 INFO:teuthology.orchestra.run.smithi028.stdout: journal zero on create = true 2024-04-26T20:55:47.923 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.923 INFO:teuthology.orchestra.run.smithi028.stdout: filestore ondisk finisher threads = 3 2024-04-26T20:55:47.923 INFO:teuthology.orchestra.run.smithi028.stdout: filestore apply finisher threads = 3 2024-04-26T20:55:47.923 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.923 INFO:teuthology.orchestra.run.smithi028.stdout: bdev debug aio = true 2024-04-26T20:55:47.923 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug misdirected ops = true 2024-04-26T20:55:47.923 INFO:teuthology.orchestra.run.smithi028.stdout: bdev async discard = True 2024-04-26T20:55:47.923 INFO:teuthology.orchestra.run.smithi028.stdout: bdev enable discard = True 2024-04-26T20:55:47.923 INFO:teuthology.orchestra.run.smithi028.stdout: bluestore allocator = bitmap 2024-04-26T20:55:47.923 INFO:teuthology.orchestra.run.smithi028.stdout: bluestore block size = 96636764160 2024-04-26T20:55:47.923 INFO:teuthology.orchestra.run.smithi028.stdout: bluestore fsck on mount = True 2024-04-26T20:55:47.923 INFO:teuthology.orchestra.run.smithi028.stdout: debug bluefs = 1/20 2024-04-26T20:55:47.923 INFO:teuthology.orchestra.run.smithi028.stdout: debug bluestore = 1/20 2024-04-26T20:55:47.924 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2024-04-26T20:55:47.924 INFO:teuthology.orchestra.run.smithi028.stdout: debug osd = 20 2024-04-26T20:55:47.924 INFO:teuthology.orchestra.run.smithi028.stdout: debug rocksdb = 4/10 2024-04-26T20:55:47.924 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T20:55:47.924 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd full ratio = 0.9 2024-04-26T20:55:47.924 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd nearfull ratio = 0.8 2024-04-26T20:55:47.924 INFO:teuthology.orchestra.run.smithi028.stdout: osd failsafe full ratio = 0.95 2024-04-26T20:55:47.924 INFO:teuthology.orchestra.run.smithi028.stdout: osd objectstore = bluestore 2024-04-26T20:55:47.924 INFO:teuthology.orchestra.run.smithi028.stdout: osd op complaint time = 180 2024-04-26T20:55:47.924 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.924 INFO:teuthology.orchestra.run.smithi028.stdout:[mgr] 2024-04-26T20:55:47.924 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2024-04-26T20:55:47.924 INFO:teuthology.orchestra.run.smithi028.stdout: debug mgr = 20 2024-04-26T20:55:47.924 INFO:teuthology.orchestra.run.smithi028.stdout: debug mon = 20 2024-04-26T20:55:47.924 INFO:teuthology.orchestra.run.smithi028.stdout: debug auth = 20 2024-04-26T20:55:47.925 INFO:teuthology.orchestra.run.smithi028.stdout: mon reweight min pgs per osd = 4 2024-04-26T20:55:47.925 INFO:teuthology.orchestra.run.smithi028.stdout: mon reweight min bytes per osd = 10 2024-04-26T20:55:47.925 INFO:teuthology.orchestra.run.smithi028.stdout: mgr/telemetry/nag = false 2024-04-26T20:55:47.925 INFO:teuthology.orchestra.run.smithi028.stdout: debug client = 20 2024-04-26T20:55:47.925 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.925 INFO:teuthology.orchestra.run.smithi028.stdout:[mon] 2024-04-26T20:55:47.925 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2024-04-26T20:55:47.925 INFO:teuthology.orchestra.run.smithi028.stdout: debug mon = 20 2024-04-26T20:55:47.925 INFO:teuthology.orchestra.run.smithi028.stdout: debug paxos = 20 2024-04-26T20:55:47.925 INFO:teuthology.orchestra.run.smithi028.stdout: debug auth = 20 2024-04-26T20:55:47.925 INFO:teuthology.orchestra.run.smithi028.stdout: mon data avail warn = 5 2024-04-26T20:55:47.925 INFO:teuthology.orchestra.run.smithi028.stdout: mon mgr mkfs grace = 240 2024-04-26T20:55:47.925 INFO:teuthology.orchestra.run.smithi028.stdout: mon reweight min pgs per osd = 4 2024-04-26T20:55:47.925 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd reporter subtree level = osd 2024-04-26T20:55:47.925 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd prime pg temp = true 2024-04-26T20:55:47.925 INFO:teuthology.orchestra.run.smithi028.stdout: mon reweight min bytes per osd = 10 2024-04-26T20:55:47.926 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.926 INFO:teuthology.orchestra.run.smithi028.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T20:55:47.926 INFO:teuthology.orchestra.run.smithi028.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T20:55:47.926 INFO:teuthology.orchestra.run.smithi028.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T20:55:47.926 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.926 INFO:teuthology.orchestra.run.smithi028.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T20:55:47.926 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T20:55:47.926 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T20:55:47.926 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.926 INFO:teuthology.orchestra.run.smithi028.stdout: # 1m isn't quite enough 2024-04-26T20:55:47.926 INFO:teuthology.orchestra.run.smithi028.stdout: mon_down_mkfs_grace = 2m 2024-04-26T20:55:47.926 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.926 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_filestore_osds = false 2024-04-26T20:55:47.926 INFO:teuthology.orchestra.run.smithi028.stdout: mon down mkfs grace = 300 2024-04-26T20:55:47.926 INFO:teuthology.orchestra.run.smithi028.stdout: mon op complaint time = 120 2024-04-26T20:55:47.927 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:55:47.927 INFO:teuthology.orchestra.run.smithi028.stdout:[client] 2024-04-26T20:55:47.927 INFO:teuthology.orchestra.run.smithi028.stdout: rgw cache enabled = true 2024-04-26T20:55:47.927 INFO:teuthology.orchestra.run.smithi028.stdout: rgw enable ops log = true 2024-04-26T20:55:47.927 INFO:teuthology.orchestra.run.smithi028.stdout: rgw enable usage log = true 2024-04-26T20:55:47.927 INFO:teuthology.orchestra.run.smithi028.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T20:55:47.927 INFO:teuthology.orchestra.run.smithi028.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T20:55:47.927 INFO:teuthology.orchestra.run.smithi028.stdout: client mount timeout = 600 2024-04-26T20:55:47.927 INFO:teuthology.orchestra.run.smithi028.stdout: debug client = 20 2024-04-26T20:55:47.927 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2024-04-26T20:55:47.927 INFO:teuthology.orchestra.run.smithi028.stdout: pid file = "" 2024-04-26T20:55:47.927 INFO:teuthology.orchestra.run.smithi028.stdout: rados mon op timeout = 900 2024-04-26T20:55:47.927 INFO:teuthology.orchestra.run.smithi028.stdout: rados osd op timeout = 900 2024-04-26T20:55:47.927 INFO:teuthology.orchestra.run.smithi028.stdout:[mon.a] 2024-04-26T20:55:47.927 INFO:teuthology.orchestra.run.smithi028.stdout:[mon.b] 2024-04-26T20:55:47.928 INFO:teuthology.orchestra.run.smithi028.stdout:[mon.c] 2024-04-26T20:55:47.928 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.a] 2024-04-26T20:55:47.928 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.c] 2024-04-26T20:55:47.928 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.b] 2024-04-26T20:55:47.928 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.d] 2024-04-26T20:55:47.928 INFO:teuthology.orchestra.run.smithi028.stdout:[mds] 2024-04-26T20:55:47.928 INFO:teuthology.orchestra.run.smithi028.stdout: debug mds = 20 2024-04-26T20:55:47.928 INFO:teuthology.orchestra.run.smithi028.stdout: debug mds balancer = 20 2024-04-26T20:55:47.928 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2024-04-26T20:55:47.928 INFO:teuthology.orchestra.run.smithi028.stdout: mds debug frag = True 2024-04-26T20:55:47.928 INFO:teuthology.orchestra.run.smithi028.stdout: mds debug scatterstat = True 2024-04-26T20:55:47.928 INFO:teuthology.orchestra.run.smithi028.stdout: mds op complaint time = 180 2024-04-26T20:55:47.928 INFO:teuthology.orchestra.run.smithi028.stdout: mds verify scatter = True 2024-04-26T20:55:47.928 INFO:teuthology.orchestra.run.smithi028.stdout: osd op complaint time = 180 2024-04-26T20:55:47.928 INFO:teuthology.orchestra.run.smithi028.stdout: rados mon op timeout = 900 2024-04-26T20:55:47.928 INFO:teuthology.orchestra.run.smithi028.stdout: rados osd op timeout = 900 2024-04-26T20:55:47.929 INFO:teuthology.orchestra.run.smithi103.stdout:[global] 2024-04-26T20:55:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: chdir = "" 2024-04-26T20:55:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T20:55:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: auth supported = cephx 2024-04-26T20:55:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: filestore xattr use omap = true 2024-04-26T20:55:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: mon clock drift allowed = 1.000 2024-04-26T20:55:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.929 INFO:teuthology.orchestra.run.smithi103.stdout: osd crush chooseleaf type = 0 2024-04-26T20:55:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: auth debug = true 2024-04-26T20:55:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: ms die on old message = true 2024-04-26T20:55:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: ms die on bug = true 2024-04-26T20:55:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T20:55:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: mon pg warn max object skew = 0 2024-04-26T20:55:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T20:55:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T20:55:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: osd pool default size = 2 2024-04-26T20:55:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.930 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd allow primary affinity = true 2024-04-26T20:55:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd allow pg remap = true 2024-04-26T20:55:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: mon warn on legacy crush tunables = false 2024-04-26T20:55:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: mon warn on crush straw calc version zero = false 2024-04-26T20:55:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: mon warn on no sortbitwise = false 2024-04-26T20:55:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: mon warn on osd down out interval zero = false 2024-04-26T20:55:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: mon warn on too few osds = false 2024-04-26T20:55:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T20:55:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T20:55:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: mon_allow_pool_size_one = true 2024-04-26T20:55:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T20:55:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: osd default data pool replay window = 5 2024-04-26T20:55:47.931 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: mon allow pool delete = true 2024-04-26T20:55:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: mon cluster log file level = debug 2024-04-26T20:55:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: debug asserts on shutdown = true 2024-04-26T20:55:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: mon health detail to clog = false 2024-04-26T20:55:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: mon host = "172.21.15.28,172.21.15.103,[v2:172.21.15.103:3301,v1:172.21.15.103:6790]" 2024-04-26T20:55:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: fsid = 79845bd7-2c94-4fa6-8082-292285d530c9 2024-04-26T20:55:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.932 INFO:teuthology.orchestra.run.smithi103.stdout:[osd] 2024-04-26T20:55:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: osd journal size = 100 2024-04-26T20:55:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: osd scrub load threshold = 5.0 2024-04-26T20:55:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: osd scrub max interval = 600 2024-04-26T20:55:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: osd mclock profile = high_recovery_ops 2024-04-26T20:55:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.932 INFO:teuthology.orchestra.run.smithi103.stdout: osd recover clone overlap = true 2024-04-26T20:55:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: osd recovery max chunk = 1048576 2024-04-26T20:55:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug shutdown = true 2024-04-26T20:55:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug op order = true 2024-04-26T20:55:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug verify stray on activate = true 2024-04-26T20:55:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug trim objects = true 2024-04-26T20:55:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: osd open classes on start = true 2024-04-26T20:55:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug pg log writeout = true 2024-04-26T20:55:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: osd deep scrub update digest min age = 30 2024-04-26T20:55:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: osd map max advance = 10 2024-04-26T20:55:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: journal zero on create = true 2024-04-26T20:55:47.933 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: filestore ondisk finisher threads = 3 2024-04-26T20:55:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: filestore apply finisher threads = 3 2024-04-26T20:55:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: bdev debug aio = true 2024-04-26T20:55:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: osd debug misdirected ops = true 2024-04-26T20:55:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: bdev async discard = True 2024-04-26T20:55:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: bdev enable discard = True 2024-04-26T20:55:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: bluestore allocator = bitmap 2024-04-26T20:55:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: bluestore block size = 96636764160 2024-04-26T20:55:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: bluestore fsck on mount = True 2024-04-26T20:55:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: debug bluefs = 1/20 2024-04-26T20:55:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: debug bluestore = 1/20 2024-04-26T20:55:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: debug ms = 1 2024-04-26T20:55:47.934 INFO:teuthology.orchestra.run.smithi103.stdout: debug osd = 20 2024-04-26T20:55:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: debug rocksdb = 4/10 2024-04-26T20:55:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T20:55:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd full ratio = 0.9 2024-04-26T20:55:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd nearfull ratio = 0.8 2024-04-26T20:55:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: osd failsafe full ratio = 0.95 2024-04-26T20:55:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: osd objectstore = bluestore 2024-04-26T20:55:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: osd op complaint time = 180 2024-04-26T20:55:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.935 INFO:teuthology.orchestra.run.smithi103.stdout:[mgr] 2024-04-26T20:55:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: debug ms = 1 2024-04-26T20:55:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: debug mgr = 20 2024-04-26T20:55:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: debug mon = 20 2024-04-26T20:55:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: debug auth = 20 2024-04-26T20:55:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: mon reweight min pgs per osd = 4 2024-04-26T20:55:47.935 INFO:teuthology.orchestra.run.smithi103.stdout: mon reweight min bytes per osd = 10 2024-04-26T20:55:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: mgr/telemetry/nag = false 2024-04-26T20:55:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: debug client = 20 2024-04-26T20:55:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.936 INFO:teuthology.orchestra.run.smithi103.stdout:[mon] 2024-04-26T20:55:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: debug ms = 1 2024-04-26T20:55:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: debug mon = 20 2024-04-26T20:55:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: debug paxos = 20 2024-04-26T20:55:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: debug auth = 20 2024-04-26T20:55:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: mon data avail warn = 5 2024-04-26T20:55:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: mon mgr mkfs grace = 240 2024-04-26T20:55:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: mon reweight min pgs per osd = 4 2024-04-26T20:55:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd reporter subtree level = osd 2024-04-26T20:55:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: mon osd prime pg temp = true 2024-04-26T20:55:47.936 INFO:teuthology.orchestra.run.smithi103.stdout: mon reweight min bytes per osd = 10 2024-04-26T20:55:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T20:55:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T20:55:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T20:55:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T20:55:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T20:55:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T20:55:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: # 1m isn't quite enough 2024-04-26T20:55:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: mon_down_mkfs_grace = 2m 2024-04-26T20:55:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: mon_warn_on_filestore_osds = false 2024-04-26T20:55:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: mon down mkfs grace = 300 2024-04-26T20:55:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: mon op complaint time = 120 2024-04-26T20:55:47.937 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T20:55:47.938 INFO:teuthology.orchestra.run.smithi103.stdout:[client] 2024-04-26T20:55:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: rgw cache enabled = true 2024-04-26T20:55:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: rgw enable ops log = true 2024-04-26T20:55:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: rgw enable usage log = true 2024-04-26T20:55:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T20:55:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T20:55:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: client mount timeout = 600 2024-04-26T20:55:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: debug client = 20 2024-04-26T20:55:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: debug ms = 1 2024-04-26T20:55:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: pid file = "" 2024-04-26T20:55:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: rados mon op timeout = 900 2024-04-26T20:55:47.938 INFO:teuthology.orchestra.run.smithi103.stdout: rados osd op timeout = 900 2024-04-26T20:55:47.938 INFO:teuthology.orchestra.run.smithi103.stdout:[mon.a] 2024-04-26T20:55:47.938 INFO:teuthology.orchestra.run.smithi103.stdout:[mon.b] 2024-04-26T20:55:47.939 INFO:teuthology.orchestra.run.smithi103.stdout:[mon.c] 2024-04-26T20:55:47.939 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.a] 2024-04-26T20:55:47.939 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.c] 2024-04-26T20:55:47.939 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.b] 2024-04-26T20:55:47.939 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.d] 2024-04-26T20:55:47.939 INFO:teuthology.orchestra.run.smithi103.stdout:[mds] 2024-04-26T20:55:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: debug mds = 20 2024-04-26T20:55:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: debug mds balancer = 20 2024-04-26T20:55:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: debug ms = 1 2024-04-26T20:55:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: mds debug frag = True 2024-04-26T20:55:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: mds debug scatterstat = True 2024-04-26T20:55:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: mds op complaint time = 180 2024-04-26T20:55:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: mds verify scatter = True 2024-04-26T20:55:47.939 INFO:teuthology.orchestra.run.smithi103.stdout: osd op complaint time = 180 2024-04-26T20:55:47.940 INFO:teuthology.orchestra.run.smithi103.stdout: rados mon op timeout = 900 2024-04-26T20:55:47.940 INFO:teuthology.orchestra.run.smithi103.stdout: rados osd op timeout = 900 2024-04-26T20:55:47.953 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-26T20:55:47.953 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-26T20:55:48.063 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-26T20:55:48.063 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:55:48.063 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-26T20:55:48.102 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:55:48.102 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-26T20:55:48.184 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T20:55:48.184 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:55:48.185 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T20:55:48.185 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T20:55:48.315 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:55:48.315 DEBUG:teuthology.orchestra.run.smithi028:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:55:48.396 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi103.front.sepia.ceph.com 2024-04-26T20:55:48.396 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:55:48.396 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T20:55:48.396 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T20:55:48.509 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:55:48.509 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:55:48.598 INFO:tasks.ceph:Setting up mon nodes... 2024-04-26T20:55:48.598 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-26T20:55:48.599 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T20:55:48.723 INFO:teuthology.orchestra.run.smithi028.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T20:55:48.729 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T20:55:48.861 INFO:teuthology.orchestra.run.smithi103.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T20:55:48.868 INFO:tasks.ceph:Setting up mds nodes... 2024-04-26T20:55:48.868 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T20:55:48.990 INFO:teuthology.orchestra.run.smithi028.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T20:55:48.996 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-26T20:55:49.062 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T20:55:49.178 INFO:teuthology.orchestra.run.smithi028.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T20:55:49.185 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-26T20:55:49.246 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T20:55:49.372 INFO:teuthology.orchestra.run.smithi103.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T20:55:49.378 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-26T20:55:49.451 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T20:55:49.583 INFO:teuthology.orchestra.run.smithi103.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T20:55:49.592 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-26T20:55:49.671 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-26T20:55:49.671 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T20:55:49.772 INFO:teuthology.orchestra.run.smithi028.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-26T20:55:49.805 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-26T20:55:49.893 INFO:teuthology.orchestra.run.smithi028.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-26T20:55:49.930 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-26T20:55:50.048 INFO:teuthology.orchestra.run.smithi103.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-26T20:55:50.088 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-26T20:55:50.193 INFO:teuthology.orchestra.run.smithi103.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-26T20:55:50.234 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-26T20:55:50.234 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi028.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi103.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T20:55:50.234 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-26T20:55:50.325 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T20:55:50.325 INFO:tasks.ceph:role: osd.0 2024-04-26T20:55:50.325 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T20:55:50.326 DEBUG:teuthology.orchestra.run.smithi028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T20:55:50.403 INFO:teuthology.orchestra.run.smithi028.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:55:50.404 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:55:50.404 INFO:teuthology.orchestra.run.smithi028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:55:50.404 INFO:teuthology.orchestra.run.smithi028.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:55:50.404 INFO:teuthology.orchestra.run.smithi028.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:55:50.404 INFO:teuthology.orchestra.run.smithi028.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:55:50.404 INFO:teuthology.orchestra.run.smithi028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:55:50.404 INFO:teuthology.orchestra.run.smithi028.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:55:50.404 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:55:50.404 INFO:teuthology.orchestra.run.smithi028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:55:50.439 INFO:teuthology.orchestra.run.smithi028.stdout:Discarding blocks...Done. 2024-04-26T20:55:50.444 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi028.front.sepia.ceph.com -o noatime 2024-04-26T20:55:50.445 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-26T20:55:50.554 DEBUG:teuthology.orchestra.run.smithi028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-26T20:55:50.632 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-26T20:55:50.715 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T20:55:50.715 INFO:tasks.ceph:role: osd.1 2024-04-26T20:55:50.715 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T20:55:50.715 DEBUG:teuthology.orchestra.run.smithi028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T20:55:50.791 INFO:teuthology.orchestra.run.smithi028.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:55:50.791 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:55:50.791 INFO:teuthology.orchestra.run.smithi028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:55:50.791 INFO:teuthology.orchestra.run.smithi028.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:55:50.791 INFO:teuthology.orchestra.run.smithi028.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:55:50.791 INFO:teuthology.orchestra.run.smithi028.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:55:50.791 INFO:teuthology.orchestra.run.smithi028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:55:50.791 INFO:teuthology.orchestra.run.smithi028.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:55:50.791 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:55:50.791 INFO:teuthology.orchestra.run.smithi028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:55:50.825 INFO:teuthology.orchestra.run.smithi028.stdout:Discarding blocks...Done. 2024-04-26T20:55:50.832 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi028.front.sepia.ceph.com -o noatime 2024-04-26T20:55:50.832 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-26T20:55:50.940 DEBUG:teuthology.orchestra.run.smithi028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-26T20:55:51.027 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-26T20:55:51.100 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T20:55:51.101 INFO:tasks.ceph:role: osd.2 2024-04-26T20:55:51.101 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T20:55:51.101 DEBUG:teuthology.orchestra.run.smithi028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T20:55:51.179 INFO:teuthology.orchestra.run.smithi028.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:55:51.179 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:55:51.179 INFO:teuthology.orchestra.run.smithi028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:55:51.180 INFO:teuthology.orchestra.run.smithi028.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:55:51.180 INFO:teuthology.orchestra.run.smithi028.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:55:51.180 INFO:teuthology.orchestra.run.smithi028.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:55:51.180 INFO:teuthology.orchestra.run.smithi028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:55:51.180 INFO:teuthology.orchestra.run.smithi028.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:55:51.180 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:55:51.180 INFO:teuthology.orchestra.run.smithi028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:55:51.214 INFO:teuthology.orchestra.run.smithi028.stdout:Discarding blocks...Done. 2024-04-26T20:55:51.219 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi028.front.sepia.ceph.com -o noatime 2024-04-26T20:55:51.220 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-26T20:55:51.315 DEBUG:teuthology.orchestra.run.smithi028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-26T20:55:51.386 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-26T20:55:51.449 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T20:55:51.449 INFO:tasks.ceph:role: osd.3 2024-04-26T20:55:51.449 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T20:55:51.450 DEBUG:teuthology.orchestra.run.smithi028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T20:55:51.523 INFO:teuthology.orchestra.run.smithi028.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:55:51.523 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:55:51.523 INFO:teuthology.orchestra.run.smithi028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:55:51.523 INFO:teuthology.orchestra.run.smithi028.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:55:51.523 INFO:teuthology.orchestra.run.smithi028.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:55:51.523 INFO:teuthology.orchestra.run.smithi028.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:55:51.524 INFO:teuthology.orchestra.run.smithi028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:55:51.524 INFO:teuthology.orchestra.run.smithi028.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:55:51.524 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:55:51.524 INFO:teuthology.orchestra.run.smithi028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:55:51.559 INFO:teuthology.orchestra.run.smithi028.stdout:Discarding blocks...Done. 2024-04-26T20:55:51.564 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi028.front.sepia.ceph.com -o noatime 2024-04-26T20:55:51.565 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-26T20:55:51.653 DEBUG:teuthology.orchestra.run.smithi028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-26T20:55:51.725 DEBUG:teuthology.orchestra.run.smithi028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:55:51.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:55:51.896+0000 7f9ab0231640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-26T20:55:51.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:55:51.896+0000 7f9ab0231640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-26T20:55:51.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:55:51.896+0000 7f9ab0231640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T20:55:51.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:55:51.896+0000 7f9ab0231640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T20:55:51.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:55:51.896+0000 7f9ab0231640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T20:55:51.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:55:51.901+0000 7f9ab0231640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-26T20:55:55.131 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T20:55:55.226 DEBUG:teuthology.orchestra.run.smithi028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:55:55.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:55:55.360+0000 7f5dc0a20640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-26T20:55:55.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:55:55.360+0000 7f5dc0a20640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-26T20:55:55.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:55:55.360+0000 7f5dc0a20640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T20:55:55.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:55:55.360+0000 7f5dc0a20640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T20:55:55.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:55:55.360+0000 7f5dc0a20640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T20:55:55.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:55:55.362+0000 7f5dc0a20640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-26T20:55:58.337 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T20:55:58.428 DEBUG:teuthology.orchestra.run.smithi028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:55:58.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:55:58.563+0000 7f2c1e4e1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-26T20:55:58.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:55:58.564+0000 7f2c1e4e1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-26T20:55:58.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:55:58.564+0000 7f2c1e4e1640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T20:55:58.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:55:58.564+0000 7f2c1e4e1640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T20:55:58.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:55:58.564+0000 7f2c1e4e1640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T20:55:58.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:55:58.566+0000 7f2c1e4e1640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-26T20:56:01.774 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T20:56:01.855 DEBUG:teuthology.orchestra.run.smithi028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:56:01.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:01.985+0000 7f01afe82640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-26T20:56:01.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:01.986+0000 7f01afe82640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-26T20:56:01.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:01.986+0000 7f01afe82640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T20:56:01.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:01.986+0000 7f01afe82640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T20:56:01.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:01.986+0000 7f01afe82640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T20:56:01.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:01.988+0000 7f01afe82640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-26T20:56:04.959 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T20:56:05.048 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-26T20:56:05.140 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T20:56:05.140 INFO:tasks.ceph:role: osd.4 2024-04-26T20:56:05.140 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi103.front.sepia.ceph.com 2024-04-26T20:56:05.140 DEBUG:teuthology.orchestra.run.smithi103:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T20:56:05.230 INFO:teuthology.orchestra.run.smithi103.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:56:05.230 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:56:05.230 INFO:teuthology.orchestra.run.smithi103.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:56:05.230 INFO:teuthology.orchestra.run.smithi103.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:56:05.230 INFO:teuthology.orchestra.run.smithi103.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:56:05.230 INFO:teuthology.orchestra.run.smithi103.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:56:05.231 INFO:teuthology.orchestra.run.smithi103.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:56:05.231 INFO:teuthology.orchestra.run.smithi103.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:56:05.231 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:56:05.231 INFO:teuthology.orchestra.run.smithi103.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:56:05.645 INFO:teuthology.orchestra.run.smithi103.stdout:Discarding blocks...Done. 2024-04-26T20:56:05.653 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi103.front.sepia.ceph.com -o noatime 2024-04-26T20:56:05.654 DEBUG:teuthology.orchestra.run.smithi103:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-26T20:56:05.796 DEBUG:teuthology.orchestra.run.smithi103:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-26T20:56:05.871 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-26T20:56:05.937 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T20:56:05.938 INFO:tasks.ceph:role: osd.5 2024-04-26T20:56:05.938 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi103.front.sepia.ceph.com 2024-04-26T20:56:05.938 DEBUG:teuthology.orchestra.run.smithi103:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T20:56:06.014 INFO:teuthology.orchestra.run.smithi103.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:56:06.014 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:56:06.014 INFO:teuthology.orchestra.run.smithi103.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:56:06.014 INFO:teuthology.orchestra.run.smithi103.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:56:06.015 INFO:teuthology.orchestra.run.smithi103.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:56:06.015 INFO:teuthology.orchestra.run.smithi103.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:56:06.015 INFO:teuthology.orchestra.run.smithi103.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:56:06.015 INFO:teuthology.orchestra.run.smithi103.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:56:06.015 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:56:06.015 INFO:teuthology.orchestra.run.smithi103.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:56:06.369 INFO:teuthology.orchestra.run.smithi103.stdout:Discarding blocks...Done. 2024-04-26T20:56:06.374 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi103.front.sepia.ceph.com -o noatime 2024-04-26T20:56:06.374 DEBUG:teuthology.orchestra.run.smithi103:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-26T20:56:06.468 DEBUG:teuthology.orchestra.run.smithi103:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-26T20:56:06.538 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-26T20:56:06.605 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T20:56:06.605 INFO:tasks.ceph:role: osd.6 2024-04-26T20:56:06.605 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi103.front.sepia.ceph.com 2024-04-26T20:56:06.605 DEBUG:teuthology.orchestra.run.smithi103:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T20:56:06.683 INFO:teuthology.orchestra.run.smithi103.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:56:06.683 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:56:06.683 INFO:teuthology.orchestra.run.smithi103.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:56:06.683 INFO:teuthology.orchestra.run.smithi103.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:56:06.683 INFO:teuthology.orchestra.run.smithi103.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:56:06.683 INFO:teuthology.orchestra.run.smithi103.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:56:06.683 INFO:teuthology.orchestra.run.smithi103.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:56:06.683 INFO:teuthology.orchestra.run.smithi103.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:56:06.683 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:56:06.684 INFO:teuthology.orchestra.run.smithi103.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:56:07.073 INFO:teuthology.orchestra.run.smithi103.stdout:Discarding blocks...Done. 2024-04-26T20:56:07.079 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi103.front.sepia.ceph.com -o noatime 2024-04-26T20:56:07.079 DEBUG:teuthology.orchestra.run.smithi103:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-26T20:56:07.190 DEBUG:teuthology.orchestra.run.smithi103:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-26T20:56:07.259 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-26T20:56:07.327 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T20:56:07.327 INFO:tasks.ceph:role: osd.7 2024-04-26T20:56:07.327 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi103.front.sepia.ceph.com 2024-04-26T20:56:07.327 DEBUG:teuthology.orchestra.run.smithi103:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T20:56:07.407 INFO:teuthology.orchestra.run.smithi103.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T20:56:07.407 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T20:56:07.407 INFO:teuthology.orchestra.run.smithi103.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T20:56:07.407 INFO:teuthology.orchestra.run.smithi103.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T20:56:07.407 INFO:teuthology.orchestra.run.smithi103.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T20:56:07.408 INFO:teuthology.orchestra.run.smithi103.stdout: = sunit=0 swidth=0 blks 2024-04-26T20:56:07.408 INFO:teuthology.orchestra.run.smithi103.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T20:56:07.408 INFO:teuthology.orchestra.run.smithi103.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T20:56:07.408 INFO:teuthology.orchestra.run.smithi103.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T20:56:07.408 INFO:teuthology.orchestra.run.smithi103.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T20:56:07.820 INFO:teuthology.orchestra.run.smithi103.stdout:Discarding blocks...Done. 2024-04-26T20:56:07.826 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi103.front.sepia.ceph.com -o noatime 2024-04-26T20:56:07.826 DEBUG:teuthology.orchestra.run.smithi103:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-26T20:56:07.965 DEBUG:teuthology.orchestra.run.smithi103:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-26T20:56:08.045 DEBUG:teuthology.orchestra.run.smithi103:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:56:08.173 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:08.172+0000 7fcbe4084640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-26T20:56:08.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:08.173+0000 7fcbe4084640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-26T20:56:08.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:08.173+0000 7fcbe4084640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T20:56:08.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:08.173+0000 7fcbe4084640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T20:56:08.174 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:08.173+0000 7fcbe4084640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T20:56:08.177 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:08.176+0000 7fcbe4084640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-26T20:56:11.408 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T20:56:11.494 DEBUG:teuthology.orchestra.run.smithi103:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:56:11.619 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:11.619+0000 7fb9fda18640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-26T20:56:11.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:11.620+0000 7fb9fda18640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-26T20:56:11.620 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:11.620+0000 7fb9fda18640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T20:56:11.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:11.620+0000 7fb9fda18640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T20:56:11.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:11.620+0000 7fb9fda18640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T20:56:11.622 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:11.622+0000 7fb9fda18640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-26T20:56:14.582 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T20:56:14.666 DEBUG:teuthology.orchestra.run.smithi103:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:56:14.796 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:14.795+0000 7feddc144640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-26T20:56:14.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:14.796+0000 7feddc144640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-26T20:56:14.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:14.796+0000 7feddc144640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T20:56:14.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:14.796+0000 7feddc144640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T20:56:14.797 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:14.796+0000 7feddc144640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T20:56:14.799 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:14.798+0000 7feddc144640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-26T20:56:15.237 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:56:15.245 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:56:17.780 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-26T20:56:17.865 DEBUG:teuthology.orchestra.run.smithi103:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:56:17.989 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:17.988+0000 7f4ef5c32640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-26T20:56:17.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:17.989+0000 7f4ef5c32640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-26T20:56:17.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:17.989+0000 7f4ef5c32640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T20:56:17.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:17.989+0000 7f4ef5c32640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T20:56:17.990 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:17.989+0000 7f4ef5c32640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T20:56:17.992 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:17.991+0000 7f4ef5c32640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-26T20:56:20.985 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-26T20:56:21.068 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-26T20:56:21.068 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:56:21.068 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-26T20:56:21.163 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:56:21.163 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-26T20:56:21.249 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:56:21.249 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-26T20:56:21.339 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:56:21.340 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-26T20:56:21.419 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:56:21.419 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-26T20:56:21.498 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:56:21.498 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-26T20:56:21.584 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:56:21.584 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-26T20:56:21.667 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:56:21.667 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-26T20:56:21.752 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:56:21.752 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-26T20:56:21.835 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:56:21.835 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-26T20:56:21.918 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:56:21.918 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-26T20:56:21.995 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:56:21.995 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-26T20:56:22.073 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:56:22.073 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-26T20:56:22.202 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:56:22.203 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-26T20:56:22.295 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:56:22.296 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-26T20:56:22.357 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:56:22.357 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-26T20:56:22.409 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:56:22.409 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-26T20:56:22.465 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:56:22.466 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-26T20:56:22.516 INFO:tasks.ceph:Adding keys to all mons... 2024-04-26T20:56:22.516 DEBUG:teuthology.orchestra.run.smithi028:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T20:56:22.523 DEBUG:teuthology.orchestra.run.smithi103:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T20:56:22.602 INFO:teuthology.orchestra.run.smithi103.stdout:[mgr.y] 2024-04-26T20:56:22.603 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQDUFCxmod8TKxAA5myHdFdSXxwOb3F4fMcyiA== 2024-04-26T20:56:22.603 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.a] 2024-04-26T20:56:22.603 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQDUFCxm570OOxAAwJmAfkKZtNjHl5XzgYUjSQ== 2024-04-26T20:56:22.603 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.c] 2024-04-26T20:56:22.603 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQDVFCxmpDSsChAAYAKo1Jj6ExMaDgsWl0NT3g== 2024-04-26T20:56:22.603 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.0] 2024-04-26T20:56:22.603 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQDXFCxmfzl7NRAAGayH8lsv4xFqpTC/xS/l3A== 2024-04-26T20:56:22.603 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.1] 2024-04-26T20:56:22.603 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQDbFCxmkZiIFRAAMBKG5zWUKbjum+GOHLr4Cg== 2024-04-26T20:56:22.603 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.2] 2024-04-26T20:56:22.603 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQDeFCxmKVSqIRAAawTDWXhcZjzrq2dswt+H2g== 2024-04-26T20:56:22.603 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.3] 2024-04-26T20:56:22.603 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQDhFCxmtKLGOhAAqvk9Jg+vsKnIgems1e8Ytg== 2024-04-26T20:56:22.604 INFO:teuthology.orchestra.run.smithi103.stdout:[mgr.x] 2024-04-26T20:56:22.604 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQDUFCxmh49VMxAAYQbMibMZBy6hsiO53AZYXw== 2024-04-26T20:56:22.604 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.b] 2024-04-26T20:56:22.604 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQDVFCxm5oQyFhAAhrxIFQkOub2iuwfXwrSi4g== 2024-04-26T20:56:22.604 INFO:teuthology.orchestra.run.smithi103.stdout:[mds.d] 2024-04-26T20:56:22.604 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQDVFCxm9YPLIhAAbgtxdShF5uQEQMCabsUY0A== 2024-04-26T20:56:22.604 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.4] 2024-04-26T20:56:22.604 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQDoFCxmwx9WChAA1zzH0x5gZXgcrVUQey4dwQ== 2024-04-26T20:56:22.604 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.5] 2024-04-26T20:56:22.604 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQDrFCxmsGDzJBAAnhFkDmONidO/0RgaVFbDGg== 2024-04-26T20:56:22.604 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.6] 2024-04-26T20:56:22.604 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQDuFCxmkj9/LxAA+8IvdhuCnsvEM56+y9stnQ== 2024-04-26T20:56:22.604 INFO:teuthology.orchestra.run.smithi103.stdout:[osd.7] 2024-04-26T20:56:22.604 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQDxFCxmvsb6OhAA50znn/4LbyZqxI1xgMJIEw== 2024-04-26T20:56:22.604 INFO:teuthology.orchestra.run.smithi103.stdout:[client.0] 2024-04-26T20:56:22.604 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQDVFCxm/rQRLhAAGsnaSU0YseRGCeDvxvvp/A== 2024-04-26T20:56:22.605 INFO:teuthology.orchestra.run.smithi103.stdout:[client.2] 2024-04-26T20:56:22.605 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQDVFCxm+ntKNRAABSG+8Hggkp8p4fQhi9UudA== 2024-04-26T20:56:22.605 INFO:teuthology.orchestra.run.smithi103.stdout:[client.1] 2024-04-26T20:56:22.605 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQDWFCxmFhfoAhAA3JGoo80M5CvuNjJyFg3akw== 2024-04-26T20:56:22.605 INFO:teuthology.orchestra.run.smithi103.stdout:[client.3] 2024-04-26T20:56:22.605 INFO:teuthology.orchestra.run.smithi103.stdout: key = AQDWFCxmur+LCxAAqUus6BuHeNXG2R5zM2j00A== 2024-04-26T20:56:22.605 INFO:teuthology.orchestra.run.smithi028.stdout:[mgr.y] 2024-04-26T20:56:22.605 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDUFCxmod8TKxAA5myHdFdSXxwOb3F4fMcyiA== 2024-04-26T20:56:22.605 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.a] 2024-04-26T20:56:22.605 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDUFCxm570OOxAAwJmAfkKZtNjHl5XzgYUjSQ== 2024-04-26T20:56:22.605 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.c] 2024-04-26T20:56:22.606 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDVFCxmpDSsChAAYAKo1Jj6ExMaDgsWl0NT3g== 2024-04-26T20:56:22.606 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.0] 2024-04-26T20:56:22.606 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDXFCxmfzl7NRAAGayH8lsv4xFqpTC/xS/l3A== 2024-04-26T20:56:22.606 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.1] 2024-04-26T20:56:22.606 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDbFCxmkZiIFRAAMBKG5zWUKbjum+GOHLr4Cg== 2024-04-26T20:56:22.606 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.2] 2024-04-26T20:56:22.606 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDeFCxmKVSqIRAAawTDWXhcZjzrq2dswt+H2g== 2024-04-26T20:56:22.606 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.3] 2024-04-26T20:56:22.606 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDhFCxmtKLGOhAAqvk9Jg+vsKnIgems1e8Ytg== 2024-04-26T20:56:22.606 INFO:teuthology.orchestra.run.smithi028.stdout:[mgr.x] 2024-04-26T20:56:22.606 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDUFCxmh49VMxAAYQbMibMZBy6hsiO53AZYXw== 2024-04-26T20:56:22.606 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.b] 2024-04-26T20:56:22.606 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDVFCxm5oQyFhAAhrxIFQkOub2iuwfXwrSi4g== 2024-04-26T20:56:22.607 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.d] 2024-04-26T20:56:22.607 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDVFCxm9YPLIhAAbgtxdShF5uQEQMCabsUY0A== 2024-04-26T20:56:22.607 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.4] 2024-04-26T20:56:22.607 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDoFCxmwx9WChAA1zzH0x5gZXgcrVUQey4dwQ== 2024-04-26T20:56:22.607 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.5] 2024-04-26T20:56:22.607 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDrFCxmsGDzJBAAnhFkDmONidO/0RgaVFbDGg== 2024-04-26T20:56:22.607 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.6] 2024-04-26T20:56:22.607 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDuFCxmkj9/LxAA+8IvdhuCnsvEM56+y9stnQ== 2024-04-26T20:56:22.607 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.7] 2024-04-26T20:56:22.607 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDxFCxmvsb6OhAA50znn/4LbyZqxI1xgMJIEw== 2024-04-26T20:56:22.607 INFO:teuthology.orchestra.run.smithi028.stdout:[client.0] 2024-04-26T20:56:22.607 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDVFCxm/rQRLhAAGsnaSU0YseRGCeDvxvvp/A== 2024-04-26T20:56:22.607 INFO:teuthology.orchestra.run.smithi028.stdout:[client.2] 2024-04-26T20:56:22.607 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDVFCxm+ntKNRAABSG+8Hggkp8p4fQhi9UudA== 2024-04-26T20:56:22.607 INFO:teuthology.orchestra.run.smithi028.stdout:[client.1] 2024-04-26T20:56:22.607 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDWFCxmFhfoAhAA3JGoo80M5CvuNjJyFg3akw== 2024-04-26T20:56:22.608 INFO:teuthology.orchestra.run.smithi028.stdout:[client.3] 2024-04-26T20:56:22.608 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDWFCxmur+LCxAAqUus6BuHeNXG2R5zM2j00A== 2024-04-26T20:56:22.609 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T20:56:22.656 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T20:56:22.799 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:56:22.804 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:56:22.939 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:56:22.944 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:56:23.080 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:56:23.086 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:56:23.219 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:56:23.225 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:56:23.360 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:56:23.368 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:56:23.503 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:56:23.509 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:56:23.648 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T20:56:23.654 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T20:56:23.788 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:56:23.792 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:56:23.932 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:56:23.937 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T20:56:24.073 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:56:24.079 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:56:24.217 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:56:24.222 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:56:24.351 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:56:24.357 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:56:24.487 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:56:24.493 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T20:56:24.629 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:56:24.635 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:56:24.770 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:56:24.776 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:56:24.912 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:56:24.917 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:56:25.053 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:56:25.057 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T20:56:25.198 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-26T20:56:25.198 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-26T20:56:25.288 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T20:56:25.443 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-26T20:56:25.529 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-26T20:56:25.618 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T20:56:25.766 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-26T20:56:25.841 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-26T20:56:25.926 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T20:56:26.072 DEBUG:teuthology.orchestra.run.smithi103:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-26T20:56:26.156 DEBUG:teuthology.orchestra.run.smithi028:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:56:26.164 DEBUG:teuthology.orchestra.run.smithi103:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T20:56:26.215 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-26T20:56:26.216 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-26T20:56:26.216 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-26T20:56:26.223 INFO:tasks.ceph.mon.a:Started 2024-04-26T20:56:26.223 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-26T20:56:26.223 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-26T20:56:26.230 INFO:tasks.ceph.mon.b:Started 2024-04-26T20:56:26.230 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-26T20:56:26.230 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-26T20:56:26.236 INFO:tasks.ceph.mon.c:Started 2024-04-26T20:56:26.236 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-26T20:56:26.237 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-26T20:56:26.237 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-26T20:56:26.243 INFO:tasks.ceph.mgr.y:Started 2024-04-26T20:56:26.244 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-26T20:56:26.244 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-26T20:56:26.248 INFO:tasks.ceph.mgr.x:Started 2024-04-26T20:56:26.248 INFO:tasks.ceph:Setting crush tunables to default 2024-04-26T20:56:26.248 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --cluster ceph osd crush tunables default 2024-04-26T20:56:26.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.421+0000 7f6339600640 1 Processor -- start 2024-04-26T20:56:26.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.421+0000 7f6339600640 1 -- start start 2024-04-26T20:56:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.422+0000 7f6339600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 0x7f63340d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.422+0000 7f6339600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 0x7f63340b8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.422+0000 7f6339600640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63340d4110 con 0x7f63340d1f90 2024-04-26T20:56:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.422+0000 7f6339600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f63340d23f0 con 0x7f63340d3af0 2024-04-26T20:56:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.422+0000 7f6339600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f63340d43f0 con 0x7f633415f680 2024-04-26T20:56:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.422+0000 7f6333400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 0x7f63340b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.422+0000 7f6333400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 0x7f63340b8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42474/0 (socket says 172.21.15.28:42474) 2024-04-26T20:56:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.422+0000 7f6333400640 1 -- 172.21.15.28:0/185184344 learned_addr learned my addr 172.21.15.28:0/185184344 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.422+0000 7f6332a00640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 0x7f63340d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.423+0000 7f6333400640 1 -- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 msgr2=0x7f63340b8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T20:56:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.423+0000 7f6333400640 1 -- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 msgr2=0x7f63340b8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:56:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.423+0000 7f6333400640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 0x7f63340b8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:56:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.423+0000 7f6333400640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 0x7f63340b8e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T20:56:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.423+0000 7f6332a00640 1 -- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 msgr2=0x7f63340d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T20:56:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.423+0000 7f6332a00640 1 -- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 msgr2=0x7f63340d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:56:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.423+0000 7f6332a00640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 0x7f63340d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:56:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.423+0000 7f6332a00640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 0x7f63340d1a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T20:56:26.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.624+0000 7f6333400640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 0x7f63340b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:26.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.624+0000 7f6332a00640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 0x7f63340d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:26.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.625+0000 7f6333400640 1 -- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 msgr2=0x7f63340b8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T20:56:26.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.625+0000 7f6333400640 1 -- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 msgr2=0x7f63340b8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:56:26.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.625+0000 7f6333400640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 0x7f63340b8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:56:26.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.625+0000 7f6332a00640 1 -- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 msgr2=0x7f63340d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T20:56:26.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.625+0000 7f6332a00640 1 -- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 msgr2=0x7f63340d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:56:26.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.625+0000 7f6332a00640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 0x7f63340d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:56:26.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.625+0000 7f6332a00640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 0x7f63340d1a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T20:56:26.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:26.625+0000 7f6333400640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 0x7f63340b8e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T20:56:27.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:27.026+0000 7f6333400640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 0x7f63340b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:27.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:27.026+0000 7f6332a00640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 0x7f63340d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:27.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:27.027+0000 7f6333400640 1 -- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 msgr2=0x7f63340b8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T20:56:27.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:27.027+0000 7f6333400640 1 -- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 msgr2=0x7f63340b8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:56:27.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:27.027+0000 7f6333400640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 0x7f63340b8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:56:27.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:27.027+0000 7f6332a00640 1 -- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 msgr2=0x7f63340d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T20:56:27.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:27.027+0000 7f6332a00640 1 -- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 msgr2=0x7f63340d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:56:27.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:27.027+0000 7f6332a00640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 0x7f63340d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:56:27.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:27.027+0000 7f6332a00640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 0x7f63340d1a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T20:56:27.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:27.027+0000 7f6333400640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 0x7f63340b8e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T20:56:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:27.829+0000 7f6332a00640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 0x7f63340d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:27.829+0000 7f6333400640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 0x7f63340b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:27.829+0000 7f6332a00640 1 -- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 msgr2=0x7f63340d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T20:56:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:27.829+0000 7f6332a00640 1 -- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 msgr2=0x7f63340d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:56:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:27.829+0000 7f6332a00640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 0x7f63340d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:56:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:27.829+0000 7f6333400640 1 -- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 msgr2=0x7f63340b8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-26T20:56:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:27.829+0000 7f6333400640 1 -- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 msgr2=0x7f63340b8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:56:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:27.829+0000 7f6333400640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 0x7f63340b8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:56:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:27.829+0000 7f6332a00640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 0x7f63340d1a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T20:56:27.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:27.829+0000 7f6333400640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 0x7f63340b8e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T20:56:29.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:29.423+0000 7f632be00640 1 -- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 msgr2=0x7f63340d1a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:56:29.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:29.423+0000 7f632be00640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f633415f680 0x7f63340d1a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:29.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:29.423+0000 7f632be00640 1 -- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 msgr2=0x7f63340b8e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:56:29.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:29.423+0000 7f632be00640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.28:3300/0 conn(0x7f63340d3af0 0x7f63340b8e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:29.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:29.423+0000 7f632be00640 1 -- 172.21.15.28:0/185184344 >> v1:172.21.15.103:6789/0 conn(0x7f63340d1f90 legacy=0x7f63340d33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:29.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:29.423+0000 7f632be00640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 0x7f6314009810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:29.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:29.423+0000 7f632be00640 1 -- 172.21.15.28:0/185184344 --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63340d46d0 con 0x7f63140036a0 2024-04-26T20:56:29.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:29.423+0000 7f632be00640 1 -- 172.21.15.28:0/185184344 --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f63340d43f0 con 0x7f6314007350 2024-04-26T20:56:29.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:29.423+0000 7f6332000640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 0x7f6314009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:29.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:29.424+0000 7f6332000640 1 -- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 msgr2=0x7f6314009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T20:56:29.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:29.424+0000 7f6332000640 1 -- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 msgr2=0x7f6314009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:56:29.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:29.424+0000 7f6332000640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 0x7f6314009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:56:29.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:29.424+0000 7f6332000640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 0x7f6314009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T20:56:29.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:29.625+0000 7f6332000640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 0x7f6314009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:29.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:29.626+0000 7f6332000640 1 -- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 msgr2=0x7f6314009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T20:56:29.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:29.626+0000 7f6332000640 1 -- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 msgr2=0x7f6314009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:56:29.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:29.626+0000 7f6332000640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 0x7f6314009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:56:29.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:29.626+0000 7f6332000640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 0x7f6314009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T20:56:30.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:30.027+0000 7f6332000640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 0x7f6314009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:30.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:30.027+0000 7f6332000640 1 -- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 msgr2=0x7f6314009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T20:56:30.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:30.027+0000 7f6332000640 1 -- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 msgr2=0x7f6314009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:56:30.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:30.027+0000 7f6332000640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 0x7f6314009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:56:30.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:30.028+0000 7f6332000640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 0x7f6314009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T20:56:30.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:30.829+0000 7f6332000640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 0x7f6314009810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:30.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:30.830+0000 7f6332000640 1 -- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 msgr2=0x7f6314009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T20:56:30.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:30.830+0000 7f6332000640 1 -- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 msgr2=0x7f6314009810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T20:56:30.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:30.830+0000 7f6332000640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 0x7f6314009810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T20:56:30.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:30.830+0000 7f6332000640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 0x7f6314009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T20:56:31.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.474+0000 7f6331600640 1 -- 172.21.15.28:0/185184344 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2814670972 0 0) 0x7f63340d46d0 con 0x7f63140036a0 2024-04-26T20:56:31.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.474+0000 7f6331600640 1 -- 172.21.15.28:0/185184344 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6318003520 con 0x7f63140036a0 2024-04-26T20:56:31.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.475+0000 7f6331600640 1 -- 172.21.15.28:0/185184344 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3591157545 0 0) 0x7f6318003520 con 0x7f63140036a0 2024-04-26T20:56:31.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.475+0000 7f6331600640 1 -- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 msgr2=0x7f6314009810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:56:31.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.475+0000 7f6331600640 1 --2- 172.21.15.28:0/185184344 >> v2:172.21.15.103:3300/0 conn(0x7f6314007350 0x7f6314009810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:31.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.475+0000 7f6331600640 1 -- 172.21.15.28:0/185184344 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631400af10 con 0x7f63140036a0 2024-04-26T20:56:31.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.476+0000 7f6331600640 1 -- 172.21.15.28:0/185184344 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f632400d580 con 0x7f63140036a0 2024-04-26T20:56:31.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.476+0000 7f6331600640 1 -- 172.21.15.28:0/185184344 >> v1:172.21.15.28:6789/0 conn(0x7f63140036a0 legacy=0x7f6314005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:31.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.477+0000 7f6331600640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6318003a00 0x7f6318003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:31.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.477+0000 7f6331600640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6318004800 0x7f6318004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:31.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.477+0000 7f6333400640 1 -- 172.21.15.28:0/185184344 reap_dead start 2024-04-26T20:56:31.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.477+0000 7f6331600640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f63180053c0 0x7f631800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:31.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.477+0000 7f6331600640 1 -- 172.21.15.28:0/185184344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f63340d43f0 con 0x7f6318003a00 2024-04-26T20:56:31.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.477+0000 7f6331600640 1 -- 172.21.15.28:0/185184344 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f631800c280 con 0x7f6318004800 2024-04-26T20:56:31.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.477+0000 7f6331600640 1 -- 172.21.15.28:0/185184344 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f631800c480 con 0x7f63180053c0 2024-04-26T20:56:31.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.477+0000 7f6332000640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6318004800 0x7f6318004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:31.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.478+0000 7f6332a00640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6318003a00 0x7f6318003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:31.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.478+0000 7f6333400640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f63180053c0 0x7f631800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:31.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.479+0000 7f6332a00640 1 -- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f63180053c0 msgr2=0x7f631800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:31.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.479+0000 7f6332a00640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f63180053c0 0x7f631800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:31.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.479+0000 7f6332a00640 1 -- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6318004800 msgr2=0x7f6318004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:31.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.479+0000 7f6332a00640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6318004800 0x7f6318004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:31.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.479+0000 7f6332a00640 1 -- 172.21.15.28:0/185184344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6318003520 con 0x7f6318003a00 2024-04-26T20:56:31.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.479+0000 7f6332000640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6318004800 0x7f6318004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:56:31.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.481+0000 7f6332a00640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6318003a00 0x7f6318003e00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6314007260 tx=0x7f6318005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca45f86e46924d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:31.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.482+0000 7f6331600640 1 -- 172.21.15.28:0/185184344 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324031070 con 0x7f6318003a00 2024-04-26T20:56:31.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.482+0000 7f6331600640 1 -- 172.21.15.28:0/185184344 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f632402bd00 con 0x7f6318003a00 2024-04-26T20:56:31.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.482+0000 7f6331600640 1 -- 172.21.15.28:0/185184344 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632400ddb0 con 0x7f6318003a00 2024-04-26T20:56:31.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.483+0000 7f6339600640 1 -- 172.21.15.28:0/185184344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6318003a00 msgr2=0x7f6318003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:31.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.483+0000 7f6339600640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6318003a00 0x7f6318003e00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6314007260 tx=0x7f6318005830 comp rx=0 tx=0).stop 2024-04-26T20:56:31.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.484+0000 7f6339600640 1 -- 172.21.15.28:0/185184344 shutdown_connections 2024-04-26T20:56:31.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.484+0000 7f6339600640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f63180053c0 0x7f631800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:31.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.484+0000 7f6339600640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6318004800 0x7f6318004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:31.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.484+0000 7f6339600640 1 --2- 172.21.15.28:0/185184344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6318003a00 0x7f6318003e00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:31.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.484+0000 7f6339600640 1 -- 172.21.15.28:0/185184344 >> 172.21.15.28:0/185184344 conn(0x7f633406ca10 msgr2=0x7f6334072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:31.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.484+0000 7f6339600640 1 -- 172.21.15.28:0/185184344 shutdown_connections 2024-04-26T20:56:31.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.484+0000 7f6339600640 1 -- 172.21.15.28:0/185184344 wait complete. 2024-04-26T20:56:31.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.487+0000 7f6339600640 1 Processor -- start 2024-04-26T20:56:31.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.488+0000 7f6339600640 1 -- start start 2024-04-26T20:56:31.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.489+0000 7f6339600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6318003a00 0x7f6334150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:31.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.489+0000 7f6339600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6318004800 0x7f63340b9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:31.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.489+0000 7f6339600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63180053c0 0x7f63340b9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:31.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.489+0000 7f6339600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f63340d1a50 con 0x7f63180053c0 2024-04-26T20:56:31.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.489+0000 7f6339600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f63340b8940 con 0x7f6318003a00 2024-04-26T20:56:31.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.489+0000 7f6339600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f63340d1bd0 con 0x7f6318004800 2024-04-26T20:56:31.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.489+0000 7f6333400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63180053c0 0x7f63340b9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:31.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.489+0000 7f6333400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63180053c0 0x7f63340b9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52496/0 (socket says 172.21.15.28:52496) 2024-04-26T20:56:31.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.489+0000 7f6333400640 1 -- 172.21.15.28:0/815966346 learned_addr learned my addr 172.21.15.28:0/815966346 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:31.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.489+0000 7f6332a00640 1 --2- 172.21.15.28:0/815966346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6318003a00 0x7f6334150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:31.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.489+0000 7f6332000640 1 --2- 172.21.15.28:0/815966346 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6318004800 0x7f63340b9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:31.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.490+0000 7f6333400640 1 -- 172.21.15.28:0/815966346 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6318004800 msgr2=0x7f63340b9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:31.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.490+0000 7f6333400640 1 --2- 172.21.15.28:0/815966346 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6318004800 0x7f63340b9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:31.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.490+0000 7f6333400640 1 -- 172.21.15.28:0/815966346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6318003a00 msgr2=0x7f6334150b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:31.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.490+0000 7f6333400640 1 --2- 172.21.15.28:0/815966346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6318003a00 0x7f6334150b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:31.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.490+0000 7f6333400640 1 -- 172.21.15.28:0/815966346 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6334151a80 con 0x7f63180053c0 2024-04-26T20:56:31.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.490+0000 7f6333400640 1 --2- 172.21.15.28:0/815966346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63180053c0 0x7f63340b9cb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f632c0096d0 tx=0x7f632c014e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:31.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.491+0000 7f632aa00640 1 -- 172.21.15.28:0/815966346 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c01b030 con 0x7f63180053c0 2024-04-26T20:56:31.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.491+0000 7f632aa00640 1 -- 172.21.15.28:0/815966346 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f632c012a70 con 0x7f63180053c0 2024-04-26T20:56:31.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.491+0000 7f632aa00640 1 -- 172.21.15.28:0/815966346 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c012c60 con 0x7f63180053c0 2024-04-26T20:56:31.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.491+0000 7f6339600640 1 -- 172.21.15.28:0/815966346 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63340cdd90 con 0x7f63180053c0 2024-04-26T20:56:31.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.491+0000 7f6339600640 1 -- 172.21.15.28:0/815966346 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63340ce350 con 0x7f63180053c0 2024-04-26T20:56:31.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.491+0000 7f632aa00640 1 -- 172.21.15.28:0/815966346 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f632c015040 con 0x7f63180053c0 2024-04-26T20:56:31.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.491+0000 7f632aa00640 1 -- 172.21.15.28:0/815966346 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f632c0207a0 con 0x7f63180053c0 2024-04-26T20:56:31.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.492+0000 7f6339600640 1 -- 172.21.15.28:0/815966346 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f633406a360 con 0x7f63180053c0 2024-04-26T20:56:31.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.496+0000 7f632aa00640 1 -- 172.21.15.28:0/815966346 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f632c025030 con 0x7f63180053c0 2024-04-26T20:56:31.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.572+0000 7f6339600640 1 -- 172.21.15.28:0/815966346 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f63341e9d20 con 0x7f63180053c0 2024-04-26T20:56:31.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.576+0000 7f632aa00640 1 -- 172.21.15.28:0/815966346 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f6334151a80 con 0x7f63180053c0 2024-04-26T20:56:31.577 INFO:teuthology.orchestra.run.smithi028.stderr:adjusted tunables profile to default 2024-04-26T20:56:31.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.577+0000 7f6339600640 1 -- 172.21.15.28:0/815966346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63180053c0 msgr2=0x7f63340b9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:31.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.577+0000 7f6339600640 1 --2- 172.21.15.28:0/815966346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63180053c0 0x7f63340b9cb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f632c0096d0 tx=0x7f632c014e30 comp rx=0 tx=0).stop 2024-04-26T20:56:31.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.577+0000 7f6339600640 1 -- 172.21.15.28:0/815966346 shutdown_connections 2024-04-26T20:56:31.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.577+0000 7f6339600640 1 --2- 172.21.15.28:0/815966346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f63180053c0 0x7f63340b9cb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:31.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.577+0000 7f6339600640 1 --2- 172.21.15.28:0/815966346 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6318004800 0x7f63340b9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:31.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.577+0000 7f6339600640 1 --2- 172.21.15.28:0/815966346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6318003a00 0x7f6334150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:31.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.577+0000 7f6339600640 1 -- 172.21.15.28:0/815966346 >> 172.21.15.28:0/815966346 conn(0x7f633406ca10 msgr2=0x7f63340c8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:31.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.577+0000 7f6339600640 1 -- 172.21.15.28:0/815966346 shutdown_connections 2024-04-26T20:56:31.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:31.577+0000 7f6339600640 1 -- 172.21.15.28:0/815966346 wait complete. 2024-04-26T20:56:31.597 INFO:tasks.ceph:check_enable_crimson: False 2024-04-26T20:56:31.598 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-26T20:56:31.598 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:56:31.598 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-26T20:56:31.637 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:31.636+0000 7f1338f21040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T20:56:31.665 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:56:31.665 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-26T20:56:31.792 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:56:31.793 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-26T20:56:31.831 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:31.830+0000 7f1338f21040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:56:31.876 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:56:31.876 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-26T20:56:31.901 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:31.900+0000 7f1338f21040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T20:56:31.992 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:56:31.992 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-26T20:56:32.080 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:56:32.081 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-26T20:56:32.164 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:56:32.164 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-26T20:56:32.219 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:32.218+0000 7f1338f21040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T20:56:32.248 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:56:32.248 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-26T20:56:32.275 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:32.274+0000 7f1338f21040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T20:56:32.331 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd new ae2c534d-cc92-4d2a-8e08-d341decd44ea 0 2024-04-26T20:56:32.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.520+0000 7f9733400640 1 Processor -- start 2024-04-26T20:56:32.521 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.520+0000 7f9733400640 1 -- start start 2024-04-26T20:56:32.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.522+0000 7f9733400640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f972c15f680 0x7f972c0d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:32.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.522+0000 7f9733400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f972c0d1f90 0x7f972c0d33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:32.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.522+0000 7f9733400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f972c0d4110 con 0x7f972c0d3920 2024-04-26T20:56:32.523 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.522+0000 7f9733400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f972c0d23f0 con 0x7f972c0d1f90 2024-04-26T20:56:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.522+0000 7f9733400640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f972c0d43f0 con 0x7f972c15f680 2024-04-26T20:56:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.522+0000 7f9730a00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f972c15f680 0x7f972c0d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.522+0000 7f9730a00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f972c15f680 0x7f972c0d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46160/0 (socket says 172.21.15.103:46160) 2024-04-26T20:56:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.522+0000 7f9730a00640 1 -- 172.21.15.103:0/844266322 learned_addr learned my addr 172.21.15.103:0/844266322 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-26T20:56:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.522+0000 7f9723e00640 1 --2- 172.21.15.103:0/844266322 >> v2:172.21.15.28:3300/0 conn(0x7f972c0d1f90 0x7f972c0d33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.523+0000 7f9730a00640 1 -- 172.21.15.103:0/844266322 >> v2:172.21.15.28:3300/0 conn(0x7f972c0d1f90 msgr2=0x7f972c0d33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.523+0000 7f9730a00640 1 --2- 172.21.15.103:0/844266322 >> v2:172.21.15.28:3300/0 conn(0x7f972c0d1f90 0x7f972c0d33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.523+0000 7f9730a00640 1 -- 172.21.15.103:0/844266322 >> v1:172.21.15.28:6789/0 conn(0x7f972c0d3920 legacy=0x7f972c0b6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.523+0000 7f9730a00640 1 -- 172.21.15.103:0/844266322 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972c0d46d0 con 0x7f972c15f680 2024-04-26T20:56:32.524 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.523+0000 7f9723e00640 1 --2- 172.21.15.103:0/844266322 >> v2:172.21.15.28:3300/0 conn(0x7f972c0d1f90 0x7f972c0d33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:56:32.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.524+0000 7f9730a00640 1 --2- 172.21.15.103:0/844266322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f972c15f680 0x7f972c0d1a50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f972c0d26e0 tx=0x7f971c02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=635e6c4c8ab821c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:32.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.524+0000 7f972be00640 1 -- 172.21.15.103:0/844266322 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c037070 con 0x7f972c15f680 2024-04-26T20:56:32.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.524+0000 7f972be00640 1 -- 172.21.15.103:0/844266322 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f971c032440 con 0x7f972c15f680 2024-04-26T20:56:32.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.524+0000 7f972be00640 1 -- 172.21.15.103:0/844266322 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971c032810 con 0x7f972c15f680 2024-04-26T20:56:32.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.525+0000 7f9733400640 1 -- 172.21.15.103:0/844266322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f972c15f680 msgr2=0x7f972c0d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:32.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.525+0000 7f9733400640 1 --2- 172.21.15.103:0/844266322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f972c15f680 0x7f972c0d1a50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f972c0d26e0 tx=0x7f971c02a230 comp rx=0 tx=0).stop 2024-04-26T20:56:32.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.525+0000 7f9733400640 1 -- 172.21.15.103:0/844266322 shutdown_connections 2024-04-26T20:56:32.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.525+0000 7f9733400640 1 --2- 172.21.15.103:0/844266322 >> v2:172.21.15.28:3300/0 conn(0x7f972c0d1f90 0x7f972c0d33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:32.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.525+0000 7f9733400640 1 --2- 172.21.15.103:0/844266322 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f972c15f680 0x7f972c0d1a50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:32.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.525+0000 7f9733400640 1 -- 172.21.15.103:0/844266322 >> 172.21.15.103:0/844266322 conn(0x7f972c06ca10 msgr2=0x7f972c072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:32.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.525+0000 7f9733400640 1 -- 172.21.15.103:0/844266322 shutdown_connections 2024-04-26T20:56:32.526 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.525+0000 7f9733400640 1 -- 172.21.15.103:0/844266322 wait complete. 2024-04-26T20:56:32.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.527+0000 7f9733400640 1 Processor -- start 2024-04-26T20:56:32.527 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.527+0000 7f9733400640 1 -- start start 2024-04-26T20:56:32.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.528+0000 7f9733400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f972c0d1f90 0x7f972c1f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:32.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.528+0000 7f9733400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f972c0d3920 0x7f972c154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:32.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.528+0000 7f9733400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f972c15f680 0x7f972c1f84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:32.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.528+0000 7f9733400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f972c0d3d90 con 0x7f972c0d1f90 2024-04-26T20:56:32.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.528+0000 7f9733400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f972c0b8940 con 0x7f972c15f680 2024-04-26T20:56:32.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.528+0000 7f9733400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f972c0b9e70 con 0x7f972c0d3920 2024-04-26T20:56:32.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.528+0000 7f9723e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f972c0d3920 0x7f972c154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:32.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.528+0000 7f9731400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f972c15f680 0x7f972c1f84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:32.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.528+0000 7f9723e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f972c0d3920 0x7f972c154500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:58582/0 (socket says 172.21.15.103:58582) 2024-04-26T20:56:32.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.528+0000 7f9723e00640 1 -- 172.21.15.103:0/972332861 learned_addr learned my addr 172.21.15.103:0/972332861 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-26T20:56:32.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.528+0000 7f9730a00640 1 --2- 172.21.15.103:0/972332861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f972c0d1f90 0x7f972c1f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:32.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.528+0000 7f9731400640 1 -- 172.21.15.103:0/972332861 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f972c0d3920 msgr2=0x7f972c154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:32.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.528+0000 7f9731400640 1 --2- 172.21.15.103:0/972332861 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f972c0d3920 0x7f972c154500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:32.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.528+0000 7f9731400640 1 -- 172.21.15.103:0/972332861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f972c0d1f90 msgr2=0x7f972c1f23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:32.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.528+0000 7f9731400640 1 --2- 172.21.15.103:0/972332861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f972c0d1f90 0x7f972c1f23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:32.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.528+0000 7f9731400640 1 -- 172.21.15.103:0/972332861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f972c156030 con 0x7f972c15f680 2024-04-26T20:56:32.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.529+0000 7f9731400640 1 --2- 172.21.15.103:0/972332861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f972c15f680 0x7f972c1f84a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f972400c780 tx=0x7f972400cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:32.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.529+0000 7f9729600640 1 -- 172.21.15.103:0/972332861 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724016020 con 0x7f972c15f680 2024-04-26T20:56:32.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.529+0000 7f9729600640 1 -- 172.21.15.103:0/972332861 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f972400aa50 con 0x7f972c15f680 2024-04-26T20:56:32.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.529+0000 7f9729600640 1 -- 172.21.15.103:0/972332861 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972400ce30 con 0x7f972c15f680 2024-04-26T20:56:32.531 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.529+0000 7f9733400640 1 -- 172.21.15.103:0/972332861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f972c1562b0 con 0x7f972c15f680 2024-04-26T20:56:32.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.529+0000 7f9733400640 1 -- 172.21.15.103:0/972332861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f972c156870 con 0x7f972c15f680 2024-04-26T20:56:32.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.530+0000 7f9729600640 1 -- 172.21.15.103:0/972332861 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9724019020 con 0x7f972c15f680 2024-04-26T20:56:32.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.530+0000 7f9729600640 1 -- 172.21.15.103:0/972332861 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f9724012420 con 0x7f972c15f680 2024-04-26T20:56:32.532 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.530+0000 7f9733400640 1 -- 172.21.15.103:0/972332861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f972c06a360 con 0x7f972c15f680 2024-04-26T20:56:32.534 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.533+0000 7f9729600640 1 -- 172.21.15.103:0/972332861 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f9724020030 con 0x7f972c15f680 2024-04-26T20:56:32.540 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:32.539+0000 7f68ac612040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T20:56:32.613 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.613+0000 7f9733400640 1 -- 172.21.15.103:0/972332861 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ae2c534d-cc92-4d2a-8e08-d341decd44ea", "id": 0} v 0) -- 0x7f972c068d20 con 0x7f972c15f680 2024-04-26T20:56:32.621 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.621+0000 7f9729600640 1 -- 172.21.15.103:0/972332861 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ae2c534d-cc92-4d2a-8e08-d341decd44ea", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f972c156030 con 0x7f972c15f680 2024-04-26T20:56:32.621 INFO:teuthology.orchestra.run.smithi103.stdout:0 2024-04-26T20:56:32.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.622+0000 7f9733400640 1 -- 172.21.15.103:0/972332861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f972c15f680 msgr2=0x7f972c1f84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:32.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.622+0000 7f9733400640 1 --2- 172.21.15.103:0/972332861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f972c15f680 0x7f972c1f84a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f972400c780 tx=0x7f972400cc40 comp rx=0 tx=0).stop 2024-04-26T20:56:32.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.622+0000 7f9733400640 1 -- 172.21.15.103:0/972332861 shutdown_connections 2024-04-26T20:56:32.623 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.622+0000 7f9733400640 1 --2- 172.21.15.103:0/972332861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f972c15f680 0x7f972c1f84a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:32.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.622+0000 7f9733400640 1 --2- 172.21.15.103:0/972332861 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f972c0d3920 0x7f972c154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:32.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.622+0000 7f9733400640 1 --2- 172.21.15.103:0/972332861 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f972c0d1f90 0x7f972c1f23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:32.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.622+0000 7f9733400640 1 -- 172.21.15.103:0/972332861 >> 172.21.15.103:0/972332861 conn(0x7f972c06ca10 msgr2=0x7f972c0c8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:32.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.622+0000 7f9733400640 1 -- 172.21.15.103:0/972332861 shutdown_connections 2024-04-26T20:56:32.624 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.622+0000 7f9733400640 1 -- 172.21.15.103:0/972332861 wait complete. 2024-04-26T20:56:32.641 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd new db8ba836-8a8b-49d2-9e2f-30a0b1e537a2 1 2024-04-26T20:56:32.735 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:32.734+0000 7f68ac612040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:56:32.802 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:32.801+0000 7f68ac612040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T20:56:32.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.824+0000 7f6ec7e00640 1 Processor -- start 2024-04-26T20:56:32.824 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.824+0000 7f6ec7e00640 1 -- start start 2024-04-26T20:56:32.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.825+0000 7f6ec7e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f6ec815f1d0 0x7f6ec80d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:32.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.825+0000 7f6ec7e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ec80d34b0 0x7f6ec80cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:32.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.825+0000 7f6ec7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6ec80d0140 0x7f6ec806c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:32.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.825+0000 7f6ec7e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f6ec815f5b0 con 0x7f6ec80d0140 2024-04-26T20:56:32.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.825+0000 7f6ec7e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f6ec80d3a30 con 0x7f6ec815f1d0 2024-04-26T20:56:32.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.825+0000 7f6ec7e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f6ec80d3d10 con 0x7f6ec80d34b0 2024-04-26T20:56:32.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.825+0000 7f6ec6000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ec80d34b0 0x7f6ec80cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:32.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.825+0000 7f6ec6a00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f6ec815f1d0 0x7f6ec80d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:32.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.825+0000 7f6ec6000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ec80d34b0 0x7f6ec80cfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:58588/0 (socket says 172.21.15.103:58588) 2024-04-26T20:56:32.827 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.825+0000 7f6ec6a00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f6ec815f1d0 0x7f6ec80d2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46174/0 (socket says 172.21.15.103:46174) 2024-04-26T20:56:32.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.825+0000 7f6ec6000640 1 -- 172.21.15.103:0/2691476001 learned_addr learned my addr 172.21.15.103:0/2691476001 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-26T20:56:32.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.826+0000 7f6ec7400640 1 --2- 172.21.15.103:0/2691476001 >> v2:172.21.15.28:3300/0 conn(0x7f6ec80d0140 0x7f6ec806c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:32.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.826+0000 7f6ec6a00640 1 -- 172.21.15.103:0/2691476001 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ec80d34b0 msgr2=0x7f6ec80cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:32.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.826+0000 7f6ec6a00640 1 --2- 172.21.15.103:0/2691476001 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ec80d34b0 0x7f6ec80cfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:32.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.826+0000 7f6ec6a00640 1 -- 172.21.15.103:0/2691476001 >> v2:172.21.15.28:3300/0 conn(0x7f6ec80d0140 msgr2=0x7f6ec806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:32.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.826+0000 7f6ec6a00640 1 --2- 172.21.15.103:0/2691476001 >> v2:172.21.15.28:3300/0 conn(0x7f6ec80d0140 0x7f6ec806c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:32.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.826+0000 7f6ec6a00640 1 -- 172.21.15.103:0/2691476001 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec80d3ff0 con 0x7f6ec815f1d0 2024-04-26T20:56:32.828 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.826+0000 7f6ec6000640 1 --2- 172.21.15.103:0/2691476001 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ec80d34b0 0x7f6ec80cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:56:32.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.827+0000 7f6ec6a00640 1 --2- 172.21.15.103:0/2691476001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ec815f1d0 0x7f6ec80d2f70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6ec808d890 tx=0x7f6eb802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7071c2f937a9828f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:32.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.827+0000 7f6ec5600640 1 -- 172.21.15.103:0/2691476001 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8037070 con 0x7f6ec815f1d0 2024-04-26T20:56:32.829 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.827+0000 7f6ec5600640 1 -- 172.21.15.103:0/2691476001 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6eb8032440 con 0x7f6ec815f1d0 2024-04-26T20:56:32.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.827+0000 7f6ec5600640 1 -- 172.21.15.103:0/2691476001 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb802ae60 con 0x7f6ec815f1d0 2024-04-26T20:56:32.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.828+0000 7f6ec7e00640 1 -- 172.21.15.103:0/2691476001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ec815f1d0 msgr2=0x7f6ec80d2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:32.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.828+0000 7f6ec7e00640 1 --2- 172.21.15.103:0/2691476001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ec815f1d0 0x7f6ec80d2f70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6ec808d890 tx=0x7f6eb802a110 comp rx=0 tx=0).stop 2024-04-26T20:56:32.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.828+0000 7f6ec7e00640 1 -- 172.21.15.103:0/2691476001 shutdown_connections 2024-04-26T20:56:32.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.828+0000 7f6ec7e00640 1 --2- 172.21.15.103:0/2691476001 >> v2:172.21.15.28:3300/0 conn(0x7f6ec80d0140 0x7f6ec806c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:32.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.828+0000 7f6ec7e00640 1 --2- 172.21.15.103:0/2691476001 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ec80d34b0 0x7f6ec80cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:32.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.828+0000 7f6ec7e00640 1 --2- 172.21.15.103:0/2691476001 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ec815f1d0 0x7f6ec80d2f70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:32.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.828+0000 7f6ec7e00640 1 -- 172.21.15.103:0/2691476001 >> 172.21.15.103:0/2691476001 conn(0x7f6ec80d5030 msgr2=0x7f6ec80d5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:32.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.828+0000 7f6ec7e00640 1 -- 172.21.15.103:0/2691476001 shutdown_connections 2024-04-26T20:56:32.830 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.828+0000 7f6ec7e00640 1 -- 172.21.15.103:0/2691476001 wait complete. 2024-04-26T20:56:32.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.830+0000 7f6ec7e00640 1 Processor -- start 2024-04-26T20:56:32.831 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.830+0000 7f6ec7e00640 1 -- start start 2024-04-26T20:56:32.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.831+0000 7f6ec7e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ec80d0140 0x7f6ec81f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:32.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.831+0000 7f6ec7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ec80d34b0 0x7f6ec81579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:32.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.831+0000 7f6ec7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ec815f1d0 0x7f6ec80c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:32.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.831+0000 7f6ec7e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec80bacf0 con 0x7f6ec815f1d0 2024-04-26T20:56:32.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.831+0000 7f6ec7e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec80d45b0 con 0x7f6ec80d34b0 2024-04-26T20:56:32.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.831+0000 7f6ec7e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f6ec80bae70 con 0x7f6ec80d0140 2024-04-26T20:56:32.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.831+0000 7f6ec6a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ec80d0140 0x7f6ec81f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:32.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.831+0000 7f6ec6000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ec80d34b0 0x7f6ec81579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:32.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.831+0000 7f6ec6a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ec80d0140 0x7f6ec81f4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:58594/0 (socket says 172.21.15.103:58594) 2024-04-26T20:56:32.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.831+0000 7f6ec6a00640 1 -- 172.21.15.103:0/2871894448 learned_addr learned my addr 172.21.15.103:0/2871894448 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-26T20:56:32.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.831+0000 7f6ec7400640 1 --2- 172.21.15.103:0/2871894448 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ec815f1d0 0x7f6ec80c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:32.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.832+0000 7f6ec6a00640 1 -- 172.21.15.103:0/2871894448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ec80d34b0 msgr2=0x7f6ec81579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:32.833 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.832+0000 7f6ec6a00640 1 --2- 172.21.15.103:0/2871894448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ec80d34b0 0x7f6ec81579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:32.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.832+0000 7f6ec6a00640 1 -- 172.21.15.103:0/2871894448 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ec815f1d0 msgr2=0x7f6ec80c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:32.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.832+0000 7f6ec6a00640 1 --2- 172.21.15.103:0/2871894448 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ec815f1d0 0x7f6ec80c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:32.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.832+0000 7f6ec6a00640 1 -- 172.21.15.103:0/2871894448 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec80cb2a0 con 0x7f6ec80d0140 2024-04-26T20:56:32.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.832+0000 7f6ec6000640 1 --2- 172.21.15.103:0/2871894448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ec80d34b0 0x7f6ec81579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:56:32.834 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.832+0000 7f6ec6a00640 1 --2- 172.21.15.103:0/2871894448 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ec80d0140 0x7f6ec81f4d40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8000c00 tx=0x7f6eb80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:32.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.832+0000 7f6ebf400640 1 -- 172.21.15.103:0/2871894448 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8037040 con 0x7f6ec80d0140 2024-04-26T20:56:32.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.833+0000 7f6ebf400640 1 -- 172.21.15.103:0/2871894448 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6eb8042070 con 0x7f6ec80d0140 2024-04-26T20:56:32.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.833+0000 7f6ebf400640 1 -- 172.21.15.103:0/2871894448 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8030960 con 0x7f6ec80d0140 2024-04-26T20:56:32.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.833+0000 7f6ec7e00640 1 -- 172.21.15.103:0/2871894448 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec80cb580 con 0x7f6ec80d0140 2024-04-26T20:56:32.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.833+0000 7f6ec7e00640 1 -- 172.21.15.103:0/2871894448 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec80c9800 con 0x7f6ec80d0140 2024-04-26T20:56:32.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.833+0000 7f6ebf400640 1 -- 172.21.15.103:0/2871894448 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6eb8030b00 con 0x7f6ec80d0140 2024-04-26T20:56:32.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.833+0000 7f6ebf400640 1 -- 172.21.15.103:0/2871894448 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f6eb8047b80 con 0x7f6ec80d0140 2024-04-26T20:56:32.835 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.834+0000 7f6ec7e00640 1 -- 172.21.15.103:0/2871894448 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ec806a250 con 0x7f6ec80d0140 2024-04-26T20:56:32.837 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.836+0000 7f6ebf400640 1 -- 172.21.15.103:0/2871894448 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f6eb8047460 con 0x7f6ec80d0140 2024-04-26T20:56:32.917 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.916+0000 7f6ec7e00640 1 -- 172.21.15.103:0/2871894448 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd new", "uuid": "db8ba836-8a8b-49d2-9e2f-30a0b1e537a2", "id": 1} v 0) -- 0x7f6ec815e6b0 con 0x7f6ec80d0140 2024-04-26T20:56:32.923 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.922+0000 7f6ebf400640 1 -- 172.21.15.103:0/2871894448 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "db8ba836-8a8b-49d2-9e2f-30a0b1e537a2", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f6eb8047640 con 0x7f6ec80d0140 2024-04-26T20:56:32.923 INFO:teuthology.orchestra.run.smithi103.stdout:1 2024-04-26T20:56:32.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.923+0000 7f6ec7e00640 1 -- 172.21.15.103:0/2871894448 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ec80d0140 msgr2=0x7f6ec81f4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:32.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.923+0000 7f6ec7e00640 1 --2- 172.21.15.103:0/2871894448 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ec80d0140 0x7f6ec81f4d40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8000c00 tx=0x7f6eb80046c0 comp rx=0 tx=0).stop 2024-04-26T20:56:32.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.923+0000 7f6ec7e00640 1 -- 172.21.15.103:0/2871894448 shutdown_connections 2024-04-26T20:56:32.924 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.923+0000 7f6ec7e00640 1 --2- 172.21.15.103:0/2871894448 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ec815f1d0 0x7f6ec80c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:32.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.923+0000 7f6ec7e00640 1 --2- 172.21.15.103:0/2871894448 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ec80d34b0 0x7f6ec81579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:32.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.923+0000 7f6ec7e00640 1 --2- 172.21.15.103:0/2871894448 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ec80d0140 0x7f6ec81f4d40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:32.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.923+0000 7f6ec7e00640 1 -- 172.21.15.103:0/2871894448 >> 172.21.15.103:0/2871894448 conn(0x7f6ec80d5030 msgr2=0x7f6ec80b6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:32.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.923+0000 7f6ec7e00640 1 -- 172.21.15.103:0/2871894448 shutdown_connections 2024-04-26T20:56:32.925 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:32.924+0000 7f6ec7e00640 1 -- 172.21.15.103:0/2871894448 wait complete. 2024-04-26T20:56:32.941 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd new 3e379b23-d512-4825-a8e8-8fa26f754094 2 2024-04-26T20:56:33.101 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:33.101+0000 7f68ac612040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T20:56:33.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.119+0000 7ffbb7e00640 1 Processor -- start 2024-04-26T20:56:33.120 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.119+0000 7ffbb7e00640 1 -- start start 2024-04-26T20:56:33.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.121+0000 7ffbb7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffbb815f1d0 0x7ffbb80d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:33.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.121+0000 7ffbb7e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbb815f5b0 con 0x7ffbb80d0310 2024-04-26T20:56:33.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.121+0000 7ffbb7e00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbb80d3a30 con 0x7ffbb80d34b0 2024-04-26T20:56:33.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.121+0000 7ffbb7e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ffbb80d3d10 con 0x7ffbb815f1d0 2024-04-26T20:56:33.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.121+0000 7ffbb6000640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7ffbb80d34b0 0x7ffbb80cfc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:47114/0 (socket says 172.21.15.103:47114) 2024-04-26T20:56:33.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.121+0000 7ffbb6000640 1 -- 172.21.15.103:0/2011458022 learned_addr learned my addr 172.21.15.103:0/2011458022 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-04-26T20:56:33.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.121+0000 7ffbb6a00640 1 --2- 172.21.15.103:0/2011458022 >> v2:172.21.15.28:3300/0 conn(0x7ffbb815f1d0 0x7ffbb80d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:33.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.122+0000 7ffbb5600640 1 -- 172.21.15.103:0/2011458022 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 171165987 0 0) 0x7ffbb80d3a30 con 0x7ffbb80d34b0 2024-04-26T20:56:33.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.122+0000 7ffbb5600640 1 -- 172.21.15.103:0/2011458022 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb9c003520 con 0x7ffbb80d34b0 2024-04-26T20:56:33.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.122+0000 7ffbb5600640 1 -- 172.21.15.103:0/2011458022 <== mon.1 v1:172.21.15.103:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1004225923 0 0) 0x7ffb9c003520 con 0x7ffbb80d34b0 2024-04-26T20:56:33.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.122+0000 7ffbb5600640 1 -- 172.21.15.103:0/2011458022 >> v2:172.21.15.28:3300/0 conn(0x7ffbb815f1d0 msgr2=0x7ffbb80d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:33.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.122+0000 7ffbb5600640 1 --2- 172.21.15.103:0/2011458022 >> v2:172.21.15.28:3300/0 conn(0x7ffbb815f1d0 0x7ffbb80d2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.122+0000 7ffbb5600640 1 -- 172.21.15.103:0/2011458022 >> v1:172.21.15.28:6789/0 conn(0x7ffbb80d0310 legacy=0x7ffbb80b7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:33.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.122+0000 7ffbb5600640 1 -- 172.21.15.103:0/2011458022 --> v1:172.21.15.103:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb80d3ff0 con 0x7ffbb80d34b0 2024-04-26T20:56:33.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.122+0000 7ffbb5600640 1 -- 172.21.15.103:0/2011458022 <== mon.1 v1:172.21.15.103:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7ffba4001670 con 0x7ffbb80d34b0 2024-04-26T20:56:33.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.122+0000 7ffbb5600640 1 -- 172.21.15.103:0/2011458022 >> v1:172.21.15.103:6789/0 conn(0x7ffbb80d34b0 legacy=0x7ffbb80cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:33.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.122+0000 7ffbb5600640 1 --2- 172.21.15.103:0/2011458022 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb9c0039b0 0x7ffb9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:33.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.122+0000 7ffbb5600640 1 --2- 172.21.15.103:0/2011458022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb9c004800 0x7ffb9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:33.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.122+0000 7ffbb5600640 1 --2- 172.21.15.103:0/2011458022 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb9c0051c0 0x7ffb9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:33.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.122+0000 7ffbb5600640 1 -- 172.21.15.103:0/2011458022 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb80d3d10 con 0x7ffb9c0051c0 2024-04-26T20:56:33.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.122+0000 7ffbb5600640 1 -- 172.21.15.103:0/2011458022 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9c00c060 con 0x7ffb9c004800 2024-04-26T20:56:33.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.122+0000 7ffbb5600640 1 -- 172.21.15.103:0/2011458022 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ffb9c00c260 con 0x7ffb9c0039b0 2024-04-26T20:56:33.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.122+0000 7ffbb6a00640 1 --2- 172.21.15.103:0/2011458022 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb9c0039b0 0x7ffb9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:33.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.122+0000 7ffbb6000640 1 --2- 172.21.15.103:0/2011458022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb9c004800 0x7ffb9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:33.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.123+0000 7ffbb6000640 1 -- 172.21.15.103:0/2011458022 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb9c0039b0 msgr2=0x7ffb9c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:33.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.123+0000 7ffbb6000640 1 --2- 172.21.15.103:0/2011458022 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb9c0039b0 0x7ffb9c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.123+0000 7ffbb6000640 1 -- 172.21.15.103:0/2011458022 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb9c0051c0 msgr2=0x7ffb9c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:33.125 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.123+0000 7ffbb6000640 1 --2- 172.21.15.103:0/2011458022 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb9c0051c0 0x7ffb9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.123+0000 7ffbb6000640 1 -- 172.21.15.103:0/2011458022 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb9c003520 con 0x7ffb9c004800 2024-04-26T20:56:33.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.123+0000 7ffbb7400640 1 -- 172.21.15.103:0/2011458022 reap_dead start 2024-04-26T20:56:33.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.124+0000 7ffbb6000640 1 --2- 172.21.15.103:0/2011458022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb9c004800 0x7ffb9c004c80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ffba400c240 tx=0x7ffb9c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1da9668b427166e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:33.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.124+0000 7ffbb5600640 1 -- 172.21.15.103:0/2011458022 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba4007d10 con 0x7ffb9c004800 2024-04-26T20:56:33.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.124+0000 7ffbb5600640 1 -- 172.21.15.103:0/2011458022 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffba4005ad0 con 0x7ffb9c004800 2024-04-26T20:56:33.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.124+0000 7ffbb5600640 1 -- 172.21.15.103:0/2011458022 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba4007070 con 0x7ffb9c004800 2024-04-26T20:56:33.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.125+0000 7ffbb7e00640 1 -- 172.21.15.103:0/2011458022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb9c004800 msgr2=0x7ffb9c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:33.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.125+0000 7ffbb7e00640 1 --2- 172.21.15.103:0/2011458022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb9c004800 0x7ffb9c004c80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ffba400c240 tx=0x7ffb9c005830 comp rx=0 tx=0).stop 2024-04-26T20:56:33.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.125+0000 7ffbb7e00640 1 -- 172.21.15.103:0/2011458022 shutdown_connections 2024-04-26T20:56:33.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.125+0000 7ffbb7e00640 1 --2- 172.21.15.103:0/2011458022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffb9c004800 0x7ffb9c004c80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.125+0000 7ffbb7e00640 1 -- 172.21.15.103:0/2011458022 >> 172.21.15.103:0/2011458022 conn(0x7ffbb80d5030 msgr2=0x7ffbb80d5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:33.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.125+0000 7ffbb7e00640 1 -- 172.21.15.103:0/2011458022 shutdown_connections 2024-04-26T20:56:33.127 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.125+0000 7ffbb7e00640 1 -- 172.21.15.103:0/2011458022 wait complete. 2024-04-26T20:56:33.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.127+0000 7ffbb7e00640 1 Processor -- start 2024-04-26T20:56:33.128 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.127+0000 7ffbb7e00640 1 -- start start 2024-04-26T20:56:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.128+0000 7ffbb7e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb9c004800 0x7ffbb815f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.128+0000 7ffbb7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbb8150650 0x7ffbb814d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.128+0000 7ffbb7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbb81620a0 0x7ffbb814f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.128+0000 7ffbb7e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb80bacf0 con 0x7ffbb81620a0 2024-04-26T20:56:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.128+0000 7ffbb7e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb80d45b0 con 0x7ffbb8150650 2024-04-26T20:56:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.128+0000 7ffbb7e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb80bae70 con 0x7ffb9c004800 2024-04-26T20:56:33.130 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.128+0000 7ffbb6a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb9c004800 0x7ffbb815f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.128+0000 7ffbb6000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbb8150650 0x7ffbb814d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.128+0000 7ffbb6000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbb8150650 0x7ffbb814d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46196/0 (socket says 172.21.15.103:46196) 2024-04-26T20:56:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.128+0000 7ffbb6000640 1 -- 172.21.15.103:0/1134468945 learned_addr learned my addr 172.21.15.103:0/1134468945 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-26T20:56:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.128+0000 7ffbb7400640 1 --2- 172.21.15.103:0/1134468945 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbb81620a0 0x7ffbb814f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.128+0000 7ffbb6000640 1 -- 172.21.15.103:0/1134468945 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb9c004800 msgr2=0x7ffbb815f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.128+0000 7ffbb6000640 1 --2- 172.21.15.103:0/1134468945 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb9c004800 0x7ffbb815f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.128+0000 7ffbb6000640 1 -- 172.21.15.103:0/1134468945 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbb81620a0 msgr2=0x7ffbb814f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.128+0000 7ffbb6000640 1 --2- 172.21.15.103:0/1134468945 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbb81620a0 0x7ffbb814f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.128+0000 7ffbb6000640 1 -- 172.21.15.103:0/1134468945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb814d5f0 con 0x7ffbb8150650 2024-04-26T20:56:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.128+0000 7ffbb7400640 1 --2- 172.21.15.103:0/1134468945 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbb81620a0 0x7ffbb814f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:56:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.129+0000 7ffbb6000640 1 --2- 172.21.15.103:0/1134468945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbb8150650 0x7ffbb814d0b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffba4001970 tx=0x7ffbb80d41c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.129+0000 7ffbaea00640 1 -- 172.21.15.103:0/1134468945 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba4041070 con 0x7ffbb8150650 2024-04-26T20:56:33.131 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.129+0000 7ffbaea00640 1 -- 172.21.15.103:0/1134468945 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffba4007070 con 0x7ffbb8150650 2024-04-26T20:56:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.129+0000 7ffbaea00640 1 -- 172.21.15.103:0/1134468945 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba4007d10 con 0x7ffbb8150650 2024-04-26T20:56:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.129+0000 7ffbb7e00640 1 -- 172.21.15.103:0/1134468945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb814da30 con 0x7ffbb8150650 2024-04-26T20:56:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.129+0000 7ffbb7e00640 1 -- 172.21.15.103:0/1134468945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb814dff0 con 0x7ffbb8150650 2024-04-26T20:56:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.129+0000 7ffbaea00640 1 -- 172.21.15.103:0/1134468945 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffba4044070 con 0x7ffbb8150650 2024-04-26T20:56:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.129+0000 7ffbaea00640 1 -- 172.21.15.103:0/1134468945 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7ffba4046450 con 0x7ffbb8150650 2024-04-26T20:56:33.132 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.130+0000 7ffbb7e00640 1 -- 172.21.15.103:0/1134468945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb806a250 con 0x7ffbb8150650 2024-04-26T20:56:33.133 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.133+0000 7ffbaea00640 1 -- 172.21.15.103:0/1134468945 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7ffba4007420 con 0x7ffbb8150650 2024-04-26T20:56:33.159 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:33.159+0000 7f68ac612040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T20:56:33.212 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.212+0000 7ffbb7e00640 1 -- 172.21.15.103:0/1134468945 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd new", "uuid": "3e379b23-d512-4825-a8e8-8fa26f754094", "id": 2} v 0) -- 0x7ffbb80696e0 con 0x7ffbb8150650 2024-04-26T20:56:33.220 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.219+0000 7ffbaea00640 1 -- 172.21.15.103:0/1134468945 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3e379b23-d512-4825-a8e8-8fa26f754094", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7ffba40468b0 con 0x7ffbb8150650 2024-04-26T20:56:33.220 INFO:teuthology.orchestra.run.smithi103.stdout:2 2024-04-26T20:56:33.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.220+0000 7ffbb7e00640 1 -- 172.21.15.103:0/1134468945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbb8150650 msgr2=0x7ffbb814d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:33.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.220+0000 7ffbb7e00640 1 --2- 172.21.15.103:0/1134468945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbb8150650 0x7ffbb814d0b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffba4001970 tx=0x7ffbb80d41c0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.221+0000 7ffbb7e00640 1 -- 172.21.15.103:0/1134468945 shutdown_connections 2024-04-26T20:56:33.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.221+0000 7ffbb7e00640 1 --2- 172.21.15.103:0/1134468945 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbb81620a0 0x7ffbb814f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.221+0000 7ffbb7e00640 1 --2- 172.21.15.103:0/1134468945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffbb8150650 0x7ffbb814d0b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.221+0000 7ffbb7e00640 1 --2- 172.21.15.103:0/1134468945 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffb9c004800 0x7ffbb815f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.221+0000 7ffbb7e00640 1 -- 172.21.15.103:0/1134468945 >> 172.21.15.103:0/1134468945 conn(0x7ffbb80d5030 msgr2=0x7ffbb80b6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:33.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.221+0000 7ffbb7e00640 1 -- 172.21.15.103:0/1134468945 shutdown_connections 2024-04-26T20:56:33.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.221+0000 7ffbb7e00640 1 -- 172.21.15.103:0/1134468945 wait complete. 2024-04-26T20:56:33.238 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd new 85e64a2e-c12e-44cb-a4ef-74229ad82940 3 2024-04-26T20:56:33.317 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:33.316+0000 7f1338f21040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T20:56:33.422 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.422+0000 7f1acbe00640 1 Processor -- start 2024-04-26T20:56:33.423 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.422+0000 7f1acbe00640 1 -- start start 2024-04-26T20:56:33.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.423+0000 7f1acbe00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1acc0cfc00 0x7f1acc0d0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:33.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.423+0000 7f1acbe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1acc0b7f40 0x7f1acc0b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:33.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.423+0000 7f1acbe00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1acc15f5b0 con 0x7f1acc15f1d0 2024-04-26T20:56:33.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.423+0000 7f1acbe00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f1acc0d05a0 con 0x7f1acc0b7f40 2024-04-26T20:56:33.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.423+0000 7f1acbe00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f1acc0d3a30 con 0x7f1acc0cfc00 2024-04-26T20:56:33.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.424+0000 7f1aca000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1acc0cfc00 0x7f1acc0d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:33.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.424+0000 7f1acaa00640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f1acc15f1d0 0x7f1acc0d2f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:47128/0 (socket says 172.21.15.103:47128) 2024-04-26T20:56:33.425 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.424+0000 7f1acaa00640 1 -- 172.21.15.103:0/2712631743 learned_addr learned my addr 172.21.15.103:0/2712631743 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-04-26T20:56:33.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.424+0000 7f1acb400640 1 --2- 172.21.15.103:0/2712631743 >> v2:172.21.15.28:3300/0 conn(0x7f1acc0b7f40 0x7f1acc0b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:33.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.424+0000 7f1aca000640 1 -- 172.21.15.103:0/2712631743 >> v2:172.21.15.28:3300/0 conn(0x7f1acc0b7f40 msgr2=0x7f1acc0b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:33.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.424+0000 7f1aca000640 1 --2- 172.21.15.103:0/2712631743 >> v2:172.21.15.28:3300/0 conn(0x7f1acc0b7f40 0x7f1acc0b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.424+0000 7f1ac9600640 1 -- 172.21.15.103:0/2712631743 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2176567112 0 0) 0x7f1acc15f5b0 con 0x7f1acc15f1d0 2024-04-26T20:56:33.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.424+0000 7f1aca000640 1 -- 172.21.15.103:0/2712631743 >> v1:172.21.15.103:6789/0 conn(0x7f1acc15f1d0 legacy=0x7f1acc0d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:33.426 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.424+0000 7f1aca000640 1 -- 172.21.15.103:0/2712631743 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1acc0d3d10 con 0x7f1acc0cfc00 2024-04-26T20:56:33.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.426+0000 7f1aca000640 1 --2- 172.21.15.103:0/2712631743 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1acc0cfc00 0x7f1acc0d0060 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80097a0 tx=0x7f1ab802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1644d5f6a96396e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:33.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.426+0000 7f1ac9600640 1 -- 172.21.15.103:0/2712631743 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8037070 con 0x7f1acc0cfc00 2024-04-26T20:56:33.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.426+0000 7f1ac9600640 1 -- 172.21.15.103:0/2712631743 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab8032440 con 0x7f1acc0cfc00 2024-04-26T20:56:33.427 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.426+0000 7f1ac9600640 1 -- 172.21.15.103:0/2712631743 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8032810 con 0x7f1acc0cfc00 2024-04-26T20:56:33.428 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:33.426+0000 7f1338f21040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T20:56:33.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.427+0000 7f1acbe00640 1 -- 172.21.15.103:0/2712631743 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1acc0cfc00 msgr2=0x7f1acc0d0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:33.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.427+0000 7f1acbe00640 1 --2- 172.21.15.103:0/2712631743 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1acc0cfc00 0x7f1acc0d0060 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80097a0 tx=0x7f1ab802a180 comp rx=0 tx=0).stop 2024-04-26T20:56:33.428 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.427+0000 7f1acbe00640 1 -- 172.21.15.103:0/2712631743 shutdown_connections 2024-04-26T20:56:33.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.427+0000 7f1acbe00640 1 --2- 172.21.15.103:0/2712631743 >> v2:172.21.15.28:3300/0 conn(0x7f1acc0b7f40 0x7f1acc0b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.427+0000 7f1acbe00640 1 --2- 172.21.15.103:0/2712631743 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1acc0cfc00 0x7f1acc0d0060 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.427+0000 7f1acbe00640 1 -- 172.21.15.103:0/2712631743 >> 172.21.15.103:0/2712631743 conn(0x7f1acc0d5030 msgr2=0x7f1acc0d5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:33.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.427+0000 7f1acbe00640 1 -- 172.21.15.103:0/2712631743 shutdown_connections 2024-04-26T20:56:33.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.427+0000 7f1acbe00640 1 -- 172.21.15.103:0/2712631743 wait complete. 2024-04-26T20:56:33.429 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.429+0000 7f1acbe00640 1 Processor -- start 2024-04-26T20:56:33.430 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.429+0000 7f1acbe00640 1 -- start start 2024-04-26T20:56:33.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.430+0000 7f1acbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1acc0b7f40 0x7f1acc150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:33.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.430+0000 7f1acbe00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1acc0cfc00 0x7f1acc161630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:33.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.430+0000 7f1acbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1acc15f1d0 0x7f1acc0b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:33.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.430+0000 7f1acbe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc0d45b0 con 0x7f1acc0b7f40 2024-04-26T20:56:33.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.430+0000 7f1acbe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc0d42d0 con 0x7f1acc15f1d0 2024-04-26T20:56:33.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.430+0000 7f1acbe00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f1acc0bacf0 con 0x7f1acc0cfc00 2024-04-26T20:56:33.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.431+0000 7f1acb400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1acc15f1d0 0x7f1acc0b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:33.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.431+0000 7f1acb400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1acc15f1d0 0x7f1acc0b6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46208/0 (socket says 172.21.15.103:46208) 2024-04-26T20:56:33.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.431+0000 7f1acb400640 1 -- 172.21.15.103:0/1309828341 learned_addr learned my addr 172.21.15.103:0/1309828341 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-26T20:56:33.433 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.431+0000 7f1acaa00640 1 --2- 172.21.15.103:0/1309828341 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1acc0b7f40 0x7f1acc150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:33.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.431+0000 7f1aca000640 1 --2- 172.21.15.103:0/1309828341 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1acc0cfc00 0x7f1acc161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:33.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.431+0000 7f1acb400640 1 -- 172.21.15.103:0/1309828341 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1acc0cfc00 msgr2=0x7f1acc161630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:33.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.431+0000 7f1acb400640 1 --2- 172.21.15.103:0/1309828341 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1acc0cfc00 0x7f1acc161630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.431+0000 7f1acb400640 1 -- 172.21.15.103:0/1309828341 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1acc0b7f40 msgr2=0x7f1acc150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:33.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.431+0000 7f1acb400640 1 --2- 172.21.15.103:0/1309828341 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1acc0b7f40 0x7f1acc150310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.431+0000 7f1acb400640 1 -- 172.21.15.103:0/1309828341 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1acc15f7a0 con 0x7f1acc15f1d0 2024-04-26T20:56:33.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.431+0000 7f1acaa00640 1 --2- 172.21.15.103:0/1309828341 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1acc0b7f40 0x7f1acc150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:56:33.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.431+0000 7f1acb400640 1 --2- 172.21.15.103:0/1309828341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1acc15f1d0 0x7f1acc0b6c00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400efc0 tx=0x7f1ac400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:33.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.432+0000 7f1ac2a00640 1 -- 172.21.15.103:0/1309828341 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4018020 con 0x7f1acc15f1d0 2024-04-26T20:56:33.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.432+0000 7f1ac2a00640 1 -- 172.21.15.103:0/1309828341 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ac4014480 con 0x7f1acc15f1d0 2024-04-26T20:56:33.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.432+0000 7f1acbe00640 1 -- 172.21.15.103:0/1309828341 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1acc15fa80 con 0x7f1acc15f1d0 2024-04-26T20:56:33.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.432+0000 7f1ac2a00640 1 -- 172.21.15.103:0/1309828341 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4014870 con 0x7f1acc15f1d0 2024-04-26T20:56:33.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.432+0000 7f1acbe00640 1 -- 172.21.15.103:0/1309828341 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1acc160040 con 0x7f1acc15f1d0 2024-04-26T20:56:33.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.432+0000 7f1ac2a00640 1 -- 172.21.15.103:0/1309828341 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1ac4005d10 con 0x7f1acc15f1d0 2024-04-26T20:56:33.434 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.432+0000 7f1ac2a00640 1 -- 172.21.15.103:0/1309828341 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f1ac4014da0 con 0x7f1acc15f1d0 2024-04-26T20:56:33.435 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.433+0000 7f1acbe00640 1 -- 172.21.15.103:0/1309828341 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1acc06a250 con 0x7f1acc15f1d0 2024-04-26T20:56:33.436 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.436+0000 7f1ac2a00640 1 -- 172.21.15.103:0/1309828341 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f1ac4022030 con 0x7f1acc15f1d0 2024-04-26T20:56:33.480 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:33.479+0000 7f1338f21040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T20:56:33.530 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.529+0000 7f1acbe00640 1 -- 172.21.15.103:0/1309828341 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd new", "uuid": "85e64a2e-c12e-44cb-a4ef-74229ad82940", "id": 3} v 0) -- 0x7f1acc0cf8e0 con 0x7f1acc15f1d0 2024-04-26T20:56:33.536 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.536+0000 7f1ac2a00640 1 -- 172.21.15.103:0/1309828341 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "85e64a2e-c12e-44cb-a4ef-74229ad82940", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f1ac4010040 con 0x7f1acc15f1d0 2024-04-26T20:56:33.537 INFO:teuthology.orchestra.run.smithi103.stdout:3 2024-04-26T20:56:33.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.537+0000 7f1acbe00640 1 -- 172.21.15.103:0/1309828341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1acc15f1d0 msgr2=0x7f1acc0b6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:33.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.537+0000 7f1acbe00640 1 --2- 172.21.15.103:0/1309828341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1acc15f1d0 0x7f1acc0b6c00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400efc0 tx=0x7f1ac400c490 comp rx=0 tx=0).stop 2024-04-26T20:56:33.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.537+0000 7f1acbe00640 1 -- 172.21.15.103:0/1309828341 shutdown_connections 2024-04-26T20:56:33.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.537+0000 7f1acbe00640 1 --2- 172.21.15.103:0/1309828341 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1acc15f1d0 0x7f1acc0b6c00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.537+0000 7f1acbe00640 1 --2- 172.21.15.103:0/1309828341 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1acc0cfc00 0x7f1acc161630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.537+0000 7f1acbe00640 1 --2- 172.21.15.103:0/1309828341 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1acc0b7f40 0x7f1acc150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.537+0000 7f1acbe00640 1 -- 172.21.15.103:0/1309828341 >> 172.21.15.103:0/1309828341 conn(0x7f1acc0d5030 msgr2=0x7f1acc0b8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:33.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.538+0000 7f1acbe00640 1 -- 172.21.15.103:0/1309828341 shutdown_connections 2024-04-26T20:56:33.539 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.538+0000 7f1acbe00640 1 -- 172.21.15.103:0/1309828341 wait complete. 2024-04-26T20:56:33.555 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd new 5945cd04-1c07-4d65-8642-6f5ab2bfe237 4 2024-04-26T20:56:33.679 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:33.678+0000 7f1338f21040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T20:56:33.722 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.722+0000 7f8ffa000640 1 Processor -- start 2024-04-26T20:56:33.723 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.722+0000 7f8ffa000640 1 -- start start 2024-04-26T20:56:33.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.723+0000 7f8ffa000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8ff40d2160 0x7f8ff40d33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:33.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.723+0000 7f8ffa000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8ff40d3920 0x7f8ff40b8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:33.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.723+0000 7f8ffa000640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ff40d4110 con 0x7f8ff415f680 2024-04-26T20:56:33.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.723+0000 7f8ffa000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f8ff40d43f0 con 0x7f8ff40d2160 2024-04-26T20:56:33.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.723+0000 7f8ffa000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff40d46d0 con 0x7f8ff40d3920 2024-04-26T20:56:33.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.723+0000 7f8ff3e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8ff40d3920 0x7f8ff40b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:33.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.723+0000 7f8ff3400640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f8ff415f680 0x7f8ff40d1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:47130/0 (socket says 172.21.15.103:47130) 2024-04-26T20:56:33.724 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.723+0000 7f8ff3400640 1 -- 172.21.15.103:0/1135073369 learned_addr learned my addr 172.21.15.103:0/1135073369 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-04-26T20:56:33.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.723+0000 7f8ff2a00640 1 --2- 172.21.15.103:0/1135073369 >> v2:172.21.15.28:3300/0 conn(0x7f8ff40d2160 0x7f8ff40d33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:33.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.724+0000 7f8ff3e00640 1 -- 172.21.15.103:0/1135073369 >> v2:172.21.15.28:3300/0 conn(0x7f8ff40d2160 msgr2=0x7f8ff40d33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:33.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.724+0000 7f8ff3e00640 1 --2- 172.21.15.103:0/1135073369 >> v2:172.21.15.28:3300/0 conn(0x7f8ff40d2160 0x7f8ff40d33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.724+0000 7f8ff3e00640 1 -- 172.21.15.103:0/1135073369 >> v1:172.21.15.103:6789/0 conn(0x7f8ff415f680 legacy=0x7f8ff40d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:33.725 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.724+0000 7f8ff3e00640 1 -- 172.21.15.103:0/1135073369 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff40b8660 con 0x7f8ff40d3920 2024-04-26T20:56:33.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.725+0000 7f8ff3e00640 1 --2- 172.21.15.103:0/1135073369 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8ff40d3920 0x7f8ff40b8e60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009d90 tx=0x7f8fec02a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=615d454b5f6d14cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:33.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.725+0000 7f8ff2000640 1 -- 172.21.15.103:0/1135073369 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec007740 con 0x7f8ff40d3920 2024-04-26T20:56:33.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.725+0000 7f8ff2000640 1 -- 172.21.15.103:0/1135073369 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fec0078e0 con 0x7f8ff40d3920 2024-04-26T20:56:33.726 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.725+0000 7f8ff2000640 1 -- 172.21.15.103:0/1135073369 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec007cb0 con 0x7f8ff40d3920 2024-04-26T20:56:33.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.726+0000 7f8ffa000640 1 -- 172.21.15.103:0/1135073369 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8ff40d3920 msgr2=0x7f8ff40b8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:33.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.726+0000 7f8ffa000640 1 --2- 172.21.15.103:0/1135073369 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8ff40d3920 0x7f8ff40b8e60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009d90 tx=0x7f8fec02a560 comp rx=0 tx=0).stop 2024-04-26T20:56:33.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.726+0000 7f8ffa000640 1 -- 172.21.15.103:0/1135073369 shutdown_connections 2024-04-26T20:56:33.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.726+0000 7f8ffa000640 1 --2- 172.21.15.103:0/1135073369 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8ff40d3920 0x7f8ff40b8e60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.726+0000 7f8ffa000640 1 --2- 172.21.15.103:0/1135073369 >> v2:172.21.15.28:3300/0 conn(0x7f8ff40d2160 0x7f8ff40d33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.726+0000 7f8ffa000640 1 -- 172.21.15.103:0/1135073369 >> 172.21.15.103:0/1135073369 conn(0x7f8ff406ca10 msgr2=0x7f8ff4072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:33.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.726+0000 7f8ffa000640 1 -- 172.21.15.103:0/1135073369 shutdown_connections 2024-04-26T20:56:33.727 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.726+0000 7f8ffa000640 1 -- 172.21.15.103:0/1135073369 wait complete. 2024-04-26T20:56:33.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.727+0000 7f8ffa000640 1 Processor -- start 2024-04-26T20:56:33.728 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.728+0000 7f8ffa000640 1 -- start start 2024-04-26T20:56:33.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.728+0000 7f8ffa000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ff40d2160 0x7f8ff414b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:33.730 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.728+0000 7f8ffa000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8ff40d3920 0x7f8ff4161690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:33.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.728+0000 7f8ffa000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ff415f680 0x7f8ff415bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:33.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.728+0000 7f8ffa000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff40b93a0 con 0x7f8ff415f680 2024-04-26T20:56:33.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.728+0000 7f8ffa000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff40d3db0 con 0x7f8ff40d2160 2024-04-26T20:56:33.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.728+0000 7f8ffa000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff40b9520 con 0x7f8ff40d3920 2024-04-26T20:56:33.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.729+0000 7f8ff3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ff40d2160 0x7f8ff414b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:33.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.729+0000 7f8ff3400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ff40d2160 0x7f8ff414b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46224/0 (socket says 172.21.15.103:46224) 2024-04-26T20:56:33.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.729+0000 7f8ff3400640 1 -- 172.21.15.103:0/1078891536 learned_addr learned my addr 172.21.15.103:0/1078891536 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-26T20:56:33.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.729+0000 7f8ff2a00640 1 --2- 172.21.15.103:0/1078891536 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8ff40d3920 0x7f8ff4161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:33.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.729+0000 7f8ff3e00640 1 --2- 172.21.15.103:0/1078891536 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ff415f680 0x7f8ff415bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:33.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.729+0000 7f8ff3400640 1 -- 172.21.15.103:0/1078891536 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8ff40d3920 msgr2=0x7f8ff4161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:33.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.729+0000 7f8ff3400640 1 --2- 172.21.15.103:0/1078891536 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8ff40d3920 0x7f8ff4161690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.729+0000 7f8ff3400640 1 -- 172.21.15.103:0/1078891536 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ff415f680 msgr2=0x7f8ff415bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:33.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.729+0000 7f8ff3400640 1 --2- 172.21.15.103:0/1078891536 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ff415f680 0x7f8ff415bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.729+0000 7f8ff3400640 1 -- 172.21.15.103:0/1078891536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff4158e60 con 0x7f8ff40d2160 2024-04-26T20:56:33.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.729+0000 7f8ff3400640 1 --2- 172.21.15.103:0/1078891536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ff40d2160 0x7f8ff414b440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400c570 tx=0x7f8fe400ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:33.731 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.730+0000 7f8feb400640 1 -- 172.21.15.103:0/1078891536 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4018070 con 0x7f8ff40d2160 2024-04-26T20:56:33.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.730+0000 7f8feb400640 1 -- 172.21.15.103:0/1078891536 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe40074e0 con 0x7f8ff40d2160 2024-04-26T20:56:33.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.730+0000 7f8feb400640 1 -- 172.21.15.103:0/1078891536 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4004a20 con 0x7f8ff40d2160 2024-04-26T20:56:33.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.730+0000 7f8ffa000640 1 -- 172.21.15.103:0/1078891536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff4159100 con 0x7f8ff40d2160 2024-04-26T20:56:33.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.730+0000 7f8ffa000640 1 -- 172.21.15.103:0/1078891536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff41596c0 con 0x7f8ff40d2160 2024-04-26T20:56:33.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.730+0000 7f8feb400640 1 -- 172.21.15.103:0/1078891536 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8fe401b070 con 0x7f8ff40d2160 2024-04-26T20:56:33.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.730+0000 7f8feb400640 1 -- 172.21.15.103:0/1078891536 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f8fe4012910 con 0x7f8ff40d2160 2024-04-26T20:56:33.732 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.731+0000 7f8ffa000640 1 -- 172.21.15.103:0/1078891536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff406a360 con 0x7f8ff40d2160 2024-04-26T20:56:33.734 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.733+0000 7f8feb400640 1 -- 172.21.15.103:0/1078891536 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8fe4012c50 con 0x7f8ff40d2160 2024-04-26T20:56:33.742 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:33.741+0000 7f1338f21040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T20:56:33.806 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:33.805+0000 7f1338f21040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T20:56:33.815 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.814+0000 7f8ffa000640 1 -- 172.21.15.103:0/1078891536 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5945cd04-1c07-4d65-8642-6f5ab2bfe237", "id": 4} v 0) -- 0x7f8ff4068d20 con 0x7f8ff40d2160 2024-04-26T20:56:33.821 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.820+0000 7f8feb400640 1 -- 172.21.15.103:0/1078891536 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5945cd04-1c07-4d65-8642-6f5ab2bfe237", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f8fe4004bc0 con 0x7f8ff40d2160 2024-04-26T20:56:33.821 INFO:teuthology.orchestra.run.smithi103.stdout:4 2024-04-26T20:56:33.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.821+0000 7f8ffa000640 1 -- 172.21.15.103:0/1078891536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ff40d2160 msgr2=0x7f8ff414b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:33.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.821+0000 7f8ffa000640 1 --2- 172.21.15.103:0/1078891536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ff40d2160 0x7f8ff414b440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400c570 tx=0x7f8fe400ca30 comp rx=0 tx=0).stop 2024-04-26T20:56:33.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.822+0000 7f8ffa000640 1 -- 172.21.15.103:0/1078891536 shutdown_connections 2024-04-26T20:56:33.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.822+0000 7f8ffa000640 1 --2- 172.21.15.103:0/1078891536 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ff415f680 0x7f8ff415bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.822+0000 7f8ffa000640 1 --2- 172.21.15.103:0/1078891536 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8ff40d3920 0x7f8ff4161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.822+0000 7f8ffa000640 1 --2- 172.21.15.103:0/1078891536 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ff40d2160 0x7f8ff414b440 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:33.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.822+0000 7f8ffa000640 1 -- 172.21.15.103:0/1078891536 >> 172.21.15.103:0/1078891536 conn(0x7f8ff406ca10 msgr2=0x7f8ff40cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:33.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.822+0000 7f8ffa000640 1 -- 172.21.15.103:0/1078891536 shutdown_connections 2024-04-26T20:56:33.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:33.822+0000 7f8ffa000640 1 -- 172.21.15.103:0/1078891536 wait complete. 2024-04-26T20:56:33.840 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd new 76406cbb-cc40-48c5-8698-5269d0ba54a9 5 2024-04-26T20:56:33.940 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:33.938+0000 7f1338f21040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:56:34.017 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.016+0000 7f959d800640 1 Processor -- start 2024-04-26T20:56:34.018 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.017+0000 7f959d800640 1 -- start start 2024-04-26T20:56:34.019 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.018+0000 7f959d800640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f959815f680 0x7f95980d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:34.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.018+0000 7f959d800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f95980d3af0 0x7f95980b8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:34.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.018+0000 7f959d800640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95980d4110 con 0x7f95980d1f90 2024-04-26T20:56:34.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.018+0000 7f959d800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f95980d23f0 con 0x7f95980d3af0 2024-04-26T20:56:34.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.018+0000 7f959d800640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f95980d43f0 con 0x7f959815f680 2024-04-26T20:56:34.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.018+0000 7f9596a00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f959815f680 0x7f95980d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:34.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.018+0000 7f9596a00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f959815f680 0x7f95980d1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46232/0 (socket says 172.21.15.103:46232) 2024-04-26T20:56:34.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.018+0000 7f9596a00640 1 -- 172.21.15.103:0/135441009 learned_addr learned my addr 172.21.15.103:0/135441009 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-26T20:56:34.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.018+0000 7f9597400640 1 --2- 172.21.15.103:0/135441009 >> v2:172.21.15.28:3300/0 conn(0x7f95980d3af0 0x7f95980b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:34.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.019+0000 7f9596a00640 1 -- 172.21.15.103:0/135441009 >> v2:172.21.15.28:3300/0 conn(0x7f95980d3af0 msgr2=0x7f95980b8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:34.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.019+0000 7f9596a00640 1 --2- 172.21.15.103:0/135441009 >> v2:172.21.15.28:3300/0 conn(0x7f95980d3af0 0x7f95980b8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.019+0000 7f9596a00640 1 -- 172.21.15.103:0/135441009 >> v1:172.21.15.103:6789/0 conn(0x7f95980d1f90 legacy=0x7f95980d33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:34.020 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.019+0000 7f9596a00640 1 -- 172.21.15.103:0/135441009 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95980d46d0 con 0x7f959815f680 2024-04-26T20:56:34.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.020+0000 7f9596a00640 1 --2- 172.21.15.103:0/135441009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f959815f680 0x7f95980d1a50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f95980d26e0 tx=0x7f958802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adbe1ff78353a373 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:34.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.020+0000 7f9596000640 1 -- 172.21.15.103:0/135441009 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588037070 con 0x7f959815f680 2024-04-26T20:56:34.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.020+0000 7f9596000640 1 -- 172.21.15.103:0/135441009 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9588032440 con 0x7f959815f680 2024-04-26T20:56:34.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.020+0000 7f9596000640 1 -- 172.21.15.103:0/135441009 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588032810 con 0x7f959815f680 2024-04-26T20:56:34.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.021+0000 7f959d800640 1 -- 172.21.15.103:0/135441009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f959815f680 msgr2=0x7f95980d1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:34.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.021+0000 7f959d800640 1 --2- 172.21.15.103:0/135441009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f959815f680 0x7f95980d1a50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f95980d26e0 tx=0x7f958802a270 comp rx=0 tx=0).stop 2024-04-26T20:56:34.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.021+0000 7f959d800640 1 -- 172.21.15.103:0/135441009 shutdown_connections 2024-04-26T20:56:34.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.021+0000 7f959d800640 1 --2- 172.21.15.103:0/135441009 >> v2:172.21.15.28:3300/0 conn(0x7f95980d3af0 0x7f95980b8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.021+0000 7f959d800640 1 --2- 172.21.15.103:0/135441009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f959815f680 0x7f95980d1a50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.021+0000 7f959d800640 1 -- 172.21.15.103:0/135441009 >> 172.21.15.103:0/135441009 conn(0x7f959806ca10 msgr2=0x7f9598072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:34.022 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.021+0000 7f959d800640 1 -- 172.21.15.103:0/135441009 shutdown_connections 2024-04-26T20:56:34.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.021+0000 7f959d800640 1 -- 172.21.15.103:0/135441009 wait complete. 2024-04-26T20:56:34.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.023+0000 7f959d800640 1 Processor -- start 2024-04-26T20:56:34.023 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.023+0000 7f959d800640 1 -- start start 2024-04-26T20:56:34.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.024+0000 7f959d800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f95980d1f90 0x7f95980ced10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:34.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.024+0000 7f959d800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95980d3af0 0x7f9598156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:34.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.024+0000 7f959d800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f959815f680 0x7f9598156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:34.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.024+0000 7f959d800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f95980b93a0 con 0x7f959815f680 2024-04-26T20:56:34.026 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.024+0000 7f959d800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f95980b8940 con 0x7f95980d3af0 2024-04-26T20:56:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.024+0000 7f959d800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f95980b9520 con 0x7f95980d1f90 2024-04-26T20:56:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.024+0000 7f958fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95980d3af0 0x7f9598156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.024+0000 7f9596a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f95980d1f90 0x7f95980ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.024+0000 7f958fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95980d3af0 0x7f9598156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46234/0 (socket says 172.21.15.103:46234) 2024-04-26T20:56:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.024+0000 7f958fe00640 1 -- 172.21.15.103:0/1525214759 learned_addr learned my addr 172.21.15.103:0/1525214759 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-26T20:56:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.024+0000 7f9597400640 1 --2- 172.21.15.103:0/1525214759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f959815f680 0x7f9598156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.024+0000 7f958fe00640 1 -- 172.21.15.103:0/1525214759 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f95980d1f90 msgr2=0x7f95980ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.024+0000 7f958fe00640 1 --2- 172.21.15.103:0/1525214759 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f95980d1f90 0x7f95980ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.024+0000 7f958fe00640 1 -- 172.21.15.103:0/1525214759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f959815f680 msgr2=0x7f9598156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.024+0000 7f958fe00640 1 --2- 172.21.15.103:0/1525214759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f959815f680 0x7f9598156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.024+0000 7f958fe00640 1 -- 172.21.15.103:0/1525214759 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9598157570 con 0x7f95980d3af0 2024-04-26T20:56:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.024+0000 7f9596a00640 1 --2- 172.21.15.103:0/1525214759 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f95980d1f90 0x7f95980ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:56:34.027 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.025+0000 7f958fe00640 1 --2- 172.21.15.103:0/1525214759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95980d3af0 0x7f9598156150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f958000c430 tx=0x7f958000c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:34.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.025+0000 7f958ea00640 1 -- 172.21.15.103:0/1525214759 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580017070 con 0x7f95980d3af0 2024-04-26T20:56:34.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.025+0000 7f958ea00640 1 -- 172.21.15.103:0/1525214759 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95800035f0 con 0x7f95980d3af0 2024-04-26T20:56:34.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.025+0000 7f959d800640 1 -- 172.21.15.103:0/1525214759 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9598156cd0 con 0x7f95980d3af0 2024-04-26T20:56:34.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.025+0000 7f958ea00640 1 -- 172.21.15.103:0/1525214759 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580003a20 con 0x7f95980d3af0 2024-04-26T20:56:34.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.025+0000 7f959d800640 1 -- 172.21.15.103:0/1525214759 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9598157290 con 0x7f95980d3af0 2024-04-26T20:56:34.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.025+0000 7f958ea00640 1 -- 172.21.15.103:0/1525214759 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f958001a070 con 0x7f95980d3af0 2024-04-26T20:56:34.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.025+0000 7f958ea00640 1 -- 172.21.15.103:0/1525214759 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f9580012980 con 0x7f95980d3af0 2024-04-26T20:56:34.028 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.026+0000 7f959d800640 1 -- 172.21.15.103:0/1525214759 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f959806a360 con 0x7f95980d3af0 2024-04-26T20:56:34.029 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.029+0000 7f958ea00640 1 -- 172.21.15.103:0/1525214759 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f9580021020 con 0x7f95980d3af0 2024-04-26T20:56:34.109 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.108+0000 7f959d800640 1 -- 172.21.15.103:0/1525214759 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd new", "uuid": "76406cbb-cc40-48c5-8698-5269d0ba54a9", "id": 5} v 0) -- 0x7f959815dab0 con 0x7f95980d3af0 2024-04-26T20:56:34.115 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.114+0000 7f958ea00640 1 -- 172.21.15.103:0/1525214759 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "76406cbb-cc40-48c5-8698-5269d0ba54a9", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f9580012d60 con 0x7f95980d3af0 2024-04-26T20:56:34.115 INFO:teuthology.orchestra.run.smithi103.stdout:5 2024-04-26T20:56:34.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.116+0000 7f959d800640 1 -- 172.21.15.103:0/1525214759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95980d3af0 msgr2=0x7f9598156150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:34.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.116+0000 7f959d800640 1 --2- 172.21.15.103:0/1525214759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95980d3af0 0x7f9598156150 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f958000c430 tx=0x7f958000c8f0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.116+0000 7f959d800640 1 -- 172.21.15.103:0/1525214759 shutdown_connections 2024-04-26T20:56:34.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.116+0000 7f959d800640 1 --2- 172.21.15.103:0/1525214759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f959815f680 0x7f9598156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.117 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.116+0000 7f959d800640 1 --2- 172.21.15.103:0/1525214759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95980d3af0 0x7f9598156150 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.116+0000 7f959d800640 1 --2- 172.21.15.103:0/1525214759 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f95980d1f90 0x7f95980ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.116+0000 7f959d800640 1 -- 172.21.15.103:0/1525214759 >> 172.21.15.103:0/1525214759 conn(0x7f959806ca10 msgr2=0x7f95980cfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:34.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.116+0000 7f959d800640 1 -- 172.21.15.103:0/1525214759 shutdown_connections 2024-04-26T20:56:34.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.116+0000 7f959d800640 1 -- 172.21.15.103:0/1525214759 wait complete. 2024-04-26T20:56:34.133 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd new 23ff5c7d-1a9b-4e13-96b3-c399a9d22142 6 2024-04-26T20:56:34.142 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:34.141+0000 7f68ac612040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T20:56:34.251 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:34.250+0000 7f68ac612040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T20:56:34.304 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:34.303+0000 7f68ac612040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T20:56:34.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.311+0000 7f7b27e00640 1 Processor -- start 2024-04-26T20:56:34.312 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.311+0000 7f7b27e00640 1 -- start start 2024-04-26T20:56:34.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.312+0000 7f7b27e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f7b2815f1d0 0x7f7b280d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:34.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.312+0000 7f7b27e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7b280d34b0 0x7f7b280cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:34.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.312+0000 7f7b27e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7b280d0140 0x7f7b2806c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:34.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.312+0000 7f7b27e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f7b280d05b0 con 0x7f7b280d34b0 2024-04-26T20:56:34.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.312+0000 7f7b27e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f7b2815f5b0 con 0x7f7b2815f1d0 2024-04-26T20:56:34.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.312+0000 7f7b27e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7b280d3a30 con 0x7f7b280d0140 2024-04-26T20:56:34.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.313+0000 7f7b27400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7b280d0140 0x7f7b2806c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:34.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.313+0000 7f7b27400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7b280d0140 0x7f7b2806c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:58670/0 (socket says 172.21.15.103:58670) 2024-04-26T20:56:34.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.313+0000 7f7b27400640 1 -- 172.21.15.103:0/2854848239 learned_addr learned my addr 172.21.15.103:0/2854848239 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-26T20:56:34.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.313+0000 7f7b26a00640 1 --2- 172.21.15.103:0/2854848239 >> v2:172.21.15.103:3300/0 conn(0x7f7b2815f1d0 0x7f7b280d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:34.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.313+0000 7f7b26000640 1 --2- 172.21.15.103:0/2854848239 >> v2:172.21.15.28:3300/0 conn(0x7f7b280d34b0 0x7f7b280cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:34.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.313+0000 7f7b27400640 1 -- 172.21.15.103:0/2854848239 >> v2:172.21.15.103:3300/0 conn(0x7f7b2815f1d0 msgr2=0x7f7b280d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:34.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.313+0000 7f7b27400640 1 --2- 172.21.15.103:0/2854848239 >> v2:172.21.15.103:3300/0 conn(0x7f7b2815f1d0 0x7f7b280d2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.313+0000 7f7b27400640 1 -- 172.21.15.103:0/2854848239 >> v2:172.21.15.28:3300/0 conn(0x7f7b280d34b0 msgr2=0x7f7b280cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:34.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.313+0000 7f7b27400640 1 --2- 172.21.15.103:0/2854848239 >> v2:172.21.15.28:3300/0 conn(0x7f7b280d34b0 0x7f7b280cfc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.315 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.313+0000 7f7b27400640 1 -- 172.21.15.103:0/2854848239 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b280d3d10 con 0x7f7b280d0140 2024-04-26T20:56:34.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.314+0000 7f7b27400640 1 --2- 172.21.15.103:0/2854848239 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7b280d0140 0x7f7b2806c350 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7b2808d890 tx=0x7f7b1802a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f7f32327176251f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:34.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.315+0000 7f7b25600640 1 -- 172.21.15.103:0/2854848239 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b18037070 con 0x7f7b280d0140 2024-04-26T20:56:34.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.315+0000 7f7b25600640 1 -- 172.21.15.103:0/2854848239 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b18032440 con 0x7f7b280d0140 2024-04-26T20:56:34.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.315+0000 7f7b25600640 1 -- 172.21.15.103:0/2854848239 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b18032810 con 0x7f7b280d0140 2024-04-26T20:56:34.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.315+0000 7f7b27e00640 1 -- 172.21.15.103:0/2854848239 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7b280d0140 msgr2=0x7f7b2806c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:34.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.315+0000 7f7b27e00640 1 --2- 172.21.15.103:0/2854848239 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7b280d0140 0x7f7b2806c350 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7b2808d890 tx=0x7f7b1802a1d0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.315+0000 7f7b27e00640 1 -- 172.21.15.103:0/2854848239 shutdown_connections 2024-04-26T20:56:34.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.315+0000 7f7b27e00640 1 --2- 172.21.15.103:0/2854848239 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7b280d0140 0x7f7b2806c350 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.315+0000 7f7b27e00640 1 --2- 172.21.15.103:0/2854848239 >> v2:172.21.15.28:3300/0 conn(0x7f7b280d34b0 0x7f7b280cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.315+0000 7f7b27e00640 1 --2- 172.21.15.103:0/2854848239 >> v2:172.21.15.103:3300/0 conn(0x7f7b2815f1d0 0x7f7b280d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.315+0000 7f7b27e00640 1 -- 172.21.15.103:0/2854848239 >> 172.21.15.103:0/2854848239 conn(0x7f7b280d5030 msgr2=0x7f7b280d5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:34.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.315+0000 7f7b27e00640 1 -- 172.21.15.103:0/2854848239 shutdown_connections 2024-04-26T20:56:34.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.315+0000 7f7b27e00640 1 -- 172.21.15.103:0/2854848239 wait complete. 2024-04-26T20:56:34.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.317+0000 7f7b27e00640 1 Processor -- start 2024-04-26T20:56:34.318 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.317+0000 7f7b27e00640 1 -- start start 2024-04-26T20:56:34.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.318+0000 7f7b27e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7b280d0140 0x7f7b280ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:34.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.318+0000 7f7b27e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b280d34b0 0x7f7b281f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:34.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.318+0000 7f7b27e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7b2815f1d0 0x7f7b28151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:34.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.318+0000 7f7b27e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7b280d45b0 con 0x7f7b2815f1d0 2024-04-26T20:56:34.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.318+0000 7f7b27e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7b280d42d0 con 0x7f7b280d34b0 2024-04-26T20:56:34.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.318+0000 7f7b27e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7b280bacf0 con 0x7f7b280d0140 2024-04-26T20:56:34.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.318+0000 7f7b26a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7b280d0140 0x7f7b280ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:34.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.318+0000 7f7b26a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7b280d0140 0x7f7b280ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.103:58680/0 (socket says 172.21.15.103:58680) 2024-04-26T20:56:34.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.318+0000 7f7b26a00640 1 -- 172.21.15.103:0/2391090749 learned_addr learned my addr 172.21.15.103:0/2391090749 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-26T20:56:34.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.318+0000 7f7b26000640 1 --2- 172.21.15.103:0/2391090749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b280d34b0 0x7f7b281f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:34.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.319+0000 7f7b27400640 1 --2- 172.21.15.103:0/2391090749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7b2815f1d0 0x7f7b28151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:34.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.319+0000 7f7b26a00640 1 -- 172.21.15.103:0/2391090749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b280d34b0 msgr2=0x7f7b281f5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:34.321 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.319+0000 7f7b26a00640 1 --2- 172.21.15.103:0/2391090749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b280d34b0 0x7f7b281f5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.319+0000 7f7b26a00640 1 -- 172.21.15.103:0/2391090749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7b2815f1d0 msgr2=0x7f7b28151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:34.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.319+0000 7f7b26a00640 1 --2- 172.21.15.103:0/2391090749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7b2815f1d0 0x7f7b28151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.319+0000 7f7b26a00640 1 -- 172.21.15.103:0/2391090749 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b281517d0 con 0x7f7b280d0140 2024-04-26T20:56:34.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.319+0000 7f7b26a00640 1 --2- 172.21.15.103:0/2391090749 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7b280d0140 0x7f7b280ccee0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009d10 tx=0x7f7b2000c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:34.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.319+0000 7f7b1ea00640 1 -- 172.21.15.103:0/2391090749 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20018020 con 0x7f7b280d0140 2024-04-26T20:56:34.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.319+0000 7f7b1ea00640 1 -- 172.21.15.103:0/2391090749 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b20014480 con 0x7f7b280d0140 2024-04-26T20:56:34.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.319+0000 7f7b27e00640 1 -- 172.21.15.103:0/2391090749 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b280cd570 con 0x7f7b280d0140 2024-04-26T20:56:34.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.319+0000 7f7b1ea00640 1 -- 172.21.15.103:0/2391090749 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20014830 con 0x7f7b280d0140 2024-04-26T20:56:34.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.319+0000 7f7b27e00640 1 -- 172.21.15.103:0/2391090749 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b280cdb30 con 0x7f7b280d0140 2024-04-26T20:56:34.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.320+0000 7f7b1ea00640 1 -- 172.21.15.103:0/2391090749 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7b20005d10 con 0x7f7b280d0140 2024-04-26T20:56:34.322 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.320+0000 7f7b1ea00640 1 -- 172.21.15.103:0/2391090749 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f7b200139d0 con 0x7f7b280d0140 2024-04-26T20:56:34.323 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.321+0000 7f7b27e00640 1 -- 172.21.15.103:0/2391090749 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b280d4ad0 con 0x7f7b280d0140 2024-04-26T20:56:34.325 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.325+0000 7f7b1ea00640 1 -- 172.21.15.103:0/2391090749 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7b280d4ad0 con 0x7f7b280d0140 2024-04-26T20:56:34.405 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.404+0000 7f7b27e00640 1 -- 172.21.15.103:0/2391090749 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd new", "uuid": "23ff5c7d-1a9b-4e13-96b3-c399a9d22142", "id": 6} v 0) -- 0x7f7b280766e0 con 0x7f7b280d0140 2024-04-26T20:56:34.415 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.414+0000 7f7b1ea00640 1 -- 172.21.15.103:0/2391090749 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "23ff5c7d-1a9b-4e13-96b3-c399a9d22142", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f7b280766e0 con 0x7f7b280d0140 2024-04-26T20:56:34.415 INFO:teuthology.orchestra.run.smithi103.stdout:6 2024-04-26T20:56:34.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.415+0000 7f7b27e00640 1 -- 172.21.15.103:0/2391090749 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7b280d0140 msgr2=0x7f7b280ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:34.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.415+0000 7f7b27e00640 1 --2- 172.21.15.103:0/2391090749 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7b280d0140 0x7f7b280ccee0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009d10 tx=0x7f7b2000c490 comp rx=0 tx=0).stop 2024-04-26T20:56:34.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.416+0000 7f7b27e00640 1 -- 172.21.15.103:0/2391090749 shutdown_connections 2024-04-26T20:56:34.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.416+0000 7f7b27e00640 1 --2- 172.21.15.103:0/2391090749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7b2815f1d0 0x7f7b28151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.416+0000 7f7b27e00640 1 --2- 172.21.15.103:0/2391090749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7b280d34b0 0x7f7b281f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.416+0000 7f7b27e00640 1 --2- 172.21.15.103:0/2391090749 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7b280d0140 0x7f7b280ccee0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.416+0000 7f7b27e00640 1 -- 172.21.15.103:0/2391090749 >> 172.21.15.103:0/2391090749 conn(0x7f7b280d5030 msgr2=0x7f7b2806cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:34.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.416+0000 7f7b27e00640 1 -- 172.21.15.103:0/2391090749 shutdown_connections 2024-04-26T20:56:34.417 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.416+0000 7f7b27e00640 1 -- 172.21.15.103:0/2391090749 wait complete. 2024-04-26T20:56:34.418 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:34.417+0000 7f1338f21040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T20:56:34.433 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --cluster ceph osd new 26e56afd-dfe1-498a-bced-06e368861524 7 2024-04-26T20:56:34.507 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:34.506+0000 7f68ac612040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T20:56:34.577 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:34.576+0000 7f68ac612040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T20:56:34.641 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:34.641+0000 7f68ac612040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T20:56:34.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.653+0000 7fbd9ae00640 1 Processor -- start 2024-04-26T20:56:34.654 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.653+0000 7fbd9ae00640 1 -- start start 2024-04-26T20:56:34.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.654+0000 7fbd9ae00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fbd940b9e70 0x7fbd940d3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:34.655 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.654+0000 7fbd9ae00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd940d4110 con 0x7fbd9415f680 2024-04-26T20:56:34.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.654+0000 7fbd9ae00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd940d43f0 con 0x7fbd940d2160 2024-04-26T20:56:34.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.654+0000 7fbd9ae00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fbd940d46d0 con 0x7fbd940b9e70 2024-04-26T20:56:34.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.655+0000 7fbd93400640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7fbd940d2160 0x7fbd940d33e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.103:47148/0 (socket says 172.21.15.103:47148) 2024-04-26T20:56:34.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.655+0000 7fbd93400640 1 -- 172.21.15.103:0/593648716 learned_addr learned my addr 172.21.15.103:0/593648716 (peer_addr_for_me v1:172.21.15.103:0/0) 2024-04-26T20:56:34.656 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.655+0000 7fbd98e00640 1 --2- 172.21.15.103:0/593648716 >> v2:172.21.15.103:3300/0 conn(0x7fbd940b9e70 0x7fbd940d3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:34.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.655+0000 7fbd92a00640 1 -- 172.21.15.103:0/593648716 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3124817561 0 0) 0x7fbd940d43f0 con 0x7fbd940d2160 2024-04-26T20:56:34.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.655+0000 7fbd92a00640 1 -- 172.21.15.103:0/593648716 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd78003520 con 0x7fbd940d2160 2024-04-26T20:56:34.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.655+0000 7fbd98e00640 1 -- 172.21.15.103:0/593648716 >> v1:172.21.15.103:6789/0 conn(0x7fbd940d2160 legacy=0x7fbd940d33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:34.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.655+0000 7fbd98e00640 1 -- 172.21.15.103:0/593648716 >> v1:172.21.15.28:6789/0 conn(0x7fbd9415f680 legacy=0x7fbd940d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:34.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.655+0000 7fbd98e00640 1 -- 172.21.15.103:0/593648716 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd940b8660 con 0x7fbd940b9e70 2024-04-26T20:56:34.657 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.656+0000 7fbd98e00640 1 --2- 172.21.15.103:0/593648716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd940b9e70 0x7fbd940d3af0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009d90 tx=0x7fbd8c02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df7899834f6ddea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:34.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.656+0000 7fbd92a00640 1 -- 172.21.15.103:0/593648716 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c007740 con 0x7fbd940b9e70 2024-04-26T20:56:34.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.657+0000 7fbd92a00640 1 -- 172.21.15.103:0/593648716 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd8c0078e0 con 0x7fbd940b9e70 2024-04-26T20:56:34.658 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.657+0000 7fbd92a00640 1 -- 172.21.15.103:0/593648716 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c007c90 con 0x7fbd940b9e70 2024-04-26T20:56:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.657+0000 7fbd9ae00640 1 -- 172.21.15.103:0/593648716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd940b9e70 msgr2=0x7fbd940d3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.657+0000 7fbd9ae00640 1 --2- 172.21.15.103:0/593648716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd940b9e70 0x7fbd940d3af0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c009d90 tx=0x7fbd8c02a6f0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.657+0000 7fbd9ae00640 1 -- 172.21.15.103:0/593648716 shutdown_connections 2024-04-26T20:56:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.657+0000 7fbd9ae00640 1 --2- 172.21.15.103:0/593648716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd940b9e70 0x7fbd940d3af0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.657+0000 7fbd9ae00640 1 -- 172.21.15.103:0/593648716 >> 172.21.15.103:0/593648716 conn(0x7fbd9406ca10 msgr2=0x7fbd94072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.657+0000 7fbd9ae00640 1 -- 172.21.15.103:0/593648716 shutdown_connections 2024-04-26T20:56:34.659 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.657+0000 7fbd9ae00640 1 -- 172.21.15.103:0/593648716 wait complete. 2024-04-26T20:56:34.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.659+0000 7fbd9ae00640 1 Processor -- start 2024-04-26T20:56:34.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.659+0000 7fbd9ae00640 1 -- start start 2024-04-26T20:56:34.662 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.660+0000 7fbd9ae00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbd940b9e70 0x7fbd941558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:34.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.660+0000 7fbd9ae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd940d2160 0x7fbd940cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:34.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.660+0000 7fbd9ae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd9415f680 0x7fbd9415b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:34.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.660+0000 7fbd9ae00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9416a230 con 0x7fbd9415f680 2024-04-26T20:56:34.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.660+0000 7fbd9ae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9416a0b0 con 0x7fbd940d2160 2024-04-26T20:56:34.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.660+0000 7fbd9ae00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9416a3b0 con 0x7fbd940b9e70 2024-04-26T20:56:34.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.660+0000 7fbd93400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd940d2160 0x7fbd940cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:34.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.660+0000 7fbd93e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbd940b9e70 0x7fbd941558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:34.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.660+0000 7fbd93400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd940d2160 0x7fbd940cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:46272/0 (socket says 172.21.15.103:46272) 2024-04-26T20:56:34.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.660+0000 7fbd93400640 1 -- 172.21.15.103:0/4216364794 learned_addr learned my addr 172.21.15.103:0/4216364794 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-26T20:56:34.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.660+0000 7fbd98e00640 1 --2- 172.21.15.103:0/4216364794 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd9415f680 0x7fbd9415b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:34.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.661+0000 7fbd93400640 1 -- 172.21.15.103:0/4216364794 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbd940b9e70 msgr2=0x7fbd941558f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:34.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.661+0000 7fbd93400640 1 --2- 172.21.15.103:0/4216364794 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbd940b9e70 0x7fbd941558f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.663 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.661+0000 7fbd93400640 1 -- 172.21.15.103:0/4216364794 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd9415f680 msgr2=0x7fbd9415b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.661+0000 7fbd93400640 1 --2- 172.21.15.103:0/4216364794 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd9415f680 0x7fbd9415b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.661+0000 7fbd93400640 1 -- 172.21.15.103:0/4216364794 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd94155e30 con 0x7fbd940d2160 2024-04-26T20:56:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.661+0000 7fbd93e00640 1 --2- 172.21.15.103:0/4216364794 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbd940b9e70 0x7fbd941558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:56:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.661+0000 7fbd93400640 1 --2- 172.21.15.103:0/4216364794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd940d2160 0x7fbd940cc110 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbd80001260 tx=0x7fbd8000c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.661+0000 7fbd8be00640 1 -- 172.21.15.103:0/4216364794 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd80017070 con 0x7fbd940d2160 2024-04-26T20:56:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.661+0000 7fbd8be00640 1 -- 172.21.15.103:0/4216364794 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd80007480 con 0x7fbd940d2160 2024-04-26T20:56:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.661+0000 7fbd8be00640 1 -- 172.21.15.103:0/4216364794 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd800059c0 con 0x7fbd940d2160 2024-04-26T20:56:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.661+0000 7fbd9ae00640 1 -- 172.21.15.103:0/4216364794 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd940cca80 con 0x7fbd940d2160 2024-04-26T20:56:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.662+0000 7fbd9ae00640 1 -- 172.21.15.103:0/4216364794 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd940cd040 con 0x7fbd940d2160 2024-04-26T20:56:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.662+0000 7fbd8be00640 1 -- 172.21.15.103:0/4216364794 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbd8001a070 con 0x7fbd940d2160 2024-04-26T20:56:34.664 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.662+0000 7fbd8be00640 1 -- 172.21.15.103:0/4216364794 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fbd80012a60 con 0x7fbd940d2160 2024-04-26T20:56:34.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.663+0000 7fbd9ae00640 1 -- 172.21.15.103:0/4216364794 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd940c9cf0 con 0x7fbd940d2160 2024-04-26T20:56:34.665 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.665+0000 7fbd8be00640 1 -- 172.21.15.103:0/4216364794 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fbd940c9cf0 con 0x7fbd940d2160 2024-04-26T20:56:34.727 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:34.726+0000 7f1338f21040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T20:56:34.746 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.745+0000 7fbd9ae00640 1 -- 172.21.15.103:0/4216364794 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd new", "uuid": "26e56afd-dfe1-498a-bced-06e368861524", "id": 7} v 0) -- 0x7fbd94068d20 con 0x7fbd940d2160 2024-04-26T20:56:34.755 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.755+0000 7fbd8be00640 1 -- 172.21.15.103:0/4216364794 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "26e56afd-dfe1-498a-bced-06e368861524", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fbd80021020 con 0x7fbd940d2160 2024-04-26T20:56:34.756 INFO:teuthology.orchestra.run.smithi103.stdout:7 2024-04-26T20:56:34.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.756+0000 7fbd9ae00640 1 -- 172.21.15.103:0/4216364794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd940d2160 msgr2=0x7fbd940cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:34.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.756+0000 7fbd9ae00640 1 --2- 172.21.15.103:0/4216364794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd940d2160 0x7fbd940cc110 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbd80001260 tx=0x7fbd8000c960 comp rx=0 tx=0).stop 2024-04-26T20:56:34.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.756+0000 7fbd9ae00640 1 -- 172.21.15.103:0/4216364794 shutdown_connections 2024-04-26T20:56:34.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.756+0000 7fbd9ae00640 1 --2- 172.21.15.103:0/4216364794 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd9415f680 0x7fbd9415b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.756+0000 7fbd9ae00640 1 --2- 172.21.15.103:0/4216364794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd940d2160 0x7fbd940cc110 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.756+0000 7fbd9ae00640 1 --2- 172.21.15.103:0/4216364794 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbd940b9e70 0x7fbd941558f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:34.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.756+0000 7fbd9ae00640 1 -- 172.21.15.103:0/4216364794 >> 172.21.15.103:0/4216364794 conn(0x7fbd9406ca10 msgr2=0x7fbd940d4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:34.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.756+0000 7fbd9ae00640 1 -- 172.21.15.103:0/4216364794 shutdown_connections 2024-04-26T20:56:34.757 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:56:34.756+0000 7fbd9ae00640 1 -- 172.21.15.103:0/4216364794 wait complete. 2024-04-26T20:56:34.772 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-26T20:56:34.772 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-26T20:56:34.778 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:34.778+0000 7f68ac612040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T20:56:34.780 INFO:tasks.ceph.osd.0:Started 2024-04-26T20:56:34.780 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-26T20:56:34.780 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-26T20:56:34.789 INFO:tasks.ceph.osd.1:Started 2024-04-26T20:56:34.789 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-26T20:56:34.789 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-26T20:56:34.796 INFO:tasks.ceph.osd.2:Started 2024-04-26T20:56:34.796 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-26T20:56:34.796 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-26T20:56:34.796 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:34.790+0000 7f1338f21040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T20:56:34.801 INFO:tasks.ceph.osd.3:Started 2024-04-26T20:56:34.801 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-26T20:56:34.801 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-26T20:56:34.807 INFO:tasks.ceph.osd.4:Started 2024-04-26T20:56:34.807 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-26T20:56:34.807 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-26T20:56:34.812 INFO:tasks.ceph.osd.5:Started 2024-04-26T20:56:34.812 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-26T20:56:34.812 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-26T20:56:34.817 INFO:tasks.ceph.osd.6:Started 2024-04-26T20:56:34.817 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-26T20:56:34.817 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-26T20:56:34.822 INFO:tasks.ceph.osd.7:Started 2024-04-26T20:56:34.823 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:56:34.989 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:34.988+0000 7f1338f21040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T20:56:35.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.033+0000 7fa5aa000640 1 Processor -- start 2024-04-26T20:56:35.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.033+0000 7fa5aa000640 1 -- start start 2024-04-26T20:56:35.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.034+0000 7fa5aa000640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa5a415f1b0 0x7fa5a40d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:35.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.034+0000 7fa5aa000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5a40d5540 0x7fa5a40b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:35.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.034+0000 7fa5aa000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a40d0710 con 0x7fa5a40cf700 2024-04-26T20:56:35.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.035+0000 7fa5aa000640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a40cff00 con 0x7fa5a415f1b0 2024-04-26T20:56:35.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.035+0000 7fa5aa000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a40d4270 con 0x7fa5a40d5540 2024-04-26T20:56:35.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.035+0000 7fa5a3e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa5a40cf700 0x7fa5a40d4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57380/0 (socket says 172.21.15.28:57380) 2024-04-26T20:56:35.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.035+0000 7fa5a3e00640 1 -- 172.21.15.28:0/1969655833 learned_addr learned my addr 172.21.15.28:0/1969655833 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:35.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.035+0000 7fa5a8c00640 1 --2- 172.21.15.28:0/1969655833 >> v2:172.21.15.103:3300/0 conn(0x7fa5a415f1b0 0x7fa5a40d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:35.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.035+0000 7fa5a9600640 1 --2- 172.21.15.28:0/1969655833 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5a40d5540 0x7fa5a40b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:35.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.035+0000 7fa5a3400640 1 -- 172.21.15.28:0/1969655833 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2325449745 0 0) 0x7fa590001110 con 0x7fa5a40cf700 2024-04-26T20:56:35.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.035+0000 7fa5a3400640 1 -- 172.21.15.28:0/1969655833 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa58c003520 con 0x7fa5a40cf700 2024-04-26T20:56:35.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.035+0000 7fa5a9600640 1 -- 172.21.15.28:0/1969655833 >> v2:172.21.15.103:3300/0 conn(0x7fa5a415f1b0 msgr2=0x7fa5a40d4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:35.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.035+0000 7fa5a9600640 1 --2- 172.21.15.28:0/1969655833 >> v2:172.21.15.103:3300/0 conn(0x7fa5a415f1b0 0x7fa5a40d4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:35.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.035+0000 7fa5a9600640 1 -- 172.21.15.28:0/1969655833 >> v1:172.21.15.28:6789/0 conn(0x7fa5a40cf700 legacy=0x7fa5a40d4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:35.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.035+0000 7fa5a9600640 1 -- 172.21.15.28:0/1969655833 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a40d0150 con 0x7fa5a40d5540 2024-04-26T20:56:35.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.035+0000 7fa5a8c00640 1 --2- 172.21.15.28:0/1969655833 >> v2:172.21.15.103:3300/0 conn(0x7fa5a415f1b0 0x7fa5a40d4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:56:35.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.037+0000 7fa5a9600640 1 --2- 172.21.15.28:0/1969655833 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5a40d5540 0x7fa5a40b80b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009d90 tx=0x7fa59c02a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa9c2415379be943 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:35.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.037+0000 7fa5a3400640 1 -- 172.21.15.28:0/1969655833 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c007740 con 0x7fa5a40d5540 2024-04-26T20:56:35.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.037+0000 7fa5a3400640 1 -- 172.21.15.28:0/1969655833 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59c0078e0 con 0x7fa5a40d5540 2024-04-26T20:56:35.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.037+0000 7fa5a3400640 1 -- 172.21.15.28:0/1969655833 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c007cb0 con 0x7fa5a40d5540 2024-04-26T20:56:35.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.038+0000 7fa5aa000640 1 -- 172.21.15.28:0/1969655833 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5a40d5540 msgr2=0x7fa5a40b80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:35.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.038+0000 7fa5aa000640 1 --2- 172.21.15.28:0/1969655833 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5a40d5540 0x7fa5a40b80b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009d90 tx=0x7fa59c02a6f0 comp rx=0 tx=0).stop 2024-04-26T20:56:35.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.038+0000 7fa5aa000640 1 -- 172.21.15.28:0/1969655833 shutdown_connections 2024-04-26T20:56:35.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.038+0000 7fa5aa000640 1 --2- 172.21.15.28:0/1969655833 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5a40d5540 0x7fa5a40b80b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:35.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.038+0000 7fa5aa000640 1 --2- 172.21.15.28:0/1969655833 >> v2:172.21.15.103:3300/0 conn(0x7fa5a415f1b0 0x7fa5a40d4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:35.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.038+0000 7fa5aa000640 1 -- 172.21.15.28:0/1969655833 >> 172.21.15.28:0/1969655833 conn(0x7fa5a406c770 msgr2=0x7fa5a406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:35.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.038+0000 7fa5aa000640 1 -- 172.21.15.28:0/1969655833 shutdown_connections 2024-04-26T20:56:35.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.038+0000 7fa5aa000640 1 -- 172.21.15.28:0/1969655833 wait complete. 2024-04-26T20:56:35.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.040+0000 7fa5aa000640 1 Processor -- start 2024-04-26T20:56:35.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.040+0000 7fa5aa000640 1 -- start start 2024-04-26T20:56:35.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.041+0000 7fa5aa000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5a40cf700 0x7fa5a415f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:35.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.041+0000 7fa5aa000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5a40d5540 0x7fa5a40b9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:35.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.041+0000 7fa5aa000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5a415f1b0 0x7fa5a414bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:35.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.041+0000 7fa5aa000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a415f590 con 0x7fa5a40d5540 2024-04-26T20:56:35.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.041+0000 7fa5aa000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a40d3f00 con 0x7fa5a415f1b0 2024-04-26T20:56:35.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.041+0000 7fa5aa000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a40b4410 con 0x7fa5a40cf700 2024-04-26T20:56:35.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.041+0000 7fa5a3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5a40d5540 0x7fa5a40b9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:35.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.041+0000 7fa5a3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5a40d5540 0x7fa5a40b9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52542/0 (socket says 172.21.15.28:52542) 2024-04-26T20:56:35.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.041+0000 7fa5a8c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5a40cf700 0x7fa5a415f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:35.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.041+0000 7fa5a3e00640 1 -- 172.21.15.28:0/1565122695 learned_addr learned my addr 172.21.15.28:0/1565122695 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:35.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.041+0000 7fa5a9600640 1 --2- 172.21.15.28:0/1565122695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5a415f1b0 0x7fa5a414bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:35.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.042+0000 7fa5a3e00640 1 -- 172.21.15.28:0/1565122695 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5a40cf700 msgr2=0x7fa5a415f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:35.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.042+0000 7fa5a3e00640 1 --2- 172.21.15.28:0/1565122695 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5a40cf700 0x7fa5a415f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:35.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.042+0000 7fa5a3e00640 1 -- 172.21.15.28:0/1565122695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5a415f1b0 msgr2=0x7fa5a414bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:35.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.042+0000 7fa5a3e00640 1 --2- 172.21.15.28:0/1565122695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5a415f1b0 0x7fa5a414bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:35.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.042+0000 7fa5a3e00640 1 -- 172.21.15.28:0/1565122695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a415ff40 con 0x7fa5a40d5540 2024-04-26T20:56:35.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.042+0000 7fa5a3e00640 1 --2- 172.21.15.28:0/1565122695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5a40d5540 0x7fa5a40b9740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa59000c730 tx=0x7fa59000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:35.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.042+0000 7fa5a0c00640 1 -- 172.21.15.28:0/1565122695 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590017070 con 0x7fa5a40d5540 2024-04-26T20:56:35.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.042+0000 7fa5a0c00640 1 -- 172.21.15.28:0/1565122695 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa590005ad0 con 0x7fa5a40d5540 2024-04-26T20:56:35.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.042+0000 7fa5a0c00640 1 -- 172.21.15.28:0/1565122695 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa590007e20 con 0x7fa5a40d5540 2024-04-26T20:56:35.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.042+0000 7fa5aa000640 1 -- 172.21.15.28:0/1565122695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a4160220 con 0x7fa5a40d5540 2024-04-26T20:56:35.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.042+0000 7fa5aa000640 1 -- 172.21.15.28:0/1565122695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a4160ac0 con 0x7fa5a40d5540 2024-04-26T20:56:35.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.043+0000 7fa5a0c00640 1 -- 172.21.15.28:0/1565122695 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa59001a070 con 0x7fa5a40d5540 2024-04-26T20:56:35.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.043+0000 7fa5a0c00640 1 -- 172.21.15.28:0/1565122695 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fa590012ad0 con 0x7fa5a40d5540 2024-04-26T20:56:35.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.044+0000 7fa5aa000640 1 -- 172.21.15.28:0/1565122695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a4160610 con 0x7fa5a40d5540 2024-04-26T20:56:35.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.046+0000 7fa5a0c00640 1 -- 172.21.15.28:0/1565122695 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa590021020 con 0x7fa5a40d5540 2024-04-26T20:56:35.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.111+0000 7fa5aa000640 1 -- 172.21.15.28:0/1565122695 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5a406a360 con 0x7fa5a40d5540 2024-04-26T20:56:35.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.113+0000 7fa5a0c00640 1 -- 172.21.15.28:0/1565122695 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fa5a40d0710 con 0x7fa5a40d5540 2024-04-26T20:56:35.114 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:56:35.114 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":10,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:56:34.747682+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:56:35.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.114+0000 7fa5aa000640 1 -- 172.21.15.28:0/1565122695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5a40d5540 msgr2=0x7fa5a40b9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:35.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.114+0000 7fa5aa000640 1 --2- 172.21.15.28:0/1565122695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5a40d5540 0x7fa5a40b9740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa59000c730 tx=0x7fa59000cbf0 comp rx=0 tx=0).stop 2024-04-26T20:56:35.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.114+0000 7fa5aa000640 1 -- 172.21.15.28:0/1565122695 shutdown_connections 2024-04-26T20:56:35.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.114+0000 7fa5aa000640 1 --2- 172.21.15.28:0/1565122695 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5a415f1b0 0x7fa5a414bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:35.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.114+0000 7fa5aa000640 1 --2- 172.21.15.28:0/1565122695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5a40d5540 0x7fa5a40b9740 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:35.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.114+0000 7fa5aa000640 1 --2- 172.21.15.28:0/1565122695 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5a40cf700 0x7fa5a415f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:35.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.114+0000 7fa5aa000640 1 -- 172.21.15.28:0/1565122695 >> 172.21.15.28:0/1565122695 conn(0x7fa5a406c770 msgr2=0x7fa5a40d4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:35.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.114+0000 7fa5aa000640 1 -- 172.21.15.28:0/1565122695 shutdown_connections 2024-04-26T20:56:35.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:35.114+0000 7fa5aa000640 1 -- 172.21.15.28:0/1565122695 wait complete. 2024-04-26T20:56:35.135 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-26T20:56:35.136 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-26T20:56:35.289 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:35.288+0000 7f68ac612040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T20:56:35.332 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:35.331+0000 7f1338f21040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T20:56:35.391 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:35.389+0000 7f1338f21040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T20:56:35.474 INFO:tasks.ceph.osd.1.smithi028.stderr:2024-04-26T20:56:35.473+0000 7f503240d640 -1 Falling back to public interface 2024-04-26T20:56:35.478 INFO:tasks.ceph.osd.5.smithi103.stderr:2024-04-26T20:56:35.477+0000 7f94e6415640 -1 Falling back to public interface 2024-04-26T20:56:35.483 INFO:tasks.ceph.osd.7.smithi103.stderr:2024-04-26T20:56:35.483+0000 7f2d1c3fc640 -1 Falling back to public interface 2024-04-26T20:56:35.496 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T20:56:35.495+0000 7ffb511c0640 -1 Falling back to public interface 2024-04-26T20:56:35.504 INFO:tasks.ceph.osd.2.smithi028.stderr:2024-04-26T20:56:35.503+0000 7f6c70c9a640 -1 Falling back to public interface 2024-04-26T20:56:35.507 INFO:tasks.ceph.osd.3.smithi028.stderr:2024-04-26T20:56:35.506+0000 7f6497015640 -1 Falling back to public interface 2024-04-26T20:56:35.524 INFO:tasks.ceph.osd.6.smithi103.stderr:2024-04-26T20:56:35.523+0000 7f3353c14640 -1 Falling back to public interface 2024-04-26T20:56:35.525 INFO:tasks.ceph.osd.4.smithi103.stderr:2024-04-26T20:56:35.524+0000 7f17956db640 -1 Falling back to public interface 2024-04-26T20:56:35.589 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:35.587+0000 7f1338f21040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T20:56:35.595 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:35.594+0000 7f68ac612040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T20:56:35.655 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:35.654+0000 7f68ac612040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T20:56:35.762 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:35.762+0000 7f1338f21040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T20:56:35.789 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:35.788+0000 7f68ac612040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T20:56:35.825 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:35.825+0000 7f1338f21040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T20:56:35.909 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:35.909+0000 7f1338f21040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T20:56:35.940 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T20:56:35.965 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:35.965+0000 7f1338f21040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T20:56:36.097 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.096+0000 7f55e7800640 1 Processor -- start 2024-04-26T20:56:36.097 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.097+0000 7f55e7800640 1 -- start start 2024-04-26T20:56:36.098 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.098+0000 7f55e7800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55e01468d0 0x7f55e00cb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:36.098 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.098+0000 7f55e7800640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f55e006e6a0 0x7f55e00cd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:36.098 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.098+0000 7f55e7800640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55e0146cb0 con 0x7f55e0072050 2024-04-26T20:56:36.098 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.098+0000 7f55e7800640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f55e00ccd70 con 0x7f55e006e6a0 2024-04-26T20:56:36.098 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.098+0000 7f55e7800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f55e00cc430 con 0x7f55e01468d0 2024-04-26T20:56:36.099 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.098+0000 7f55e5800640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f55e006e6a0 0x7f55e00cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:36.099 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.098+0000 7f55e4e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55e01468d0 0x7f55e00cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:36.099 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.098+0000 7f55e4e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55e01468d0 0x7f55e00cb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.28:39916/0 (socket says 172.21.15.28:39916) 2024-04-26T20:56:36.099 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.098+0000 7f55e4e00640 1 -- 172.21.15.28:0/158608543 learned_addr learned my addr 172.21.15.28:0/158608543 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:36.099 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.099+0000 7f55e4e00640 1 -- 172.21.15.28:0/158608543 >> v2:172.21.15.103:3300/0 conn(0x7f55e006e6a0 msgr2=0x7f55e00cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:36.099 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.099+0000 7f55e4e00640 1 --2- 172.21.15.28:0/158608543 >> v2:172.21.15.103:3300/0 conn(0x7f55e006e6a0 0x7f55e00cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:36.099 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.099+0000 7f55e4e00640 1 -- 172.21.15.28:0/158608543 >> v1:172.21.15.103:6789/0 conn(0x7f55e0072050 legacy=0x7f55e006df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:36.100 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.099+0000 7f55e4e00640 1 -- 172.21.15.28:0/158608543 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e00cbb90 con 0x7f55e01468d0 2024-04-26T20:56:36.100 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.099+0000 7f55e5800640 1 --2- 172.21.15.28:0/158608543 >> v2:172.21.15.103:3300/0 conn(0x7f55e006e6a0 0x7f55e00cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:56:36.100 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.100+0000 7f55e4e00640 1 --2- 172.21.15.28:0/158608543 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55e01468d0 0x7f55e00cb290 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009870 tx=0x7f55d402a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=190cb2343cdcbc06 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:36.100 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.101+0000 7f55df400640 1 -- 172.21.15.28:0/158608543 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4037070 con 0x7f55e01468d0 2024-04-26T20:56:36.101 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.101+0000 7f55df400640 1 -- 172.21.15.28:0/158608543 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d4032440 con 0x7f55e01468d0 2024-04-26T20:56:36.101 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.101+0000 7f55df400640 1 -- 172.21.15.28:0/158608543 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4032810 con 0x7f55e01468d0 2024-04-26T20:56:36.101 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.101+0000 7f55e7800640 1 -- 172.21.15.28:0/158608543 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55e01468d0 msgr2=0x7f55e00cb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:36.101 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.101+0000 7f55e7800640 1 --2- 172.21.15.28:0/158608543 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55e01468d0 0x7f55e00cb290 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009870 tx=0x7f55d402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:56:36.102 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.101+0000 7f55e7800640 1 -- 172.21.15.28:0/158608543 shutdown_connections 2024-04-26T20:56:36.102 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.101+0000 7f55e7800640 1 --2- 172.21.15.28:0/158608543 >> v2:172.21.15.103:3300/0 conn(0x7f55e006e6a0 0x7f55e00cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:36.102 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.101+0000 7f55e7800640 1 --2- 172.21.15.28:0/158608543 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55e01468d0 0x7f55e00cb290 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:36.102 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.101+0000 7f55e7800640 1 -- 172.21.15.28:0/158608543 >> 172.21.15.28:0/158608543 conn(0x7f55e00b2910 msgr2=0x7f55e00b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:36.102 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.101+0000 7f55e7800640 1 -- 172.21.15.28:0/158608543 shutdown_connections 2024-04-26T20:56:36.102 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.101+0000 7f55e7800640 1 -- 172.21.15.28:0/158608543 wait complete. 2024-04-26T20:56:36.103 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.103+0000 7f55e7800640 1 Processor -- start 2024-04-26T20:56:36.103 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.103+0000 7f55e7800640 1 -- start start 2024-04-26T20:56:36.106 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.104+0000 7f55e7800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e006e6a0 0x7f55e00cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:36.106 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.104+0000 7f55e7800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55e0072050 0x7f55e00b6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:36.106 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.104+0000 7f55e7800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f55e01468d0 0x7f55e0143680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:36.106 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.104+0000 7f55e7800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f55e00cb7d0 con 0x7f55e01468d0 2024-04-26T20:56:36.106 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.104+0000 7f55e7800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f55e00cc150 con 0x7f55e006e6a0 2024-04-26T20:56:36.106 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.104+0000 7f55e7800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f55e00cb950 con 0x7f55e0072050 2024-04-26T20:56:36.106 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.104+0000 7f55e5800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f55e01468d0 0x7f55e0143680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:36.106 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.105+0000 7f55e4e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e006e6a0 0x7f55e00cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:36.106 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.105+0000 7f55e5800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f55e01468d0 0x7f55e0143680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52558/0 (socket says 172.21.15.28:52558) 2024-04-26T20:56:36.107 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.105+0000 7f55dfe00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55e0072050 0x7f55e00b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:36.107 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.105+0000 7f55e5800640 1 -- 172.21.15.28:0/1412883763 learned_addr learned my addr 172.21.15.28:0/1412883763 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:36.107 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.105+0000 7f55e4e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e006e6a0 0x7f55e00cde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.28:43540/0 (socket says 172.21.15.28:43540) 2024-04-26T20:56:36.107 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.105+0000 7f55e5800640 1 -- 172.21.15.28:0/1412883763 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55e0072050 msgr2=0x7f55e00b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:36.107 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.105+0000 7f55e5800640 1 --2- 172.21.15.28:0/1412883763 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55e0072050 0x7f55e00b6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:36.107 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.105+0000 7f55e5800640 1 -- 172.21.15.28:0/1412883763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e006e6a0 msgr2=0x7f55e00cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:36.107 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.105+0000 7f55e5800640 1 --2- 172.21.15.28:0/1412883763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e006e6a0 0x7f55e00cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:36.107 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.105+0000 7f55e5800640 1 -- 172.21.15.28:0/1412883763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55e0141970 con 0x7f55e01468d0 2024-04-26T20:56:36.107 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.105+0000 7f55e5800640 1 --2- 172.21.15.28:0/1412883763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f55e01468d0 0x7f55e0143680 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f55d800ef90 tx=0x7f55d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:36.107 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.106+0000 7f55dcc00640 1 -- 172.21.15.28:0/1412883763 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8018020 con 0x7f55e01468d0 2024-04-26T20:56:36.107 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.106+0000 7f55dcc00640 1 -- 172.21.15.28:0/1412883763 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55d8014480 con 0x7f55e01468d0 2024-04-26T20:56:36.107 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.106+0000 7f55dcc00640 1 -- 172.21.15.28:0/1412883763 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d8014850 con 0x7f55e01468d0 2024-04-26T20:56:36.107 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.106+0000 7f55e7800640 1 -- 172.21.15.28:0/1412883763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55e01412a0 con 0x7f55e01468d0 2024-04-26T20:56:36.108 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.106+0000 7f55e7800640 1 -- 172.21.15.28:0/1412883763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55e0141ef0 con 0x7f55e01468d0 2024-04-26T20:56:36.108 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.106+0000 7f55dcc00640 1 -- 172.21.15.28:0/1412883763 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f55d8005d10 con 0x7f55e01468d0 2024-04-26T20:56:36.108 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.106+0000 7f55dcc00640 1 -- 172.21.15.28:0/1412883763 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f55d8014d80 con 0x7f55e01468d0 2024-04-26T20:56:36.108 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.107+0000 7f55e7800640 1 -- 172.21.15.28:0/1412883763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55e0072b60 con 0x7f55e01468d0 2024-04-26T20:56:36.109 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.109+0000 7f55dcc00640 1 -- 172.21.15.28:0/1412883763 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f55e0072b60 con 0x7f55e01468d0 2024-04-26T20:56:36.125 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:36.124+0000 7f68ac612040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T20:56:36.175 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.175+0000 7f55e7800640 1 -- 172.21.15.28:0/1412883763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55e0069ce0 con 0x7f55e01468d0 2024-04-26T20:56:36.176 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.176+0000 7f55dcc00640 1 -- 172.21.15.28:0/1412883763 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f55d8022030 con 0x7f55e01468d0 2024-04-26T20:56:36.176 INFO:teuthology.misc.health.smithi028.stdout: 2024-04-26T20:56:36.176 INFO:teuthology.misc.health.smithi028.stdout:{"epoch":10,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:56:34.747682+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:56:36.177 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.177+0000 7f55e7800640 1 -- 172.21.15.28:0/1412883763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f55e01468d0 msgr2=0x7f55e0143680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:36.177 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.177+0000 7f55e7800640 1 --2- 172.21.15.28:0/1412883763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f55e01468d0 0x7f55e0143680 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f55d800ef90 tx=0x7f55d800c490 comp rx=0 tx=0).stop 2024-04-26T20:56:36.177 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.177+0000 7f55e7800640 1 -- 172.21.15.28:0/1412883763 shutdown_connections 2024-04-26T20:56:36.177 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.177+0000 7f55e7800640 1 --2- 172.21.15.28:0/1412883763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f55e01468d0 0x7f55e0143680 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:36.177 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.177+0000 7f55e7800640 1 --2- 172.21.15.28:0/1412883763 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f55e0072050 0x7f55e00b6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:36.177 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.177+0000 7f55e7800640 1 --2- 172.21.15.28:0/1412883763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f55e006e6a0 0x7f55e00cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:36.177 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.177+0000 7f55e7800640 1 -- 172.21.15.28:0/1412883763 >> 172.21.15.28:0/1412883763 conn(0x7f55e00b2910 msgr2=0x7f55e00737e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:36.178 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.177+0000 7f55e7800640 1 -- 172.21.15.28:0/1412883763 shutdown_connections 2024-04-26T20:56:36.178 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:36.177+0000 7f55e7800640 1 -- 172.21.15.28:0/1412883763 wait complete. 2024-04-26T20:56:36.182 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:36.181+0000 7f68ac612040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T20:56:36.189 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T20:56:36.371 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:36.370+0000 7f68ac612040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T20:56:36.538 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:36.538+0000 7f1338f21040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T20:56:36.544 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:36.544+0000 7f68ac612040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T20:56:36.606 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:36.605+0000 7f68ac612040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T20:56:36.676 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:36.676+0000 7f1338f21040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T20:56:36.692 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:36.692+0000 7f68ac612040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T20:56:36.752 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:36.751+0000 7f68ac612040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T20:56:37.312 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:37.311+0000 7f68ac612040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T20:56:37.448 INFO:tasks.ceph.mgr.x.smithi103.stderr:2024-04-26T20:56:37.448+0000 7f68ac612040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T20:56:37.716 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:37.716+0000 7f12d8c00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T20:56:37.718 INFO:tasks.ceph.mgr.y.smithi028.stderr:2024-04-26T20:56:37.718+0000 7f12d8c00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T20:56:39.040 INFO:tasks.ceph.osd.7.smithi103.stderr:2024-04-26T20:56:39.039+0000 7f2d1c3fc640 -1 osd.7 0 log_to_monitors true 2024-04-26T20:56:39.063 INFO:tasks.ceph.osd.5.smithi103.stderr:2024-04-26T20:56:39.062+0000 7f94e6415640 -1 osd.5 0 log_to_monitors true 2024-04-26T20:56:39.112 INFO:tasks.ceph.osd.4.smithi103.stderr:2024-04-26T20:56:39.111+0000 7f17956db640 -1 osd.4 0 log_to_monitors true 2024-04-26T20:56:39.121 INFO:tasks.ceph.osd.2.smithi028.stderr:2024-04-26T20:56:39.121+0000 7f6c70c9a640 -1 osd.2 0 log_to_monitors true 2024-04-26T20:56:39.131 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T20:56:39.131+0000 7ffb511c0640 -1 osd.0 0 log_to_monitors true 2024-04-26T20:56:39.136 INFO:tasks.ceph.osd.6.smithi103.stderr:2024-04-26T20:56:39.135+0000 7f3353c14640 -1 osd.6 0 log_to_monitors true 2024-04-26T20:56:39.333 INFO:tasks.ceph.osd.1.smithi028.stderr:2024-04-26T20:56:39.333+0000 7f503240d640 -1 osd.1 0 log_to_monitors true 2024-04-26T20:56:39.444 INFO:tasks.ceph.osd.3.smithi028.stderr:2024-04-26T20:56:39.444+0000 7f6497015640 -1 osd.3 0 log_to_monitors true 2024-04-26T20:56:42.384 INFO:tasks.ceph.osd.2.smithi028.stderr:2024-04-26T20:56:42.383+0000 7f6c6bc00640 -1 osd.2 0 waiting for initial osdmap 2024-04-26T20:56:42.391 INFO:tasks.ceph.osd.2.smithi028.stderr:2024-04-26T20:56:42.390+0000 7f6c61c00640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:56:42.430 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T20:56:42.430+0000 7ffb4c000640 -1 osd.0 0 waiting for initial osdmap 2024-04-26T20:56:42.437 INFO:tasks.ceph.osd.1.smithi028.stderr:2024-04-26T20:56:42.436+0000 7f502cc00640 -1 osd.1 0 waiting for initial osdmap 2024-04-26T20:56:42.437 INFO:tasks.ceph.osd.0.smithi028.stderr:2024-04-26T20:56:42.436+0000 7ffb42000640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:56:42.458 INFO:tasks.ceph.osd.1.smithi028.stderr:2024-04-26T20:56:42.457+0000 7f5022c00640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:56:42.459 INFO:tasks.ceph.osd.3.smithi028.stderr:2024-04-26T20:56:42.458+0000 7f6491800640 -1 osd.3 0 waiting for initial osdmap 2024-04-26T20:56:42.480 INFO:tasks.ceph.osd.3.smithi028.stderr:2024-04-26T20:56:42.479+0000 7f6487800640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:56:42.996 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T20:56:43.086 INFO:tasks.ceph.osd.7.smithi103.stderr:2024-04-26T20:56:43.086+0000 7f2d17200640 -1 osd.7 0 waiting for initial osdmap 2024-04-26T20:56:43.093 INFO:tasks.ceph.osd.7.smithi103.stderr:2024-04-26T20:56:43.093+0000 7f2d0d200640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:56:43.104 INFO:tasks.ceph.osd.5.smithi103.stderr:2024-04-26T20:56:43.104+0000 7f94e1600640 -1 osd.5 0 waiting for initial osdmap 2024-04-26T20:56:43.105 INFO:tasks.ceph.osd.6.smithi103.stderr:2024-04-26T20:56:43.105+0000 7f334e400640 -1 osd.6 0 waiting for initial osdmap 2024-04-26T20:56:43.110 INFO:tasks.ceph.osd.5.smithi103.stderr:2024-04-26T20:56:43.109+0000 7f94d6c00640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:56:43.111 INFO:tasks.ceph.osd.6.smithi103.stderr:2024-04-26T20:56:43.111+0000 7f3344400640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:56:43.116 INFO:tasks.ceph.osd.4.smithi103.stderr:2024-04-26T20:56:43.116+0000 7f1791000640 -1 osd.4 0 waiting for initial osdmap 2024-04-26T20:56:43.127 INFO:tasks.ceph.osd.4.smithi103.stderr:2024-04-26T20:56:43.126+0000 7f1786600640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T20:56:43.195 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.193+0000 7ff37de00640 1 Processor -- start 2024-04-26T20:56:43.195 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.194+0000 7ff37de00640 1 -- start start 2024-04-26T20:56:43.196 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.195+0000 7ff37de00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3780d3440 0x7ff3780cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:43.196 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.195+0000 7ff37de00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff3780b4850 0x7ff3780b6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:43.196 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.195+0000 7ff37de00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3780b59f0 con 0x7ff3780b5cd0 2024-04-26T20:56:43.196 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.195+0000 7ff37de00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7ff3780b6830 con 0x7ff3780b4850 2024-04-26T20:56:43.197 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.195+0000 7ff37de00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff3780cff40 con 0x7ff3780d3440 2024-04-26T20:56:43.197 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.196+0000 7ff37d400640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff3780b4850 0x7ff3780b6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:43.197 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.196+0000 7ff37ca00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3780d3440 0x7ff3780cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:43.197 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.196+0000 7ff37ca00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3780d3440 0x7ff3780cfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.28:39530/0 (socket says 172.21.15.28:39530) 2024-04-26T20:56:43.197 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.196+0000 7ff37ca00640 1 -- 172.21.15.28:0/2468401035 learned_addr learned my addr 172.21.15.28:0/2468401035 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:43.197 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.196+0000 7ff37d400640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff3780b4850 0x7ff3780b6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.28:50004/0 (socket says 172.21.15.28:50004) 2024-04-26T20:56:43.198 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.196+0000 7ff37ca00640 1 -- 172.21.15.28:0/2468401035 >> v2:172.21.15.103:3300/0 conn(0x7ff3780b4850 msgr2=0x7ff3780b6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:43.198 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.196+0000 7ff37ca00640 1 --2- 172.21.15.28:0/2468401035 >> v2:172.21.15.103:3300/0 conn(0x7ff3780b4850 0x7ff3780b6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:43.198 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.197+0000 7ff37ca00640 1 -- 172.21.15.28:0/2468401035 >> v1:172.21.15.103:6789/0 conn(0x7ff3780b5cd0 legacy=0x7ff3780d3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:43.198 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.197+0000 7ff37ca00640 1 -- 172.21.15.28:0/2468401035 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3780b5150 con 0x7ff3780d3440 2024-04-26T20:56:43.199 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.198+0000 7ff37ca00640 1 --2- 172.21.15.28:0/2468401035 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3780d3440 0x7ff3780cfa00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff368009ea0 tx=0x7ff36802a7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3553f8abe2869fb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:43.199 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.198+0000 7ff377400640 1 -- 172.21.15.28:0/2468401035 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368036070 con 0x7ff3780d3440 2024-04-26T20:56:43.199 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.198+0000 7ff377400640 1 -- 172.21.15.28:0/2468401035 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff368031600 con 0x7ff3780d3440 2024-04-26T20:56:43.199 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.198+0000 7ff377400640 1 -- 172.21.15.28:0/2468401035 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3680319d0 con 0x7ff3780d3440 2024-04-26T20:56:43.200 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.199+0000 7ff37de00640 1 -- 172.21.15.28:0/2468401035 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3780d3440 msgr2=0x7ff3780cfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:43.200 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.199+0000 7ff37de00640 1 --2- 172.21.15.28:0/2468401035 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3780d3440 0x7ff3780cfa00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff368009ea0 tx=0x7ff36802a7b0 comp rx=0 tx=0).stop 2024-04-26T20:56:43.200 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.199+0000 7ff37de00640 1 -- 172.21.15.28:0/2468401035 shutdown_connections 2024-04-26T20:56:43.200 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.199+0000 7ff37de00640 1 --2- 172.21.15.28:0/2468401035 >> v2:172.21.15.103:3300/0 conn(0x7ff3780b4850 0x7ff3780b6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:43.200 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.199+0000 7ff37de00640 1 --2- 172.21.15.28:0/2468401035 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3780d3440 0x7ff3780cfa00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:43.200 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.199+0000 7ff37de00640 1 -- 172.21.15.28:0/2468401035 >> 172.21.15.28:0/2468401035 conn(0x7ff3780d4ef0 msgr2=0x7ff3780d5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:43.200 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.199+0000 7ff37de00640 1 -- 172.21.15.28:0/2468401035 shutdown_connections 2024-04-26T20:56:43.200 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.199+0000 7ff37de00640 1 -- 172.21.15.28:0/2468401035 wait complete. 2024-04-26T20:56:43.201 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.200+0000 7ff37de00640 1 Processor -- start 2024-04-26T20:56:43.201 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.201+0000 7ff37de00640 1 -- start start 2024-04-26T20:56:43.204 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.201+0000 7ff37de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3780b4850 0x7ff3780c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:43.204 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.201+0000 7ff37de00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3780b5cd0 0x7ff378156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:43.204 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.201+0000 7ff37de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3780d3440 0x7ff3780cad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:43.204 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.201+0000 7ff37de00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff3780d0cb0 con 0x7ff3780b4850 2024-04-26T20:56:43.204 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.201+0000 7ff37de00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff3780b5710 con 0x7ff3780d3440 2024-04-26T20:56:43.205 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.201+0000 7ff37de00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff3780d0e30 con 0x7ff3780b5cd0 2024-04-26T20:56:43.205 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.202+0000 7ff37ca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3780b4850 0x7ff3780c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:43.205 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.202+0000 7ff37ca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3780b4850 0x7ff3780c95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51616/0 (socket says 172.21.15.28:51616) 2024-04-26T20:56:43.205 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.202+0000 7ff37ca00640 1 -- 172.21.15.28:0/2342716810 learned_addr learned my addr 172.21.15.28:0/2342716810 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:43.205 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.202+0000 7ff37d400640 1 --2- 172.21.15.28:0/2342716810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3780d3440 0x7ff3780cad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:43.205 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.202+0000 7ff377e00640 1 --2- 172.21.15.28:0/2342716810 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3780b5cd0 0x7ff378156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:43.205 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.202+0000 7ff37ca00640 1 -- 172.21.15.28:0/2342716810 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3780b5cd0 msgr2=0x7ff378156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:43.205 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.202+0000 7ff37ca00640 1 --2- 172.21.15.28:0/2342716810 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3780b5cd0 0x7ff378156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:43.205 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.202+0000 7ff37ca00640 1 -- 172.21.15.28:0/2342716810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3780d3440 msgr2=0x7ff3780cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:43.205 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.202+0000 7ff37ca00640 1 --2- 172.21.15.28:0/2342716810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3780d3440 0x7ff3780cad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:43.205 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.202+0000 7ff37ca00640 1 -- 172.21.15.28:0/2342716810 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff378157270 con 0x7ff3780b4850 2024-04-26T20:56:43.205 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.202+0000 7ff37d400640 1 --2- 172.21.15.28:0/2342716810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3780d3440 0x7ff3780cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:56:43.205 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.202+0000 7ff377e00640 1 --2- 172.21.15.28:0/2342716810 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3780b5cd0 0x7ff378156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:56:43.205 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.203+0000 7ff37ca00640 1 --2- 172.21.15.28:0/2342716810 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3780b4850 0x7ff3780c95d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff368000f80 tx=0x7ff368004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:43.205 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.203+0000 7ff374c00640 1 -- 172.21.15.28:0/2342716810 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368036050 con 0x7ff3780b4850 2024-04-26T20:56:43.205 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.203+0000 7ff374c00640 1 -- 172.21.15.28:0/2342716810 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff368042070 con 0x7ff3780b4850 2024-04-26T20:56:43.206 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.203+0000 7ff374c00640 1 -- 172.21.15.28:0/2342716810 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368009510 con 0x7ff3780b4850 2024-04-26T20:56:43.206 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.203+0000 7ff37de00640 1 -- 172.21.15.28:0/2342716810 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3781f5610 con 0x7ff3780b4850 2024-04-26T20:56:43.206 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.203+0000 7ff37de00640 1 -- 172.21.15.28:0/2342716810 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3780c9e90 con 0x7ff3780b4850 2024-04-26T20:56:43.206 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.204+0000 7ff37de00640 1 -- 172.21.15.28:0/2342716810 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff378069ce0 con 0x7ff3780b4850 2024-04-26T20:56:43.206 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.205+0000 7ff374c00640 1 -- 172.21.15.28:0/2342716810 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7ff368007c70 con 0x7ff3780b4850 2024-04-26T20:56:43.207 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.206+0000 7ff374c00640 1 --2- 172.21.15.28:0/2342716810 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff35406c860 0x7ff35406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:43.207 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.206+0000 7ff374c00640 1 -- 172.21.15.28:0/2342716810 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7ff3680aba70 con 0x7ff3780b4850 2024-04-26T20:56:43.207 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.206+0000 7ff377e00640 1 --2- 172.21.15.28:0/2342716810 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff35406c860 0x7ff35406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:43.208 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.207+0000 7ff377e00640 1 --2- 172.21.15.28:0/2342716810 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff35406c860 0x7ff35406ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff37815b650 tx=0x7ff360009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:43.213 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.212+0000 7ff374c00640 1 -- 172.21.15.28:0/2342716810 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff368045360 con 0x7ff3780b4850 2024-04-26T20:56:43.419 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.418+0000 7ff37de00640 1 -- 172.21.15.28:0/2342716810 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3780b4630 con 0x7ff3780b4850 2024-04-26T20:56:43.420 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.419+0000 7ff374c00640 1 -- 172.21.15.28:0/2342716810 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10885 (secure 0 0 0) 0x7ff36807ad00 con 0x7ff3780b4850 2024-04-26T20:56:43.421 INFO:teuthology.misc.health.smithi028.stdout: 2024-04-26T20:56:43.421 INFO:teuthology.misc.health.smithi028.stdout:{"epoch":13,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:56:43.392886+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:56:43.425 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.423+0000 7ff37de00640 1 -- 172.21.15.28:0/2342716810 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff35406c860 msgr2=0x7ff35406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:43.425 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.423+0000 7ff37de00640 1 --2- 172.21.15.28:0/2342716810 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff35406c860 0x7ff35406ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff37815b650 tx=0x7ff360009040 comp rx=0 tx=0).stop 2024-04-26T20:56:43.425 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.423+0000 7ff37de00640 1 -- 172.21.15.28:0/2342716810 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3780b4850 msgr2=0x7ff3780c95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:43.425 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.423+0000 7ff37de00640 1 --2- 172.21.15.28:0/2342716810 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3780b4850 0x7ff3780c95d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff368000f80 tx=0x7ff368004a40 comp rx=0 tx=0).stop 2024-04-26T20:56:43.425 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.423+0000 7ff37de00640 1 -- 172.21.15.28:0/2342716810 shutdown_connections 2024-04-26T20:56:43.425 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.423+0000 7ff37de00640 1 --2- 172.21.15.28:0/2342716810 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff35406c860 0x7ff35406ed20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:43.425 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.423+0000 7ff37de00640 1 --2- 172.21.15.28:0/2342716810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3780d3440 0x7ff3780cad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:43.425 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.423+0000 7ff37de00640 1 --2- 172.21.15.28:0/2342716810 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff3780b5cd0 0x7ff378156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:43.425 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.424+0000 7ff37de00640 1 --2- 172.21.15.28:0/2342716810 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3780b4850 0x7ff3780c95d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:43.426 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.424+0000 7ff37de00640 1 -- 172.21.15.28:0/2342716810 >> 172.21.15.28:0/2342716810 conn(0x7ff3780d4ef0 msgr2=0x7ff3780d52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:43.426 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.424+0000 7ff37de00640 1 -- 172.21.15.28:0/2342716810 shutdown_connections 2024-04-26T20:56:43.426 INFO:teuthology.misc.health.smithi028.stderr:2024-04-26T20:56:43.424+0000 7ff37de00640 1 -- 172.21.15.28:0/2342716810 wait complete. 2024-04-26T20:56:43.438 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-26T20:56:43.438 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-26T20:56:43.438 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T20:56:43.438 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T20:56:43.443 INFO:tasks.ceph.mds.a:Started 2024-04-26T20:56:43.443 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T20:56:43.443 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T20:56:43.448 INFO:tasks.ceph.mds.c:Started 2024-04-26T20:56:43.448 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T20:56:43.448 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T20:56:43.454 INFO:tasks.ceph.mds.b:Started 2024-04-26T20:56:43.454 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T20:56:43.454 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T20:56:43.461 INFO:tasks.ceph.mds.d:Started 2024-04-26T20:56:43.461 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-26T20:56:43.462 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:56:43.579 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-04-26T20:56:43.579 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-04-26T20:56:43.602 INFO:tasks.ceph.mds.b.smithi103.stdout:starting mds.b at 2024-04-26T20:56:43.603 INFO:tasks.ceph.mds.d.smithi103.stdout:starting mds.d at 2024-04-26T20:56:43.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.629+0000 7faeb1c00640 1 Processor -- start 2024-04-26T20:56:43.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.629+0000 7faeb1c00640 1 -- start start 2024-04-26T20:56:43.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.630+0000 7faeb1c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7faeac0ce1b0 0x7faeac0cfa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:43.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.630+0000 7faeb1c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faeac06c910 con 0x7faeac15c710 2024-04-26T20:56:43.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.630+0000 7faeb1c00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faeac06cbf0 con 0x7faeac06c350 2024-04-26T20:56:43.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.630+0000 7faeb1c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7faeac13f750 con 0x7faeac0ce1b0 2024-04-26T20:56:43.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.630+0000 7faeabe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7faeac15c710 0x7faeac0cdaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50308/0 (socket says 172.21.15.28:50308) 2024-04-26T20:56:43.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.630+0000 7faeabe00640 1 -- 172.21.15.28:0/2938847886 learned_addr learned my addr 172.21.15.28:0/2938847886 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:43.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.631+0000 7faeab400640 1 --2- 172.21.15.28:0/2938847886 >> v2:172.21.15.28:3300/0 conn(0x7faeac0ce1b0 0x7faeac0cfa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:43.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.631+0000 7faeab400640 1 -- 172.21.15.28:0/2938847886 >> v1:172.21.15.103:6789/0 conn(0x7faeac06c350 legacy=0x7faeac15caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:43.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.631+0000 7faeab400640 1 -- 172.21.15.28:0/2938847886 >> v1:172.21.15.28:6789/0 conn(0x7faeac15c710 legacy=0x7faeac0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:43.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.631+0000 7faeab400640 1 -- 172.21.15.28:0/2938847886 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeac06ced0 con 0x7faeac0ce1b0 2024-04-26T20:56:43.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.632+0000 7faeab400640 1 --2- 172.21.15.28:0/2938847886 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faeac0ce1b0 0x7faeac0cfa30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fae980097d0 tx=0x7fae9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8a297e9f2ec83f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:43.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.632+0000 7faeaaa00640 1 -- 172.21.15.28:0/2938847886 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98037070 con 0x7faeac0ce1b0 2024-04-26T20:56:43.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.632+0000 7faeaaa00640 1 -- 172.21.15.28:0/2938847886 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae98032440 con 0x7faeac0ce1b0 2024-04-26T20:56:43.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.632+0000 7faeaaa00640 1 -- 172.21.15.28:0/2938847886 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98032850 con 0x7faeac0ce1b0 2024-04-26T20:56:43.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.633+0000 7faeb1c00640 1 -- 172.21.15.28:0/2938847886 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faeac0ce1b0 msgr2=0x7faeac0cfa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:43.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.633+0000 7faeb1c00640 1 --2- 172.21.15.28:0/2938847886 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faeac0ce1b0 0x7faeac0cfa30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fae980097d0 tx=0x7fae9802a270 comp rx=0 tx=0).stop 2024-04-26T20:56:43.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.633+0000 7faeb1c00640 1 -- 172.21.15.28:0/2938847886 shutdown_connections 2024-04-26T20:56:43.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.633+0000 7faeb1c00640 1 --2- 172.21.15.28:0/2938847886 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faeac0ce1b0 0x7faeac0cfa30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:43.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.633+0000 7faeb1c00640 1 -- 172.21.15.28:0/2938847886 >> 172.21.15.28:0/2938847886 conn(0x7faeac0d1b30 msgr2=0x7faeac0d1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:43.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.633+0000 7faeb1c00640 1 -- 172.21.15.28:0/2938847886 shutdown_connections 2024-04-26T20:56:43.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.633+0000 7faeb1c00640 1 -- 172.21.15.28:0/2938847886 wait complete. 2024-04-26T20:56:43.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.634+0000 7faeb1c00640 1 Processor -- start 2024-04-26T20:56:43.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.635+0000 7faeb1c00640 1 -- start start 2024-04-26T20:56:43.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.635+0000 7faeb1c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faeac06c350 0x7faeac1520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:43.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.635+0000 7faeb1c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faeac0ce1b0 0x7faeac154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:43.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.635+0000 7faeb1c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faeac15c710 0x7faeac133f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:43.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.635+0000 7faeb1c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7faeac140d70 con 0x7faeac0ce1b0 2024-04-26T20:56:43.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.635+0000 7faeb1c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faeac140bf0 con 0x7faeac15c710 2024-04-26T20:56:43.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.635+0000 7faeb1c00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7faeac140ef0 con 0x7faeac06c350 2024-04-26T20:56:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.636+0000 7faeab400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faeac0ce1b0 0x7faeac154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.636+0000 7faeab400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faeac0ce1b0 0x7faeac154400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51652/0 (socket says 172.21.15.28:51652) 2024-04-26T20:56:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.636+0000 7faeab400640 1 -- 172.21.15.28:0/4166273551 learned_addr learned my addr 172.21.15.28:0/4166273551 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.636+0000 7faeabe00640 1 --2- 172.21.15.28:0/4166273551 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faeac06c350 0x7faeac1520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.636+0000 7faeb1200640 1 --2- 172.21.15.28:0/4166273551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faeac15c710 0x7faeac133f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.636+0000 7faeab400640 1 -- 172.21.15.28:0/4166273551 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faeac06c350 msgr2=0x7faeac1520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.636+0000 7faeab400640 1 --2- 172.21.15.28:0/4166273551 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faeac06c350 0x7faeac1520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.636+0000 7faeab400640 1 -- 172.21.15.28:0/4166273551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faeac15c710 msgr2=0x7faeac133f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.636+0000 7faeab400640 1 --2- 172.21.15.28:0/4166273551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faeac15c710 0x7faeac133f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.636+0000 7faeab400640 1 -- 172.21.15.28:0/4166273551 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeac1349e0 con 0x7faeac0ce1b0 2024-04-26T20:56:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.636+0000 7faeabe00640 1 --2- 172.21.15.28:0/4166273551 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faeac06c350 0x7faeac1520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:56:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.636+0000 7faeb1200640 1 --2- 172.21.15.28:0/4166273551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faeac15c710 0x7faeac133f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:56:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.637+0000 7faeab400640 1 --2- 172.21.15.28:0/4166273551 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faeac0ce1b0 0x7faeac154400 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fae980097d0 tx=0x7fae980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.637+0000 7faea3e00640 1 -- 172.21.15.28:0/4166273551 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98041030 con 0x7faeac0ce1b0 2024-04-26T20:56:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.637+0000 7faea3e00640 1 -- 172.21.15.28:0/4166273551 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae98032e20 con 0x7faeac0ce1b0 2024-04-26T20:56:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.637+0000 7faea3e00640 1 -- 172.21.15.28:0/4166273551 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98037040 con 0x7faeac0ce1b0 2024-04-26T20:56:43.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.637+0000 7faeb1c00640 1 -- 172.21.15.28:0/4166273551 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeac153920 con 0x7faeac0ce1b0 2024-04-26T20:56:43.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.637+0000 7faeb1c00640 1 -- 172.21.15.28:0/4166273551 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeac153ee0 con 0x7faeac0ce1b0 2024-04-26T20:56:43.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.638+0000 7faeb1c00640 1 -- 172.21.15.28:0/4166273551 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeac06a360 con 0x7faeac0ce1b0 2024-04-26T20:56:43.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.639+0000 7faea3e00640 1 -- 172.21.15.28:0/4166273551 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fae98044070 con 0x7faeac0ce1b0 2024-04-26T20:56:43.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.640+0000 7faea3e00640 1 --2- 172.21.15.28:0/4166273551 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fae8c06c860 0x7fae8c06ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:43.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.640+0000 7faea3e00640 1 -- 172.21.15.28:0/4166273551 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7fae980abe40 con 0x7faeac0ce1b0 2024-04-26T20:56:43.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.640+0000 7faeabe00640 1 --2- 172.21.15.28:0/4166273551 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fae8c06c860 0x7fae8c06ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:43.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.641+0000 7faeabe00640 1 --2- 172.21.15.28:0/4166273551 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fae8c06c860 0x7fae8c06ed20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fae9c005880 tx=0x7fae9c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:43.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.645+0000 7faea3e00640 1 -- 172.21.15.28:0/4166273551 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae9807a8a0 con 0x7faeac0ce1b0 2024-04-26T20:56:43.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.843+0000 7faeb1c00640 1 -- 172.21.15.28:0/4166273551 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faeac15c2f0 con 0x7faeac0ce1b0 2024-04-26T20:56:43.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.845+0000 7faea3e00640 1 -- 172.21.15.28:0/4166273551 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+10885 (secure 0 0 0) 0x7fae9807a1a0 con 0x7faeac0ce1b0 2024-04-26T20:56:43.846 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:56:43.846 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":13,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:56:43.392886+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:56:43.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.847+0000 7faeb1c00640 1 -- 172.21.15.28:0/4166273551 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fae8c06c860 msgr2=0x7fae8c06ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:43.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.847+0000 7faeb1c00640 1 --2- 172.21.15.28:0/4166273551 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fae8c06c860 0x7fae8c06ed20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fae9c005880 tx=0x7fae9c009040 comp rx=0 tx=0).stop 2024-04-26T20:56:43.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.847+0000 7faeb1c00640 1 -- 172.21.15.28:0/4166273551 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faeac0ce1b0 msgr2=0x7faeac154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:43.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.847+0000 7faeb1c00640 1 --2- 172.21.15.28:0/4166273551 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faeac0ce1b0 0x7faeac154400 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fae980097d0 tx=0x7fae980046c0 comp rx=0 tx=0).stop 2024-04-26T20:56:43.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.847+0000 7faeb1c00640 1 -- 172.21.15.28:0/4166273551 shutdown_connections 2024-04-26T20:56:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.847+0000 7faeb1c00640 1 --2- 172.21.15.28:0/4166273551 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fae8c06c860 0x7fae8c06ed20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.847+0000 7faeb1c00640 1 --2- 172.21.15.28:0/4166273551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faeac15c710 0x7faeac133f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.847+0000 7faeb1c00640 1 --2- 172.21.15.28:0/4166273551 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faeac0ce1b0 0x7faeac154400 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.847+0000 7faeb1c00640 1 --2- 172.21.15.28:0/4166273551 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faeac06c350 0x7faeac1520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.847+0000 7faeb1c00640 1 -- 172.21.15.28:0/4166273551 >> 172.21.15.28:0/4166273551 conn(0x7faeac0d1b30 msgr2=0x7faeac0b48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.848+0000 7faeb1c00640 1 -- 172.21.15.28:0/4166273551 shutdown_connections 2024-04-26T20:56:43.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:43.848+0000 7faeb1c00640 1 -- 172.21.15.28:0/4166273551 wait complete. 2024-04-26T20:56:43.865 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2024-04-26T20:56:43.865 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:56:44.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.070+0000 7f93f1000640 1 Processor -- start 2024-04-26T20:56:44.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.071+0000 7f93f1000640 1 -- start start 2024-04-26T20:56:44.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.072+0000 7f93f1000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f93ec0d4700 0x7f93ec0d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:44.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.072+0000 7f93f1000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f93ec0d5370 0x7f93ec0b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:44.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.072+0000 7f93f1000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93ec0d0710 con 0x7f93ec15f1b0 2024-04-26T20:56:44.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.072+0000 7f93f1000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f93ec0cff00 con 0x7f93ec0d4700 2024-04-26T20:56:44.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.072+0000 7f93f1000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f93ec0d4270 con 0x7f93ec0d5370 2024-04-26T20:56:44.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.072+0000 7f93eb400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f93ec15f1b0 0x7f93ec0cf700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50322/0 (socket says 172.21.15.28:50322) 2024-04-26T20:56:44.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.072+0000 7f93eb400640 1 -- 172.21.15.28:0/2230144952 learned_addr learned my addr 172.21.15.28:0/2230144952 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:44.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.073+0000 7f93e3e00640 1 --2- 172.21.15.28:0/2230144952 >> v2:172.21.15.28:3300/0 conn(0x7f93ec0d4700 0x7f93ec0d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:44.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.073+0000 7f93ebe00640 1 --2- 172.21.15.28:0/2230144952 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f93ec0d5370 0x7f93ec0b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:44.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.073+0000 7f93eaa00640 1 -- 172.21.15.28:0/2230144952 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3593479138 0 0) 0x7f93dc001110 con 0x7f93ec15f1b0 2024-04-26T20:56:44.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.073+0000 7f93eaa00640 1 -- 172.21.15.28:0/2230144952 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93d0003520 con 0x7f93ec15f1b0 2024-04-26T20:56:44.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.073+0000 7f93e3e00640 1 -- 172.21.15.28:0/2230144952 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f93ec0d5370 msgr2=0x7f93ec0b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:44.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.073+0000 7f93e3e00640 1 --2- 172.21.15.28:0/2230144952 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f93ec0d5370 0x7f93ec0b80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:44.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.073+0000 7f93e3e00640 1 -- 172.21.15.28:0/2230144952 >> v1:172.21.15.28:6789/0 conn(0x7f93ec15f1b0 legacy=0x7f93ec0cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:44.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.073+0000 7f93e3e00640 1 -- 172.21.15.28:0/2230144952 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ec0d4b40 con 0x7f93ec0d4700 2024-04-26T20:56:44.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.074+0000 7f93e3e00640 1 --2- 172.21.15.28:0/2230144952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec0d4700 0x7f93ec0d4e30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f93d40097a0 tx=0x7f93d402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4506e28c7a29f98 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:44.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.074+0000 7f93eaa00640 1 -- 172.21.15.28:0/2230144952 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4037070 con 0x7f93ec0d4700 2024-04-26T20:56:44.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.074+0000 7f93eaa00640 1 -- 172.21.15.28:0/2230144952 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d4032440 con 0x7f93ec0d4700 2024-04-26T20:56:44.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.074+0000 7f93eaa00640 1 -- 172.21.15.28:0/2230144952 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4032810 con 0x7f93ec0d4700 2024-04-26T20:56:44.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.075+0000 7f93f1000640 1 -- 172.21.15.28:0/2230144952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec0d4700 msgr2=0x7f93ec0d4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:44.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.075+0000 7f93f1000640 1 --2- 172.21.15.28:0/2230144952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec0d4700 0x7f93ec0d4e30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f93d40097a0 tx=0x7f93d402a210 comp rx=0 tx=0).stop 2024-04-26T20:56:44.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.075+0000 7f93f1000640 1 -- 172.21.15.28:0/2230144952 shutdown_connections 2024-04-26T20:56:44.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.075+0000 7f93f1000640 1 --2- 172.21.15.28:0/2230144952 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f93ec0d5370 0x7f93ec0b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:44.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.075+0000 7f93f1000640 1 --2- 172.21.15.28:0/2230144952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec0d4700 0x7f93ec0d4e30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:44.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.075+0000 7f93f1000640 1 -- 172.21.15.28:0/2230144952 >> 172.21.15.28:0/2230144952 conn(0x7f93ec06c770 msgr2=0x7f93ec06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:44.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.075+0000 7f93f1000640 1 -- 172.21.15.28:0/2230144952 shutdown_connections 2024-04-26T20:56:44.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.075+0000 7f93f1000640 1 -- 172.21.15.28:0/2230144952 wait complete. 2024-04-26T20:56:44.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.077+0000 7f93f1000640 1 Processor -- start 2024-04-26T20:56:44.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.077+0000 7f93f1000640 1 -- start start 2024-04-26T20:56:44.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.078+0000 7f93f1000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93ec0d4700 0x7f93ec0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:44.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.078+0000 7f93f1000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec0d5370 0x7f93ec146970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:44.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.078+0000 7f93f1000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f93ec15f1b0 0x7f93ec1615f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:44.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.078+0000 7f93f1000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec0d3f00 con 0x7f93ec0d5370 2024-04-26T20:56:44.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.078+0000 7f93f1000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f93ec0d0430 con 0x7f93ec0d4700 2024-04-26T20:56:44.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.078+0000 7f93f1000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f93ec15f590 con 0x7f93ec15f1b0 2024-04-26T20:56:44.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.078+0000 7f93e3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec0d5370 0x7f93ec146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:44.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.078+0000 7f93e3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec0d5370 0x7f93ec146970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51676/0 (socket says 172.21.15.28:51676) 2024-04-26T20:56:44.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.078+0000 7f93e3e00640 1 -- 172.21.15.28:0/900037031 learned_addr learned my addr 172.21.15.28:0/900037031 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:44.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.079+0000 7f93eb400640 1 --2- 172.21.15.28:0/900037031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93ec0d4700 0x7f93ec0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:44.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.079+0000 7f93ebe00640 1 --2- 172.21.15.28:0/900037031 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f93ec15f1b0 0x7f93ec1615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:44.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.079+0000 7f93e3e00640 1 -- 172.21.15.28:0/900037031 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f93ec15f1b0 msgr2=0x7f93ec1615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:44.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.079+0000 7f93e3e00640 1 --2- 172.21.15.28:0/900037031 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f93ec15f1b0 0x7f93ec1615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:44.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.079+0000 7f93e3e00640 1 -- 172.21.15.28:0/900037031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93ec0d4700 msgr2=0x7f93ec0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:44.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.079+0000 7f93e3e00640 1 --2- 172.21.15.28:0/900037031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93ec0d4700 0x7f93ec0ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:44.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.079+0000 7f93e3e00640 1 -- 172.21.15.28:0/900037031 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93ec0ceda0 con 0x7f93ec0d5370 2024-04-26T20:56:44.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.079+0000 7f93eb400640 1 --2- 172.21.15.28:0/900037031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93ec0d4700 0x7f93ec0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:56:44.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.079+0000 7f93e3e00640 1 --2- 172.21.15.28:0/900037031 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec0d5370 0x7f93ec146970 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f93d40098c0 tx=0x7f93d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:44.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.079+0000 7f93e3400640 1 -- 172.21.15.28:0/900037031 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4041030 con 0x7f93ec0d5370 2024-04-26T20:56:44.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.079+0000 7f93e3400640 1 -- 172.21.15.28:0/900037031 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d403d4e0 con 0x7f93ec0d5370 2024-04-26T20:56:44.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.079+0000 7f93e3400640 1 -- 172.21.15.28:0/900037031 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4037040 con 0x7f93ec0d5370 2024-04-26T20:56:44.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.079+0000 7f93f1000640 1 -- 172.21.15.28:0/900037031 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93ec1620b0 con 0x7f93ec0d5370 2024-04-26T20:56:44.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.080+0000 7f93f1000640 1 -- 172.21.15.28:0/900037031 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93ec146170 con 0x7f93ec0d5370 2024-04-26T20:56:44.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.081+0000 7f93f1000640 1 -- 172.21.15.28:0/900037031 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93ec15dde0 con 0x7f93ec0d5370 2024-04-26T20:56:44.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.081+0000 7f93e3400640 1 -- 172.21.15.28:0/900037031 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f93d4044070 con 0x7f93ec0d5370 2024-04-26T20:56:44.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.083+0000 7f93e3400640 1 --2- 172.21.15.28:0/900037031 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f93c806c860 0x7f93c806ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:44.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.083+0000 7f93e3400640 1 -- 172.21.15.28:0/900037031 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 4471+0+0 (secure 0 0 0) 0x7f93d40abc80 con 0x7f93ec0d5370 2024-04-26T20:56:44.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.083+0000 7f93eb400640 1 --2- 172.21.15.28:0/900037031 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f93c806c860 0x7f93c806ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:44.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.084+0000 7f93eb400640 1 --2- 172.21.15.28:0/900037031 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f93c806c860 0x7f93c806ed20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0012b0 tx=0x7f93dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:44.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.087+0000 7f93e3400640 1 -- 172.21.15.28:0/900037031 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f93ec15dde0 con 0x7f93ec0d5370 2024-04-26T20:56:44.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.324+0000 7f93f1000640 1 -- 172.21.15.28:0/900037031 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93ec06a360 con 0x7f93ec0d5370 2024-04-26T20:56:44.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.325+0000 7f93e3400640 1 -- 172.21.15.28:0/900037031 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1389 (secure 0 0 0) 0x7f93ec0d4b40 con 0x7f93ec0d5370 2024-04-26T20:56:44.327 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:56:44.327 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/259712913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":259712913},{"type":"v1","addr":"172.21.15.28:6836","nonce":259712913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T20:56:44.327 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 2 2024-04-26T20:56:44.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.328+0000 7f93f1000640 1 -- 172.21.15.28:0/900037031 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f93c806c860 msgr2=0x7f93c806ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:44.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.328+0000 7f93f1000640 1 --2- 172.21.15.28:0/900037031 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f93c806c860 0x7f93c806ed20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0012b0 tx=0x7f93dc009040 comp rx=0 tx=0).stop 2024-04-26T20:56:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.328+0000 7f93f1000640 1 -- 172.21.15.28:0/900037031 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec0d5370 msgr2=0x7f93ec146970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.328+0000 7f93f1000640 1 --2- 172.21.15.28:0/900037031 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec0d5370 0x7f93ec146970 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f93d40098c0 tx=0x7f93d40046c0 comp rx=0 tx=0).stop 2024-04-26T20:56:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.328+0000 7f93f1000640 1 -- 172.21.15.28:0/900037031 shutdown_connections 2024-04-26T20:56:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.328+0000 7f93f1000640 1 --2- 172.21.15.28:0/900037031 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f93c806c860 0x7f93c806ed20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.328+0000 7f93f1000640 1 --2- 172.21.15.28:0/900037031 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f93ec15f1b0 0x7f93ec1615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.328+0000 7f93f1000640 1 --2- 172.21.15.28:0/900037031 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93ec0d5370 0x7f93ec146970 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.328+0000 7f93f1000640 1 --2- 172.21.15.28:0/900037031 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93ec0d4700 0x7f93ec0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.328+0000 7f93f1000640 1 -- 172.21.15.28:0/900037031 >> 172.21.15.28:0/900037031 conn(0x7f93ec06c770 msgr2=0x7f93ec0b8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.328+0000 7f93f1000640 1 -- 172.21.15.28:0/900037031 shutdown_connections 2024-04-26T20:56:44.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:44.328+0000 7f93f1000640 1 -- 172.21.15.28:0/900037031 wait complete. 2024-04-26T20:56:45.384 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:56:45.392 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:56:46.347 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:56:46.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.567+0000 7f2735400640 1 Processor -- start 2024-04-26T20:56:46.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.567+0000 7f2735400640 1 -- start start 2024-04-26T20:56:46.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.569+0000 7f2735400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f273009c160 0x7f2730167e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:46.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.569+0000 7f2735400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27300d0020 con 0x7f27300d22c0 2024-04-26T20:56:46.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.569+0000 7f2735400640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f27300d0300 con 0x7f2730157360 2024-04-26T20:56:46.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.569+0000 7f2735400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f27300d05e0 con 0x7f273009c160 2024-04-26T20:56:46.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.569+0000 7f272f400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f27300d22c0 0x7f273009ba50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50332/0 (socket says 172.21.15.28:50332) 2024-04-26T20:56:46.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.569+0000 7f272f400640 1 -- 172.21.15.28:0/72553358 learned_addr learned my addr 172.21.15.28:0/72553358 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:46.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.569+0000 7f2734a00640 1 --2- 172.21.15.28:0/72553358 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f273009c160 0x7f2730167e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:46.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.569+0000 7f272ea00640 1 -- 172.21.15.28:0/72553358 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3390647250 0 0) 0x7f27300d0020 con 0x7f27300d22c0 2024-04-26T20:56:46.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.570+0000 7f272ea00640 1 -- 172.21.15.28:0/72553358 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2714003520 con 0x7f27300d22c0 2024-04-26T20:56:46.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.570+0000 7f272ea00640 1 -- 172.21.15.28:0/72553358 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1248541932 0 0) 0x7f2714003520 con 0x7f27300d22c0 2024-04-26T20:56:46.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.570+0000 7f272ea00640 1 -- 172.21.15.28:0/72553358 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f273009c160 msgr2=0x7f2730167e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:46.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.570+0000 7f272ea00640 1 --2- 172.21.15.28:0/72553358 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f273009c160 0x7f2730167e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:46.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.570+0000 7f272ea00640 1 -- 172.21.15.28:0/72553358 >> v1:172.21.15.103:6789/0 conn(0x7f2730157360 legacy=0x7f2730161360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:46.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.570+0000 7f272ea00640 1 -- 172.21.15.28:0/72553358 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2730160b60 con 0x7f27300d22c0 2024-04-26T20:56:46.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.570+0000 7f272ea00640 1 -- 172.21.15.28:0/72553358 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f2718001670 con 0x7f27300d22c0 2024-04-26T20:56:46.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.570+0000 7f272ea00640 1 -- 172.21.15.28:0/72553358 >> v1:172.21.15.28:6789/0 conn(0x7f27300d22c0 legacy=0x7f273009ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:46.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.570+0000 7f272ea00640 1 --2- 172.21.15.28:0/72553358 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f273009c160 0x7f27140039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:46.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.570+0000 7f272ea00640 1 --2- 172.21.15.28:0/72553358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2714003f60 0x7f2714004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:46.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.570+0000 7f272ea00640 1 --2- 172.21.15.28:0/72553358 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2714004d40 0x7f27140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:46.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.570+0000 7f272ea00640 1 -- 172.21.15.28:0/72553358 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f27300d05e0 con 0x7f2714004d40 2024-04-26T20:56:46.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.570+0000 7f272ea00640 1 -- 172.21.15.28:0/72553358 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f271400baa0 con 0x7f2714003f60 2024-04-26T20:56:46.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.570+0000 7f272ea00640 1 -- 172.21.15.28:0/72553358 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f271400bca0 con 0x7f273009c160 2024-04-26T20:56:46.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.570+0000 7f2734a00640 1 --2- 172.21.15.28:0/72553358 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2714004d40 0x7f27140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:46.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.571+0000 7f272f400640 1 --2- 172.21.15.28:0/72553358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2714003f60 0x7f2714004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:46.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.571+0000 7f272fe00640 1 --2- 172.21.15.28:0/72553358 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f273009c160 0x7f27140039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:46.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.571+0000 7f2734a00640 1 -- 172.21.15.28:0/72553358 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f273009c160 msgr2=0x7f27140039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:46.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.571+0000 7f2734a00640 1 --2- 172.21.15.28:0/72553358 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f273009c160 0x7f27140039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:46.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.571+0000 7f2734a00640 1 -- 172.21.15.28:0/72553358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2714003f60 msgr2=0x7f2714004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:46.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.571+0000 7f2734a00640 1 --2- 172.21.15.28:0/72553358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2714003f60 0x7f2714004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:46.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.571+0000 7f2734a00640 1 -- 172.21.15.28:0/72553358 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2714003520 con 0x7f2714004d40 2024-04-26T20:56:46.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.571+0000 7f272fe00640 1 --2- 172.21.15.28:0/72553358 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f273009c160 0x7f27140039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:56:46.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.572+0000 7f2734a00640 1 --2- 172.21.15.28:0/72553358 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2714004d40 0x7f27140051c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2728009fc0 tx=0x7f2714005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ab6e4106fdb5f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:46.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.572+0000 7f272ea00640 1 -- 172.21.15.28:0/72553358 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728009c10 con 0x7f2714004d40 2024-04-26T20:56:46.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.572+0000 7f272ea00640 1 -- 172.21.15.28:0/72553358 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2728036070 con 0x7f2714004d40 2024-04-26T20:56:46.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.572+0000 7f272ea00640 1 -- 172.21.15.28:0/72553358 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728031880 con 0x7f2714004d40 2024-04-26T20:56:46.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.573+0000 7f2735400640 1 -- 172.21.15.28:0/72553358 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2714004d40 msgr2=0x7f27140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:46.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.573+0000 7f2735400640 1 --2- 172.21.15.28:0/72553358 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2714004d40 0x7f27140051c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2728009fc0 tx=0x7f2714005830 comp rx=0 tx=0).stop 2024-04-26T20:56:46.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.573+0000 7f2734a00640 1 -- 172.21.15.28:0/72553358 reap_dead start 2024-04-26T20:56:46.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.573+0000 7f2735400640 1 -- 172.21.15.28:0/72553358 shutdown_connections 2024-04-26T20:56:46.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.573+0000 7f2735400640 1 -- 172.21.15.28:0/72553358 >> 172.21.15.28:0/72553358 conn(0x7f27300b8080 msgr2=0x7f27300b8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:46.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.573+0000 7f2735400640 1 -- 172.21.15.28:0/72553358 shutdown_connections 2024-04-26T20:56:46.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.573+0000 7f2735400640 1 -- 172.21.15.28:0/72553358 wait complete. 2024-04-26T20:56:46.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.575+0000 7f2735400640 1 Processor -- start 2024-04-26T20:56:46.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.575+0000 7f2735400640 1 -- start start 2024-04-26T20:56:46.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.576+0000 7f2735400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2714004d40 0x7f27301f4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:46.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.576+0000 7f2735400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f273006d200 0x7f273006c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:46.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.576+0000 7f2735400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f273006c890 0x7f273015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:46.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.576+0000 7f2735400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f273015f0e0 con 0x7f273006c890 2024-04-26T20:56:46.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.576+0000 7f2735400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f273015ef60 con 0x7f273006d200 2024-04-26T20:56:46.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.576+0000 7f2735400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f2730157740 con 0x7f2714004d40 2024-04-26T20:56:46.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.577+0000 7f2734a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f273006c890 0x7f273015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:46.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.577+0000 7f2734a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f273006c890 0x7f273015b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51690/0 (socket says 172.21.15.28:51690) 2024-04-26T20:56:46.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.577+0000 7f2734a00640 1 -- 172.21.15.28:0/1438588518 learned_addr learned my addr 172.21.15.28:0/1438588518 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:46.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.577+0000 7f272f400640 1 --2- 172.21.15.28:0/1438588518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f273006d200 0x7f273006c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:46.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.577+0000 7f272fe00640 1 --2- 172.21.15.28:0/1438588518 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2714004d40 0x7f27301f4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:46.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.577+0000 7f2734a00640 1 -- 172.21.15.28:0/1438588518 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2714004d40 msgr2=0x7f27301f4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:46.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.577+0000 7f2734a00640 1 --2- 172.21.15.28:0/1438588518 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2714004d40 0x7f27301f4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:46.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.577+0000 7f2734a00640 1 -- 172.21.15.28:0/1438588518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f273006d200 msgr2=0x7f273006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:46.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.577+0000 7f2734a00640 1 --2- 172.21.15.28:0/1438588518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f273006d200 0x7f273006c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:46.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.577+0000 7f2734a00640 1 -- 172.21.15.28:0/1438588518 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27301f5520 con 0x7f273006c890 2024-04-26T20:56:46.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.577+0000 7f2734a00640 1 --2- 172.21.15.28:0/1438588518 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f273006c890 0x7f273015b7d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f272800e230 tx=0x7f2728009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:46.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.578+0000 7f2727e00640 1 -- 172.21.15.28:0/1438588518 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728009c10 con 0x7f273006c890 2024-04-26T20:56:46.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.578+0000 7f2727e00640 1 -- 172.21.15.28:0/1438588518 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2728031d40 con 0x7f273006c890 2024-04-26T20:56:46.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.578+0000 7f2727e00640 1 -- 172.21.15.28:0/1438588518 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728045670 con 0x7f273006c890 2024-04-26T20:56:46.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.578+0000 7f2735400640 1 -- 172.21.15.28:0/1438588518 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27301f5800 con 0x7f273006c890 2024-04-26T20:56:46.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.578+0000 7f2735400640 1 -- 172.21.15.28:0/1438588518 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27301f5dc0 con 0x7f273006c890 2024-04-26T20:56:46.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.579+0000 7f2735400640 1 -- 172.21.15.28:0/1438588518 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f273006a360 con 0x7f273006c890 2024-04-26T20:56:46.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.580+0000 7f2727e00640 1 -- 172.21.15.28:0/1438588518 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f2728005d10 con 0x7f273006c890 2024-04-26T20:56:46.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.581+0000 7f2727e00640 1 --2- 172.21.15.28:0/1438588518 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f270c06c8e0 0x7f270c06eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:46.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.581+0000 7f2727e00640 1 -- 172.21.15.28:0/1438588518 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4846+0+0 (secure 0 0 0) 0x7f27280ac790 con 0x7f273006c890 2024-04-26T20:56:46.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.581+0000 7f272fe00640 1 --2- 172.21.15.28:0/1438588518 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f270c06c8e0 0x7f270c06eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:46.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.582+0000 7f272fe00640 1 --2- 172.21.15.28:0/1438588518 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f270c06c8e0 0x7f270c06eda0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2720001750 tx=0x7f2720009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:46.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.587+0000 7f2727e00640 1 -- 172.21.15.28:0/1438588518 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f272804b060 con 0x7f273006c890 2024-04-26T20:56:46.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.836+0000 7f2735400640 1 -- 172.21.15.28:0/1438588518 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27300cf880 con 0x7f273006c890 2024-04-26T20:56:46.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.837+0000 7f2727e00640 1 -- 172.21.15.28:0/1438588518 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3864 (secure 0 0 0) 0x7f272807b0a0 con 0x7f273006c890 2024-04-26T20:56:46.839 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:56:46.839 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/2818824721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2818824721},{"type":"v1","addr":"172.21.15.103:6834","nonce":2818824721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/259712913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":259712913},{"type":"v1","addr":"172.21.15.28:6836","nonce":259712913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/989575415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":989575415},{"type":"v1","addr":"172.21.15.28:6837","nonce":989575415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2361950169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":2361950169},{"type":"v1","addr":"172.21.15.103:6835","nonce":2361950169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T20:56:46.839 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 3 2024-04-26T20:56:46.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.840+0000 7f2735400640 1 -- 172.21.15.28:0/1438588518 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f270c06c8e0 msgr2=0x7f270c06eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:46.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.840+0000 7f2735400640 1 --2- 172.21.15.28:0/1438588518 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f270c06c8e0 0x7f270c06eda0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2720001750 tx=0x7f2720009040 comp rx=0 tx=0).stop 2024-04-26T20:56:46.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.840+0000 7f2735400640 1 -- 172.21.15.28:0/1438588518 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f273006c890 msgr2=0x7f273015b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:46.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.840+0000 7f2735400640 1 --2- 172.21.15.28:0/1438588518 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f273006c890 0x7f273015b7d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f272800e230 tx=0x7f2728009750 comp rx=0 tx=0).stop 2024-04-26T20:56:46.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.841+0000 7f2735400640 1 -- 172.21.15.28:0/1438588518 shutdown_connections 2024-04-26T20:56:46.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.841+0000 7f2735400640 1 --2- 172.21.15.28:0/1438588518 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f270c06c8e0 0x7f270c06eda0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:46.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.841+0000 7f2735400640 1 --2- 172.21.15.28:0/1438588518 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f273006c890 0x7f273015b7d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:46.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.841+0000 7f2735400640 1 --2- 172.21.15.28:0/1438588518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f273006d200 0x7f273006c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:46.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.841+0000 7f2735400640 1 --2- 172.21.15.28:0/1438588518 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2714004d40 0x7f27301f4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:46.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.841+0000 7f2735400640 1 -- 172.21.15.28:0/1438588518 >> 172.21.15.28:0/1438588518 conn(0x7f27300b8080 msgr2=0x7f27301567e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:46.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.841+0000 7f2735400640 1 -- 172.21.15.28:0/1438588518 shutdown_connections 2024-04-26T20:56:46.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:46.841+0000 7f2735400640 1 -- 172.21.15.28:0/1438588518 wait complete. 2024-04-26T20:56:46.864 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:56:47.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.072+0000 7f3dccc00640 1 Processor -- start 2024-04-26T20:56:47.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.072+0000 7f3dccc00640 1 -- start start 2024-04-26T20:56:47.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.074+0000 7f3dccc00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3dc80b43c0 0x7f3dc80d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:47.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.074+0000 7f3dccc00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3dc81607e0 0x7f3dc8163cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:47.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.074+0000 7f3dccc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3dc80b6b10 0x7f3dc80b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:47.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.074+0000 7f3dccc00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f3dc80b74c0 con 0x7f3dc80b6b10 2024-04-26T20:56:47.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.074+0000 7f3dccc00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f3dc80b47a0 con 0x7f3dc81607e0 2024-04-26T20:56:47.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.074+0000 7f3dccc00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f3dc80d0740 con 0x7f3dc80b43c0 2024-04-26T20:56:47.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.074+0000 7f3dc6800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3dc80b6b10 0x7f3dc80b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:47.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.074+0000 7f3dc6800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3dc80b6b10 0x7f3dc80b6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51722/0 (socket says 172.21.15.28:51722) 2024-04-26T20:56:47.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.074+0000 7f3dc6800640 1 -- 172.21.15.28:0/2793756040 learned_addr learned my addr 172.21.15.28:0/2793756040 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:47.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.075+0000 7f3dc5400640 1 --2- 172.21.15.28:0/2793756040 >> v2:172.21.15.103:3300/0 conn(0x7f3dc81607e0 0x7f3dc8163cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:47.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.075+0000 7f3dc5e00640 1 --2- 172.21.15.28:0/2793756040 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3dc80b43c0 0x7f3dc80d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:47.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.075+0000 7f3dc6800640 1 -- 172.21.15.28:0/2793756040 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3dc80b43c0 msgr2=0x7f3dc80d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:47.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.075+0000 7f3dc6800640 1 --2- 172.21.15.28:0/2793756040 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3dc80b43c0 0x7f3dc80d00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:47.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.075+0000 7f3dc6800640 1 -- 172.21.15.28:0/2793756040 >> v2:172.21.15.103:3300/0 conn(0x7f3dc81607e0 msgr2=0x7f3dc8163cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:47.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.075+0000 7f3dc6800640 1 --2- 172.21.15.28:0/2793756040 >> v2:172.21.15.103:3300/0 conn(0x7f3dc81607e0 0x7f3dc8163cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:47.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.075+0000 7f3dc6800640 1 -- 172.21.15.28:0/2793756040 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc80d0a20 con 0x7f3dc80b6b10 2024-04-26T20:56:47.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.075+0000 7f3dc5400640 1 --2- 172.21.15.28:0/2793756040 >> v2:172.21.15.103:3300/0 conn(0x7f3dc81607e0 0x7f3dc8163cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:56:47.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.076+0000 7f3dc6800640 1 --2- 172.21.15.28:0/2793756040 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dc80b6b10 0x7f3dc80b6f80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3dc80d6190 tx=0x7f3dc002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec0a1cf767f6975 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:47.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.076+0000 7f3dc4a00640 1 -- 172.21.15.28:0/2793756040 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0036070 con 0x7f3dc80b6b10 2024-04-26T20:56:47.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.076+0000 7f3dc4a00640 1 -- 172.21.15.28:0/2793756040 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc0031600 con 0x7f3dc80b6b10 2024-04-26T20:56:47.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.076+0000 7f3dc4a00640 1 -- 172.21.15.28:0/2793756040 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc00319d0 con 0x7f3dc80b6b10 2024-04-26T20:56:47.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.077+0000 7f3dccc00640 1 -- 172.21.15.28:0/2793756040 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dc80b6b10 msgr2=0x7f3dc80b6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:47.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.077+0000 7f3dccc00640 1 --2- 172.21.15.28:0/2793756040 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dc80b6b10 0x7f3dc80b6f80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3dc80d6190 tx=0x7f3dc002a610 comp rx=0 tx=0).stop 2024-04-26T20:56:47.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.077+0000 7f3dccc00640 1 -- 172.21.15.28:0/2793756040 shutdown_connections 2024-04-26T20:56:47.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.077+0000 7f3dccc00640 1 --2- 172.21.15.28:0/2793756040 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dc80b6b10 0x7f3dc80b6f80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:47.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.077+0000 7f3dccc00640 1 --2- 172.21.15.28:0/2793756040 >> v2:172.21.15.103:3300/0 conn(0x7f3dc81607e0 0x7f3dc8163cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:47.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.077+0000 7f3dccc00640 1 --2- 172.21.15.28:0/2793756040 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3dc80b43c0 0x7f3dc80d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:47.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.077+0000 7f3dccc00640 1 -- 172.21.15.28:0/2793756040 >> 172.21.15.28:0/2793756040 conn(0x7f3dc80d1b10 msgr2=0x7f3dc80d1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:47.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.077+0000 7f3dccc00640 1 -- 172.21.15.28:0/2793756040 shutdown_connections 2024-04-26T20:56:47.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.077+0000 7f3dccc00640 1 -- 172.21.15.28:0/2793756040 wait complete. 2024-04-26T20:56:47.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.079+0000 7f3dccc00640 1 Processor -- start 2024-04-26T20:56:47.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.079+0000 7f3dccc00640 1 -- start start 2024-04-26T20:56:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.080+0000 7f3dccc00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3dc80b43c0 0x7f3dc8158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.080+0000 7f3dccc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3dc80b6b10 0x7f3dc80cea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.080+0000 7f3dccc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dc81607e0 0x7f3dc815cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.080+0000 7f3dccc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc81641f0 con 0x7f3dc81607e0 2024-04-26T20:56:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.080+0000 7f3dccc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc8163790 con 0x7f3dc80b6b10 2024-04-26T20:56:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.080+0000 7f3dccc00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f3dc8160c60 con 0x7f3dc80b43c0 2024-04-26T20:56:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.081+0000 7f3dc6800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dc81607e0 0x7f3dc815cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.081+0000 7f3dc6800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dc81607e0 0x7f3dc815cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51738/0 (socket says 172.21.15.28:51738) 2024-04-26T20:56:47.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.081+0000 7f3dc6800640 1 -- 172.21.15.28:0/1393362518 learned_addr learned my addr 172.21.15.28:0/1393362518 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:47.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.081+0000 7f3dc5400640 1 --2- 172.21.15.28:0/1393362518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3dc80b6b10 0x7f3dc80cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:47.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.081+0000 7f3dc5e00640 1 --2- 172.21.15.28:0/1393362518 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3dc80b43c0 0x7f3dc8158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:47.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.081+0000 7f3dc6800640 1 -- 172.21.15.28:0/1393362518 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3dc80b43c0 msgr2=0x7f3dc8158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:47.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.081+0000 7f3dc6800640 1 --2- 172.21.15.28:0/1393362518 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3dc80b43c0 0x7f3dc8158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:47.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.081+0000 7f3dc6800640 1 -- 172.21.15.28:0/1393362518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3dc80b6b10 msgr2=0x7f3dc80cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:47.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.081+0000 7f3dc6800640 1 --2- 172.21.15.28:0/1393362518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3dc80b6b10 0x7f3dc80cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:47.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.081+0000 7f3dc6800640 1 -- 172.21.15.28:0/1393362518 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc80cc740 con 0x7f3dc81607e0 2024-04-26T20:56:47.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.081+0000 7f3dc5400640 1 --2- 172.21.15.28:0/1393362518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3dc80b6b10 0x7f3dc80cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:56:47.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.082+0000 7f3dc6800640 1 --2- 172.21.15.28:0/1393362518 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dc81607e0 0x7f3dc815cfe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009f40 tx=0x7f3dc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:47.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.083+0000 7f3dbe000640 1 -- 172.21.15.28:0/1393362518 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0036050 con 0x7f3dc81607e0 2024-04-26T20:56:47.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.083+0000 7f3dbe000640 1 -- 172.21.15.28:0/1393362518 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc0042070 con 0x7f3dc81607e0 2024-04-26T20:56:47.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.083+0000 7f3dccc00640 1 -- 172.21.15.28:0/1393362518 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dc80cc9e0 con 0x7f3dc81607e0 2024-04-26T20:56:47.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.083+0000 7f3dbe000640 1 -- 172.21.15.28:0/1393362518 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0009510 con 0x7f3dc81607e0 2024-04-26T20:56:47.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.083+0000 7f3dccc00640 1 -- 172.21.15.28:0/1393362518 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dc80ccfa0 con 0x7f3dc81607e0 2024-04-26T20:56:47.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.085+0000 7f3dccc00640 1 -- 172.21.15.28:0/1393362518 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dc806a400 con 0x7f3dc81607e0 2024-04-26T20:56:47.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.086+0000 7f3dbe000640 1 -- 172.21.15.28:0/1393362518 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f3dc0007bd0 con 0x7f3dc81607e0 2024-04-26T20:56:47.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.087+0000 7f3dbe000640 1 --2- 172.21.15.28:0/1393362518 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3da806c8b0 0x7f3da806ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:47.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.087+0000 7f3dbe000640 1 -- 172.21.15.28:0/1393362518 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f3dc00ac590 con 0x7f3dc81607e0 2024-04-26T20:56:47.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.088+0000 7f3dc5e00640 1 --2- 172.21.15.28:0/1393362518 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3da806c8b0 0x7f3da806ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:47.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.088+0000 7f3dc5e00640 1 --2- 172.21.15.28:0/1393362518 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3da806c8b0 0x7f3da806ed70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3db8006fd0 tx=0x7f3db8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:47.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.094+0000 7f3dbe000640 1 -- 172.21.15.28:0/1393362518 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3dc007ae60 con 0x7f3dc81607e0 2024-04-26T20:56:47.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.310+0000 7f3dccc00640 1 -- 172.21.15.28:0/1393362518 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3dc80b6f80 con 0x7f3dc81607e0 2024-04-26T20:56:47.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.311+0000 7f3dbe000640 1 -- 172.21.15.28:0/1393362518 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12873 (secure 0 0 0) 0x7f3dc007a760 con 0x7f3dc81607e0 2024-04-26T20:56:47.313 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:56:47.313 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":16,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:56:46.603193+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:56:47.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.314+0000 7f3dccc00640 1 -- 172.21.15.28:0/1393362518 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3da806c8b0 msgr2=0x7f3da806ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:47.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.314+0000 7f3dccc00640 1 --2- 172.21.15.28:0/1393362518 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3da806c8b0 0x7f3da806ed70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3db8006fd0 tx=0x7f3db8008040 comp rx=0 tx=0).stop 2024-04-26T20:56:47.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.314+0000 7f3dccc00640 1 -- 172.21.15.28:0/1393362518 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dc81607e0 msgr2=0x7f3dc815cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:47.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.314+0000 7f3dccc00640 1 --2- 172.21.15.28:0/1393362518 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dc81607e0 0x7f3dc815cfe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0009f40 tx=0x7f3dc0004a40 comp rx=0 tx=0).stop 2024-04-26T20:56:47.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.314+0000 7f3dccc00640 1 -- 172.21.15.28:0/1393362518 shutdown_connections 2024-04-26T20:56:47.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.314+0000 7f3dccc00640 1 --2- 172.21.15.28:0/1393362518 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3da806c8b0 0x7f3da806ed70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:47.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.314+0000 7f3dccc00640 1 --2- 172.21.15.28:0/1393362518 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3dc81607e0 0x7f3dc815cfe0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:47.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.314+0000 7f3dccc00640 1 --2- 172.21.15.28:0/1393362518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3dc80b6b10 0x7f3dc80cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:47.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.314+0000 7f3dccc00640 1 --2- 172.21.15.28:0/1393362518 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3dc80b43c0 0x7f3dc8158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:47.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.314+0000 7f3dccc00640 1 -- 172.21.15.28:0/1393362518 >> 172.21.15.28:0/1393362518 conn(0x7f3dc80d1b10 msgr2=0x7f3dc80d1ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:47.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.314+0000 7f3dccc00640 1 -- 172.21.15.28:0/1393362518 shutdown_connections 2024-04-26T20:56:47.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.314+0000 7f3dccc00640 1 -- 172.21.15.28:0/1393362518 wait complete. 2024-04-26T20:56:47.333 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:56:47.333 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:56:47.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.516+0000 7f6ef6400640 1 Processor -- start 2024-04-26T20:56:47.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.516+0000 7f6ef6400640 1 -- start start 2024-04-26T20:56:47.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.518+0000 7f6ef6400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6ef00b9e70 0x7f6ef00d3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:47.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.518+0000 7f6ef6400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ef00d4110 con 0x7f6ef00d2160 2024-04-26T20:56:47.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.518+0000 7f6ef6400640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ef00d43f0 con 0x7f6ef015f680 2024-04-26T20:56:47.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.518+0000 7f6ef6400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f6ef00d46d0 con 0x7f6ef00b9e70 2024-04-26T20:56:47.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.518+0000 7f6eefe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6ef00b9e70 0x7f6ef00d3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:47.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.518+0000 7f6ee7e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6ef00d2160 0x7f6ef00d33e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50338/0 (socket says 172.21.15.28:50338) 2024-04-26T20:56:47.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.518+0000 7f6ee7e00640 1 -- 172.21.15.28:0/1610450582 learned_addr learned my addr 172.21.15.28:0/1610450582 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:47.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.519+0000 7f6eeea00640 1 -- 172.21.15.28:0/1610450582 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 685477618 0 0) 0x7f6ef00d4110 con 0x7f6ef00d2160 2024-04-26T20:56:47.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.519+0000 7f6eeea00640 1 -- 172.21.15.28:0/1610450582 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ed4003520 con 0x7f6ef00d2160 2024-04-26T20:56:47.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.519+0000 7f6eefe00640 1 -- 172.21.15.28:0/1610450582 >> v1:172.21.15.103:6789/0 conn(0x7f6ef015f680 legacy=0x7f6ef00d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:47.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.519+0000 7f6eefe00640 1 -- 172.21.15.28:0/1610450582 >> v1:172.21.15.28:6789/0 conn(0x7f6ef00d2160 legacy=0x7f6ef00d33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:47.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.519+0000 7f6eefe00640 1 -- 172.21.15.28:0/1610450582 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef00b8660 con 0x7f6ef00b9e70 2024-04-26T20:56:47.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.520+0000 7f6eefe00640 1 --2- 172.21.15.28:0/1610450582 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef00b9e70 0x7f6ef00d3af0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009d90 tx=0x7f6ee802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de05d6692a7257c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:47.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.520+0000 7f6eeea00640 1 -- 172.21.15.28:0/1610450582 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8007740 con 0x7f6ef00b9e70 2024-04-26T20:56:47.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.520+0000 7f6eeea00640 1 -- 172.21.15.28:0/1610450582 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee80078e0 con 0x7f6ef00b9e70 2024-04-26T20:56:47.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.521+0000 7f6eeea00640 1 -- 172.21.15.28:0/1610450582 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8007c90 con 0x7f6ef00b9e70 2024-04-26T20:56:47.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.521+0000 7f6ef6400640 1 -- 172.21.15.28:0/1610450582 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef00b9e70 msgr2=0x7f6ef00d3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:47.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.521+0000 7f6ef6400640 1 --2- 172.21.15.28:0/1610450582 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef00b9e70 0x7f6ef00d3af0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009d90 tx=0x7f6ee802a6f0 comp rx=0 tx=0).stop 2024-04-26T20:56:47.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.522+0000 7f6ef6400640 1 -- 172.21.15.28:0/1610450582 shutdown_connections 2024-04-26T20:56:47.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.522+0000 7f6ef6400640 1 --2- 172.21.15.28:0/1610450582 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef00b9e70 0x7f6ef00d3af0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:47.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.522+0000 7f6ef6400640 1 -- 172.21.15.28:0/1610450582 >> 172.21.15.28:0/1610450582 conn(0x7f6ef006ca10 msgr2=0x7f6ef0072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:47.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.522+0000 7f6ef6400640 1 -- 172.21.15.28:0/1610450582 shutdown_connections 2024-04-26T20:56:47.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.522+0000 7f6ef6400640 1 -- 172.21.15.28:0/1610450582 wait complete. 2024-04-26T20:56:47.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.524+0000 7f6ef6400640 1 Processor -- start 2024-04-26T20:56:47.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.524+0000 7f6ef6400640 1 -- start start 2024-04-26T20:56:47.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.525+0000 7f6ef6400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ef00b9e70 0x7f6ef0158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:47.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.525+0000 7f6ef6400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef00d2160 0x7f6ef015b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:47.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.525+0000 7f6ef6400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ef015f680 0x7f6ef0161a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:47.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.525+0000 7f6ef6400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef016a230 con 0x7f6ef00d2160 2024-04-26T20:56:47.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.525+0000 7f6ef6400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef016a0b0 con 0x7f6ef00b9e70 2024-04-26T20:56:47.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.525+0000 7f6ef6400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f6ef016a3b0 con 0x7f6ef015f680 2024-04-26T20:56:47.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.525+0000 7f6ee7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef00d2160 0x7f6ef015b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:47.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.526+0000 7f6ee7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef00d2160 0x7f6ef015b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51760/0 (socket says 172.21.15.28:51760) 2024-04-26T20:56:47.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.526+0000 7f6ee7e00640 1 -- 172.21.15.28:0/671763534 learned_addr learned my addr 172.21.15.28:0/671763534 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:47.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.526+0000 7f6eef400640 1 --2- 172.21.15.28:0/671763534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ef00b9e70 0x7f6ef0158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:47.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.526+0000 7f6eefe00640 1 --2- 172.21.15.28:0/671763534 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ef015f680 0x7f6ef0161a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:47.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.526+0000 7f6ee7e00640 1 -- 172.21.15.28:0/671763534 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ef015f680 msgr2=0x7f6ef0161a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:47.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.526+0000 7f6ee7e00640 1 --2- 172.21.15.28:0/671763534 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ef015f680 0x7f6ef0161a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:47.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.526+0000 7f6ee7e00640 1 -- 172.21.15.28:0/671763534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ef00b9e70 msgr2=0x7f6ef0158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:47.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.526+0000 7f6ee7e00640 1 --2- 172.21.15.28:0/671763534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ef00b9e70 0x7f6ef0158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:47.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.526+0000 7f6ee7e00640 1 -- 172.21.15.28:0/671763534 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef015a2a0 con 0x7f6ef00d2160 2024-04-26T20:56:47.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.526+0000 7f6ee7e00640 1 --2- 172.21.15.28:0/671763534 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef00d2160 0x7f6ef015b060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8001260 tx=0x7f6ed800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:47.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.527+0000 7f6ee7400640 1 -- 172.21.15.28:0/671763534 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8017070 con 0x7f6ef00d2160 2024-04-26T20:56:47.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.527+0000 7f6ee7400640 1 -- 172.21.15.28:0/671763534 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ed8007480 con 0x7f6ef00d2160 2024-04-26T20:56:47.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.527+0000 7f6ee7400640 1 -- 172.21.15.28:0/671763534 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed80059c0 con 0x7f6ef00d2160 2024-04-26T20:56:47.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.527+0000 7f6ef6400640 1 -- 172.21.15.28:0/671763534 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef015a580 con 0x7f6ef00d2160 2024-04-26T20:56:47.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.527+0000 7f6ef6400640 1 -- 172.21.15.28:0/671763534 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef0158e60 con 0x7f6ef00d2160 2024-04-26T20:56:47.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.528+0000 7f6ef6400640 1 -- 172.21.15.28:0/671763534 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef006a360 con 0x7f6ef00d2160 2024-04-26T20:56:47.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.529+0000 7f6ee7400640 1 -- 172.21.15.28:0/671763534 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f6ed801a070 con 0x7f6ef00d2160 2024-04-26T20:56:47.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.530+0000 7f6ee7400640 1 --2- 172.21.15.28:0/671763534 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6ecc06c860 0x7f6ecc06ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:47.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.530+0000 7f6ee7400640 1 -- 172.21.15.28:0/671763534 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f6ed808ab80 con 0x7f6ef00d2160 2024-04-26T20:56:47.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.531+0000 7f6eef400640 1 --2- 172.21.15.28:0/671763534 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6ecc06c860 0x7f6ecc06ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:47.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.531+0000 7f6eef400640 1 --2- 172.21.15.28:0/671763534 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6ecc06c860 0x7f6ecc06ed20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0000f60 tx=0x7f6ee0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:47.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.536+0000 7f6ee7400640 1 -- 172.21.15.28:0/671763534 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6ed80593d0 con 0x7f6ef00d2160 2024-04-26T20:56:47.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.759+0000 7f6ef6400640 1 -- 172.21.15.28:0/671763534 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6ef00ba960 con 0x7f6ef00d2160 2024-04-26T20:56:47.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.760+0000 7f6ee7400640 1 -- 172.21.15.28:0/671763534 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f6ed8058cd0 con 0x7f6ef00d2160 2024-04-26T20:56:47.761 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:56:47.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.763+0000 7f6ef6400640 1 -- 172.21.15.28:0/671763534 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6ecc06c860 msgr2=0x7f6ecc06ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:47.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.763+0000 7f6ef6400640 1 --2- 172.21.15.28:0/671763534 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6ecc06c860 0x7f6ecc06ed20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0000f60 tx=0x7f6ee0009040 comp rx=0 tx=0).stop 2024-04-26T20:56:47.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.763+0000 7f6ef6400640 1 -- 172.21.15.28:0/671763534 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef00d2160 msgr2=0x7f6ef015b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:47.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.763+0000 7f6ef6400640 1 --2- 172.21.15.28:0/671763534 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef00d2160 0x7f6ef015b060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6ed8001260 tx=0x7f6ed800c960 comp rx=0 tx=0).stop 2024-04-26T20:56:47.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.763+0000 7f6ef6400640 1 -- 172.21.15.28:0/671763534 shutdown_connections 2024-04-26T20:56:47.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.763+0000 7f6ef6400640 1 --2- 172.21.15.28:0/671763534 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6ecc06c860 0x7f6ecc06ed20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:47.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.763+0000 7f6ef6400640 1 --2- 172.21.15.28:0/671763534 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ef015f680 0x7f6ef0161a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:47.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.763+0000 7f6ef6400640 1 --2- 172.21.15.28:0/671763534 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ef00d2160 0x7f6ef015b060 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:47.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.763+0000 7f6ef6400640 1 --2- 172.21.15.28:0/671763534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ef00b9e70 0x7f6ef0158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:47.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.763+0000 7f6ef6400640 1 -- 172.21.15.28:0/671763534 >> 172.21.15.28:0/671763534 conn(0x7f6ef006ca10 msgr2=0x7f6ef00c9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:47.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.763+0000 7f6ef6400640 1 -- 172.21.15.28:0/671763534 shutdown_connections 2024-04-26T20:56:47.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.763+0000 7f6ef6400640 1 -- 172.21.15.28:0/671763534 wait complete. 2024-04-26T20:56:47.783 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T20:56:47.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.993+0000 7f58a6a00640 1 Processor -- start 2024-04-26T20:56:47.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.993+0000 7f58a6a00640 1 -- start start 2024-04-26T20:56:47.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.995+0000 7f58a6a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f58a01611f0 0x7f58a00cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:47.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.995+0000 7f58a6a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58a015ae90 con 0x7f58a015aab0 2024-04-26T20:56:47.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.995+0000 7f58a6a00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58a00cfc90 con 0x7f58a0073d10 2024-04-26T20:56:47.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.995+0000 7f58a6a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f58a00cfa40 con 0x7f58a01611f0 2024-04-26T20:56:47.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.995+0000 7f58a4c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f58a01611f0 0x7f58a00cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:47.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.995+0000 7f58a4c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f58a01611f0 0x7f58a00cd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51768/0 (socket says 172.21.15.28:51768) 2024-04-26T20:56:47.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.995+0000 7f58a4c00640 1 -- 172.21.15.28:0/2307443116 learned_addr learned my addr 172.21.15.28:0/2307443116 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:47.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.996+0000 7f58a4c00640 1 -- 172.21.15.28:0/2307443116 >> v1:172.21.15.103:6789/0 conn(0x7f58a0073d10 legacy=0x7f58a00724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:47.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.996+0000 7f58a4c00640 1 -- 172.21.15.28:0/2307443116 >> v1:172.21.15.28:6789/0 conn(0x7f58a015aab0 legacy=0x7f58a0160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:47.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.996+0000 7f58a4c00640 1 -- 172.21.15.28:0/2307443116 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a00cff70 con 0x7f58a01611f0 2024-04-26T20:56:47.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.997+0000 7f58a4c00640 1 --2- 172.21.15.28:0/2307443116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a01611f0 0x7f58a00cd450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f588c009870 tx=0x7f588c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715f7ae5a17eb360 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:47.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.997+0000 7f589fe00640 1 -- 172.21.15.28:0/2307443116 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c037070 con 0x7f58a01611f0 2024-04-26T20:56:47.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.997+0000 7f589fe00640 1 -- 172.21.15.28:0/2307443116 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f588c032440 con 0x7f58a01611f0 2024-04-26T20:56:47.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.997+0000 7f589fe00640 1 -- 172.21.15.28:0/2307443116 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c032810 con 0x7f58a01611f0 2024-04-26T20:56:47.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.998+0000 7f58a6a00640 1 -- 172.21.15.28:0/2307443116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a01611f0 msgr2=0x7f58a00cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:47.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.998+0000 7f58a6a00640 1 --2- 172.21.15.28:0/2307443116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a01611f0 0x7f58a00cd450 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f588c009870 tx=0x7f588c02a210 comp rx=0 tx=0).stop 2024-04-26T20:56:47.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.998+0000 7f58a6a00640 1 -- 172.21.15.28:0/2307443116 shutdown_connections 2024-04-26T20:56:47.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.998+0000 7f58a6a00640 1 --2- 172.21.15.28:0/2307443116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a01611f0 0x7f58a00cd450 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:47.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.998+0000 7f58a6a00640 1 -- 172.21.15.28:0/2307443116 >> 172.21.15.28:0/2307443116 conn(0x7f58a00b2720 msgr2=0x7f58a00b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:47.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.998+0000 7f58a6a00640 1 -- 172.21.15.28:0/2307443116 shutdown_connections 2024-04-26T20:56:47.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:47.998+0000 7f58a6a00640 1 -- 172.21.15.28:0/2307443116 wait complete. 2024-04-26T20:56:48.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.000+0000 7f58a6a00640 1 Processor -- start 2024-04-26T20:56:48.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.000+0000 7f58a6a00640 1 -- start start 2024-04-26T20:56:48.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.001+0000 7f58a6a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f58a0073d10 0x7f58a0133b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:48.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.001+0000 7f58a6a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a015aab0 0x7f58a01f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:48.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.001+0000 7f58a6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58a01611f0 0x7f58a0135ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:48.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.001+0000 7f58a6a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f58a00d0250 con 0x7f58a015aab0 2024-04-26T20:56:48.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.001+0000 7f58a6a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f58a0072040 con 0x7f58a01611f0 2024-04-26T20:56:48.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.001+0000 7f58a6a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f58a0074120 con 0x7f58a0073d10 2024-04-26T20:56:48.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.001+0000 7f58a4c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a015aab0 0x7f58a01f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:48.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.001+0000 7f58a4c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a015aab0 0x7f58a01f8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51770/0 (socket says 172.21.15.28:51770) 2024-04-26T20:56:48.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.001+0000 7f58a4c00640 1 -- 172.21.15.28:0/1241802813 learned_addr learned my addr 172.21.15.28:0/1241802813 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:48.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.001+0000 7f58a4c00640 1 -- 172.21.15.28:0/1241802813 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f58a0073d10 msgr2=0x7f58a0133b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:48.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.001+0000 7f58a4c00640 1 --2- 172.21.15.28:0/1241802813 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f58a0073d10 0x7f58a0133b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:48.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.001+0000 7f58a4c00640 1 -- 172.21.15.28:0/1241802813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58a01611f0 msgr2=0x7f58a0135ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:48.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.001+0000 7f58a6000640 1 --2- 172.21.15.28:0/1241802813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58a01611f0 0x7f58a0135ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:48.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.001+0000 7f58a4c00640 1 --2- 172.21.15.28:0/1241802813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58a01611f0 0x7f58a0135ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:48.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.001+0000 7f58a4c00640 1 -- 172.21.15.28:0/1241802813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a01360e0 con 0x7f58a015aab0 2024-04-26T20:56:48.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.001+0000 7f58a6000640 1 --2- 172.21.15.28:0/1241802813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58a01611f0 0x7f58a0135ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:56:48.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.002+0000 7f58a4c00640 1 --2- 172.21.15.28:0/1241802813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a015aab0 0x7f58a01f8fe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f588c000c00 tx=0x7f588c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:48.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.002+0000 7f589d600640 1 -- 172.21.15.28:0/1241802813 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c037040 con 0x7f58a015aab0 2024-04-26T20:56:48.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.002+0000 7f589d600640 1 -- 172.21.15.28:0/1241802813 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f588c042070 con 0x7f58a015aab0 2024-04-26T20:56:48.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.002+0000 7f58a6a00640 1 -- 172.21.15.28:0/1241802813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a0136370 con 0x7f58a015aab0 2024-04-26T20:56:48.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.002+0000 7f589d600640 1 -- 172.21.15.28:0/1241802813 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c030bb0 con 0x7f58a015aab0 2024-04-26T20:56:48.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.002+0000 7f58a6a00640 1 -- 172.21.15.28:0/1241802813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a0136930 con 0x7f58a015aab0 2024-04-26T20:56:48.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.003+0000 7f58a6a00640 1 -- 172.21.15.28:0/1241802813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a006a360 con 0x7f58a015aab0 2024-04-26T20:56:48.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.004+0000 7f589d600640 1 -- 172.21.15.28:0/1241802813 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f588c047460 con 0x7f58a015aab0 2024-04-26T20:56:48.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.005+0000 7f589d600640 1 --2- 172.21.15.28:0/1241802813 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f588006c860 0x7f588006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:48.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.005+0000 7f589d600640 1 -- 172.21.15.28:0/1241802813 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f588c0ac3c0 con 0x7f58a015aab0 2024-04-26T20:56:48.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.006+0000 7f58a5600640 1 --2- 172.21.15.28:0/1241802813 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f588006c860 0x7f588006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:48.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.006+0000 7f58a5600640 1 --2- 172.21.15.28:0/1241802813 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f588006c860 0x7f588006ed20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5894000f60 tx=0x7f58940079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:48.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.010+0000 7f589d600640 1 -- 172.21.15.28:0/1241802813 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f588c045360 con 0x7f58a015aab0 2024-04-26T20:56:48.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.218+0000 7f58a6a00640 1 -- 172.21.15.28:0/1241802813 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f58a006fc80 con 0x7f58a015aab0 2024-04-26T20:56:48.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.219+0000 7f589d600640 1 -- 172.21.15.28:0/1241802813 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f588c07ac10 con 0x7f58a015aab0 2024-04-26T20:56:48.220 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:56:48.220 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2024-04-26T20:56:48.220 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:56:48.220 INFO:teuthology.orchestra.run.smithi028.stdout: "poolnum": 1, 2024-04-26T20:56:48.220 INFO:teuthology.orchestra.run.smithi028.stdout: "poolname": ".mgr" 2024-04-26T20:56:48.220 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:56:48.220 INFO:teuthology.orchestra.run.smithi028.stdout:] 2024-04-26T20:56:48.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.221+0000 7f58a6a00640 1 -- 172.21.15.28:0/1241802813 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f588006c860 msgr2=0x7f588006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:48.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.221+0000 7f58a6a00640 1 --2- 172.21.15.28:0/1241802813 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f588006c860 0x7f588006ed20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5894000f60 tx=0x7f58940079d0 comp rx=0 tx=0).stop 2024-04-26T20:56:48.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.221+0000 7f58a6a00640 1 -- 172.21.15.28:0/1241802813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a015aab0 msgr2=0x7f58a01f8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:48.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.221+0000 7f58a6a00640 1 --2- 172.21.15.28:0/1241802813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a015aab0 0x7f58a01f8fe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f588c000c00 tx=0x7f588c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:56:48.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.221+0000 7f58a6a00640 1 -- 172.21.15.28:0/1241802813 shutdown_connections 2024-04-26T20:56:48.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.221+0000 7f58a6a00640 1 --2- 172.21.15.28:0/1241802813 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f588006c860 0x7f588006ed20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:48.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.221+0000 7f58a6a00640 1 --2- 172.21.15.28:0/1241802813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f58a01611f0 0x7f58a0135ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:48.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.221+0000 7f58a6a00640 1 --2- 172.21.15.28:0/1241802813 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58a015aab0 0x7f58a01f8fe0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:48.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.221+0000 7f58a6a00640 1 --2- 172.21.15.28:0/1241802813 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f58a0073d10 0x7f58a0133b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:48.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.221+0000 7f58a6a00640 1 -- 172.21.15.28:0/1241802813 >> 172.21.15.28:0/1241802813 conn(0x7f58a00b2720 msgr2=0x7f58a006f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:48.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.222+0000 7f58a6a00640 1 -- 172.21.15.28:0/1241802813 shutdown_connections 2024-04-26T20:56:48.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.222+0000 7f58a6a00640 1 -- 172.21.15.28:0/1241802813 wait complete. 2024-04-26T20:56:48.241 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T20:56:48.242 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T20:56:48.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.439+0000 7f94bcc00640 1 Processor -- start 2024-04-26T20:56:48.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.440+0000 7f94bcc00640 1 -- start start 2024-04-26T20:56:48.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.441+0000 7f94bcc00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f94b80cd500 0x7f94b80af260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:48.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.441+0000 7f94bcc00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f94b80ceac0 0x7f94b80cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:48.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.441+0000 7f94bcc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f94b80d01a0 0x7f94b8162b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:48.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.441+0000 7f94bcc00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f94b80cde20 con 0x7f94b80d01a0 2024-04-26T20:56:48.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.441+0000 7f94bcc00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f94b806f8b0 con 0x7f94b80cd500 2024-04-26T20:56:48.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.441+0000 7f94bcc00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f94b806fb90 con 0x7f94b80ceac0 2024-04-26T20:56:48.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.442+0000 7f94b7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f94b80d01a0 0x7f94b8162b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:48.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.442+0000 7f94b7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f94b80d01a0 0x7f94b8162b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51788/0 (socket says 172.21.15.28:51788) 2024-04-26T20:56:48.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.442+0000 7f94b7e00640 1 -- 172.21.15.28:0/1304139694 learned_addr learned my addr 172.21.15.28:0/1304139694 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:48.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.442+0000 7f94afe00640 1 --2- 172.21.15.28:0/1304139694 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f94b80ceac0 0x7f94b80cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:48.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.442+0000 7f94b7400640 1 --2- 172.21.15.28:0/1304139694 >> v2:172.21.15.103:3300/0 conn(0x7f94b80cd500 0x7f94b80af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:48.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.442+0000 7f94b7e00640 1 -- 172.21.15.28:0/1304139694 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f94b80ceac0 msgr2=0x7f94b80cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:48.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.442+0000 7f94b7e00640 1 --2- 172.21.15.28:0/1304139694 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f94b80ceac0 0x7f94b80cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:48.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.442+0000 7f94b7e00640 1 -- 172.21.15.28:0/1304139694 >> v2:172.21.15.103:3300/0 conn(0x7f94b80cd500 msgr2=0x7f94b80af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:48.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.442+0000 7f94b7e00640 1 --2- 172.21.15.28:0/1304139694 >> v2:172.21.15.103:3300/0 conn(0x7f94b80cd500 0x7f94b80af260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:48.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.442+0000 7f94b7e00640 1 -- 172.21.15.28:0/1304139694 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b80aea60 con 0x7f94b80d01a0 2024-04-26T20:56:48.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.444+0000 7f94b7e00640 1 --2- 172.21.15.28:0/1304139694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94b80d01a0 0x7f94b8162b90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f94b806ffc0 tx=0x7f94b002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c124f8cfc008434 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:48.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.444+0000 7f94b6a00640 1 -- 172.21.15.28:0/1304139694 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0036070 con 0x7f94b80d01a0 2024-04-26T20:56:48.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.444+0000 7f94b6a00640 1 -- 172.21.15.28:0/1304139694 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b0031600 con 0x7f94b80d01a0 2024-04-26T20:56:48.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.444+0000 7f94b6a00640 1 -- 172.21.15.28:0/1304139694 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b00319d0 con 0x7f94b80d01a0 2024-04-26T20:56:48.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.445+0000 7f94bcc00640 1 -- 172.21.15.28:0/1304139694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94b80d01a0 msgr2=0x7f94b8162b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:48.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.445+0000 7f94bcc00640 1 --2- 172.21.15.28:0/1304139694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94b80d01a0 0x7f94b8162b90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f94b806ffc0 tx=0x7f94b002a610 comp rx=0 tx=0).stop 2024-04-26T20:56:48.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.445+0000 7f94bcc00640 1 -- 172.21.15.28:0/1304139694 shutdown_connections 2024-04-26T20:56:48.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.445+0000 7f94bcc00640 1 --2- 172.21.15.28:0/1304139694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94b80d01a0 0x7f94b8162b90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:48.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.445+0000 7f94bcc00640 1 --2- 172.21.15.28:0/1304139694 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f94b80ceac0 0x7f94b80cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:48.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.445+0000 7f94bcc00640 1 --2- 172.21.15.28:0/1304139694 >> v2:172.21.15.103:3300/0 conn(0x7f94b80cd500 0x7f94b80af260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:48.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.445+0000 7f94bcc00640 1 -- 172.21.15.28:0/1304139694 >> 172.21.15.28:0/1304139694 conn(0x7f94b80b2820 msgr2=0x7f94b80b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:48.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.445+0000 7f94bcc00640 1 -- 172.21.15.28:0/1304139694 shutdown_connections 2024-04-26T20:56:48.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.445+0000 7f94bcc00640 1 -- 172.21.15.28:0/1304139694 wait complete. 2024-04-26T20:56:48.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.447+0000 7f94bcc00640 1 Processor -- start 2024-04-26T20:56:48.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.447+0000 7f94bcc00640 1 -- start start 2024-04-26T20:56:48.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.448+0000 7f94bcc00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f94b80cd500 0x7f94b80ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:48.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.448+0000 7f94bcc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94b80ceac0 0x7f94b8158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:48.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.448+0000 7f94bcc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94b80d01a0 0x7f94b80adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:48.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.448+0000 7f94bcc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f94b815cfd0 con 0x7f94b80ceac0 2024-04-26T20:56:48.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.448+0000 7f94bcc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f94b815ce50 con 0x7f94b80d01a0 2024-04-26T20:56:48.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.448+0000 7f94bcc00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f94b80d0650 con 0x7f94b80cd500 2024-04-26T20:56:48.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.448+0000 7f94afe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94b80ceac0 0x7f94b8158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:48.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.448+0000 7f94afe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94b80ceac0 0x7f94b8158d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51802/0 (socket says 172.21.15.28:51802) 2024-04-26T20:56:48.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.448+0000 7f94afe00640 1 -- 172.21.15.28:0/822410616 learned_addr learned my addr 172.21.15.28:0/822410616 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:48.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.448+0000 7f94b7400640 1 --2- 172.21.15.28:0/822410616 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f94b80cd500 0x7f94b80ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:48.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.448+0000 7f94b7e00640 1 --2- 172.21.15.28:0/822410616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94b80d01a0 0x7f94b80adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:48.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.449+0000 7f94afe00640 1 -- 172.21.15.28:0/822410616 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f94b80cd500 msgr2=0x7f94b80ac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:48.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.449+0000 7f94afe00640 1 --2- 172.21.15.28:0/822410616 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f94b80cd500 0x7f94b80ac3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:48.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.449+0000 7f94afe00640 1 -- 172.21.15.28:0/822410616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94b80d01a0 msgr2=0x7f94b80adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:48.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.449+0000 7f94afe00640 1 --2- 172.21.15.28:0/822410616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94b80d01a0 0x7f94b80adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:48.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.449+0000 7f94afe00640 1 -- 172.21.15.28:0/822410616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94b80adff0 con 0x7f94b80ceac0 2024-04-26T20:56:48.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.449+0000 7f94afe00640 1 --2- 172.21.15.28:0/822410616 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94b80ceac0 0x7f94b8158d00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f94a000e9b0 tx=0x7f94a000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:48.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.449+0000 7f94af400640 1 -- 172.21.15.28:0/822410616 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a0017070 con 0x7f94b80ceac0 2024-04-26T20:56:48.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.450+0000 7f94af400640 1 -- 172.21.15.28:0/822410616 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a000c9a0 con 0x7f94b80ceac0 2024-04-26T20:56:48.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.450+0000 7f94bcc00640 1 -- 172.21.15.28:0/822410616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94b80aca10 con 0x7f94b80ceac0 2024-04-26T20:56:48.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.450+0000 7f94af400640 1 -- 172.21.15.28:0/822410616 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a000cd90 con 0x7f94b80ceac0 2024-04-26T20:56:48.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.450+0000 7f94bcc00640 1 -- 172.21.15.28:0/822410616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94b80acfd0 con 0x7f94b80ceac0 2024-04-26T20:56:48.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.451+0000 7f94bcc00640 1 -- 172.21.15.28:0/822410616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94b806a3d0 con 0x7f94b80ceac0 2024-04-26T20:56:48.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.452+0000 7f94af400640 1 -- 172.21.15.28:0/822410616 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f94a001a070 con 0x7f94b80ceac0 2024-04-26T20:56:48.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.453+0000 7f94af400640 1 --2- 172.21.15.28:0/822410616 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f949406c860 0x7f949406ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:48.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.453+0000 7f94af400640 1 -- 172.21.15.28:0/822410616 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f94a008be60 con 0x7f94b80ceac0 2024-04-26T20:56:48.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.454+0000 7f94b7400640 1 --2- 172.21.15.28:0/822410616 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f949406c860 0x7f949406ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:48.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.454+0000 7f94b7400640 1 --2- 172.21.15.28:0/822410616 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f949406c860 0x7f949406ed20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f94a8006fd0 tx=0x7f94a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:48.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.458+0000 7f94af400640 1 -- 172.21.15.28:0/822410616 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f94a005a050 con 0x7f94b80ceac0 2024-04-26T20:56:48.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:48.672+0000 7f94bcc00640 1 -- 172.21.15.28:0/822410616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f94b80cd2e0 con 0x7f94b80ceac0 2024-04-26T20:56:49.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.640+0000 7f94af400640 1 -- 172.21.15.28:0/822410616 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f94a005e350 con 0x7f94b80ceac0 2024-04-26T20:56:49.642 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' created 2024-04-26T20:56:49.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.644+0000 7f94bcc00640 1 -- 172.21.15.28:0/822410616 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f949406c860 msgr2=0x7f949406ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:49.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.644+0000 7f94bcc00640 1 --2- 172.21.15.28:0/822410616 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f949406c860 0x7f949406ed20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f94a8006fd0 tx=0x7f94a8008040 comp rx=0 tx=0).stop 2024-04-26T20:56:49.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.644+0000 7f94bcc00640 1 -- 172.21.15.28:0/822410616 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94b80ceac0 msgr2=0x7f94b8158d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:49.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.644+0000 7f94bcc00640 1 --2- 172.21.15.28:0/822410616 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94b80ceac0 0x7f94b8158d00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f94a000e9b0 tx=0x7f94a000ee70 comp rx=0 tx=0).stop 2024-04-26T20:56:49.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.645+0000 7f94bcc00640 1 -- 172.21.15.28:0/822410616 shutdown_connections 2024-04-26T20:56:49.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.645+0000 7f94bcc00640 1 --2- 172.21.15.28:0/822410616 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f949406c860 0x7f949406ed20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:49.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.645+0000 7f94bcc00640 1 --2- 172.21.15.28:0/822410616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f94b80d01a0 0x7f94b80adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:49.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.645+0000 7f94bcc00640 1 --2- 172.21.15.28:0/822410616 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94b80ceac0 0x7f94b8158d00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:49.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.645+0000 7f94bcc00640 1 --2- 172.21.15.28:0/822410616 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f94b80cd500 0x7f94b80ac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:49.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.645+0000 7f94bcc00640 1 -- 172.21.15.28:0/822410616 >> 172.21.15.28:0/822410616 conn(0x7f94b80b2820 msgr2=0x7f94b806e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:49.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.645+0000 7f94bcc00640 1 -- 172.21.15.28:0/822410616 shutdown_connections 2024-04-26T20:56:49.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.646+0000 7f94bcc00640 1 -- 172.21.15.28:0/822410616 wait complete. 2024-04-26T20:56:49.678 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T20:56:49.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.872+0000 7ffa1be00640 1 Processor -- start 2024-04-26T20:56:49.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.872+0000 7ffa1be00640 1 -- start start 2024-04-26T20:56:49.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.874+0000 7ffa1be00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ffa1c1468e0 0x7ffa1c06dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:49.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.874+0000 7ffa1be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffa1c06efd0 0x7ffa1c0733a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:49.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.874+0000 7ffa1be00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa1c0cf7e0 con 0x7ffa1c06eb40 2024-04-26T20:56:49.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.874+0000 7ffa1be00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ffa1c146cc0 con 0x7ffa1c06efd0 2024-04-26T20:56:49.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.874+0000 7ffa1be00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7ffa1c0cfac0 con 0x7ffa1c1468e0 2024-04-26T20:56:49.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.874+0000 7ffa1b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffa1c06efd0 0x7ffa1c0733a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:49.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.874+0000 7ffa1a000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ffa1c06eb40 0x7ffa1c072c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57640/0 (socket says 172.21.15.28:57640) 2024-04-26T20:56:49.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.874+0000 7ffa1a000640 1 -- 172.21.15.28:0/725865687 learned_addr learned my addr 172.21.15.28:0/725865687 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:49.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.874+0000 7ffa1b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffa1c06efd0 0x7ffa1c0733a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41932/0 (socket says 172.21.15.28:41932) 2024-04-26T20:56:49.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.874+0000 7ffa1aa00640 1 --2- 172.21.15.28:0/725865687 >> v2:172.21.15.103:3300/0 conn(0x7ffa1c1468e0 0x7ffa1c06dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.875+0000 7ffa1b400640 1 -- 172.21.15.28:0/725865687 >> v2:172.21.15.103:3300/0 conn(0x7ffa1c1468e0 msgr2=0x7ffa1c06dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.875+0000 7ffa1b400640 1 --2- 172.21.15.28:0/725865687 >> v2:172.21.15.103:3300/0 conn(0x7ffa1c1468e0 0x7ffa1c06dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.875+0000 7ffa1b400640 1 -- 172.21.15.28:0/725865687 >> v1:172.21.15.28:6789/0 conn(0x7ffa1c06eb40 legacy=0x7ffa1c072c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.875+0000 7ffa1b400640 1 -- 172.21.15.28:0/725865687 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1c0cfda0 con 0x7ffa1c06efd0 2024-04-26T20:56:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.875+0000 7ffa19600640 1 -- 172.21.15.28:0/725865687 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3010520671 0 0) 0x7ffa1c0cf7e0 con 0x7ffa1c06eb40 2024-04-26T20:56:49.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.876+0000 7ffa1b400640 1 --2- 172.21.15.28:0/725865687 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa1c06efd0 0x7ffa1c0733a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c06ddb0 tx=0x7ffa1402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8173df47b6c2717f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.876+0000 7ffa19600640 1 -- 172.21.15.28:0/725865687 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14007740 con 0x7ffa1c06efd0 2024-04-26T20:56:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.876+0000 7ffa19600640 1 -- 172.21.15.28:0/725865687 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa140078e0 con 0x7ffa1c06efd0 2024-04-26T20:56:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.876+0000 7ffa19600640 1 -- 172.21.15.28:0/725865687 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14007cb0 con 0x7ffa1c06efd0 2024-04-26T20:56:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.877+0000 7ffa1be00640 1 -- 172.21.15.28:0/725865687 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa1c06efd0 msgr2=0x7ffa1c0733a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.877+0000 7ffa1be00640 1 --2- 172.21.15.28:0/725865687 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa1c06efd0 0x7ffa1c0733a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c06ddb0 tx=0x7ffa1402a680 comp rx=0 tx=0).stop 2024-04-26T20:56:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.877+0000 7ffa1be00640 1 -- 172.21.15.28:0/725865687 shutdown_connections 2024-04-26T20:56:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.877+0000 7ffa1be00640 1 --2- 172.21.15.28:0/725865687 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa1c06efd0 0x7ffa1c0733a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.877+0000 7ffa1be00640 1 --2- 172.21.15.28:0/725865687 >> v2:172.21.15.103:3300/0 conn(0x7ffa1c1468e0 0x7ffa1c06dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.877+0000 7ffa1be00640 1 -- 172.21.15.28:0/725865687 >> 172.21.15.28:0/725865687 conn(0x7ffa1c0b2820 msgr2=0x7ffa1c0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:49.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.877+0000 7ffa1be00640 1 -- 172.21.15.28:0/725865687 shutdown_connections 2024-04-26T20:56:49.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.877+0000 7ffa1be00640 1 -- 172.21.15.28:0/725865687 wait complete. 2024-04-26T20:56:49.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.879+0000 7ffa1be00640 1 Processor -- start 2024-04-26T20:56:49.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.879+0000 7ffa1be00640 1 -- start start 2024-04-26T20:56:49.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.880+0000 7ffa1be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa1c06eb40 0x7ffa1c0b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:49.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.880+0000 7ffa1be00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa1c06efd0 0x7ffa1c13a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:49.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.880+0000 7ffa1be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa1c1468e0 0x7ffa1c15e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:49.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.880+0000 7ffa1be00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c0d0880 con 0x7ffa1c1468e0 2024-04-26T20:56:49.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.880+0000 7ffa1be00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c0d0360 con 0x7ffa1c06eb40 2024-04-26T20:56:49.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.880+0000 7ffa1be00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ffa1c06f3e0 con 0x7ffa1c06efd0 2024-04-26T20:56:49.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.880+0000 7ffa1b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa1c1468e0 0x7ffa1c15e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:49.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.880+0000 7ffa1b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa1c1468e0 0x7ffa1c15e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41940/0 (socket says 172.21.15.28:41940) 2024-04-26T20:56:49.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.880+0000 7ffa1b400640 1 -- 172.21.15.28:0/1588925140 learned_addr learned my addr 172.21.15.28:0/1588925140 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:49.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.880+0000 7ffa1b400640 1 -- 172.21.15.28:0/1588925140 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa1c06efd0 msgr2=0x7ffa1c13a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:49.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.880+0000 7ffa1aa00640 1 --2- 172.21.15.28:0/1588925140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa1c06eb40 0x7ffa1c0b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:49.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.880+0000 7ffa1b400640 1 --2- 172.21.15.28:0/1588925140 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa1c06efd0 0x7ffa1c13a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:49.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.880+0000 7ffa1b400640 1 -- 172.21.15.28:0/1588925140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa1c06eb40 msgr2=0x7ffa1c0b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:49.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.880+0000 7ffa1b400640 1 --2- 172.21.15.28:0/1588925140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa1c06eb40 0x7ffa1c0b0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:49.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.881+0000 7ffa1b400640 1 -- 172.21.15.28:0/1588925140 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1c0b0940 con 0x7ffa1c1468e0 2024-04-26T20:56:49.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.881+0000 7ffa1b400640 1 --2- 172.21.15.28:0/1588925140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa1c1468e0 0x7ffa1c15e290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009e80 tx=0x7ffa14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:49.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.881+0000 7ffa12a00640 1 -- 172.21.15.28:0/1588925140 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14040070 con 0x7ffa1c1468e0 2024-04-26T20:56:49.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.881+0000 7ffa12a00640 1 -- 172.21.15.28:0/1588925140 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa14031c40 con 0x7ffa1c1468e0 2024-04-26T20:56:49.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.881+0000 7ffa12a00640 1 -- 172.21.15.28:0/1588925140 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14009450 con 0x7ffa1c1468e0 2024-04-26T20:56:49.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.881+0000 7ffa1be00640 1 -- 172.21.15.28:0/1588925140 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1c0b0c20 con 0x7ffa1c1468e0 2024-04-26T20:56:49.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.881+0000 7ffa1be00640 1 -- 172.21.15.28:0/1588925140 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa1c0b14c0 con 0x7ffa1c1468e0 2024-04-26T20:56:49.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.882+0000 7ffa1be00640 1 -- 172.21.15.28:0/1588925140 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa1c0b0df0 con 0x7ffa1c1468e0 2024-04-26T20:56:49.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.883+0000 7ffa12a00640 1 -- 172.21.15.28:0/1588925140 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa14039040 con 0x7ffa1c1468e0 2024-04-26T20:56:49.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.885+0000 7ffa12a00640 1 --2- 172.21.15.28:0/1588925140 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff9fc06c8f0 0x7ff9fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:49.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.885+0000 7ffa12a00640 1 -- 172.21.15.28:0/1588925140 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7ffa140acc50 con 0x7ffa1c1468e0 2024-04-26T20:56:49.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.885+0000 7ffa1aa00640 1 --2- 172.21.15.28:0/1588925140 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff9fc06c8f0 0x7ff9fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:49.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.886+0000 7ffa1aa00640 1 --2- 172.21.15.28:0/1588925140 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff9fc06c8f0 0x7ff9fc06edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c006fd0 tx=0x7ffa0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:49.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:49.890+0000 7ffa12a00640 1 -- 172.21.15.28:0/1588925140 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa1407b1c0 con 0x7ffa1c1468e0 2024-04-26T20:56:50.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.100+0000 7ffa1be00640 1 -- 172.21.15.28:0/1588925140 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ffa1c06a2c0 con 0x7ffa1c1468e0 2024-04-26T20:56:50.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.647+0000 7ffa12a00640 1 -- 172.21.15.28:0/1588925140 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7ffa1407aac0 con 0x7ffa1c1468e0 2024-04-26T20:56:50.649 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' created 2024-04-26T20:56:50.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.651+0000 7ffa1be00640 1 -- 172.21.15.28:0/1588925140 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff9fc06c8f0 msgr2=0x7ff9fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:50.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.651+0000 7ffa1be00640 1 --2- 172.21.15.28:0/1588925140 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff9fc06c8f0 0x7ff9fc06edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c006fd0 tx=0x7ffa0c008040 comp rx=0 tx=0).stop 2024-04-26T20:56:50.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.651+0000 7ffa1be00640 1 -- 172.21.15.28:0/1588925140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa1c1468e0 msgr2=0x7ffa1c15e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:50.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.651+0000 7ffa1be00640 1 --2- 172.21.15.28:0/1588925140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa1c1468e0 0x7ffa1c15e290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009e80 tx=0x7ffa14004a40 comp rx=0 tx=0).stop 2024-04-26T20:56:50.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.652+0000 7ffa1be00640 1 -- 172.21.15.28:0/1588925140 shutdown_connections 2024-04-26T20:56:50.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.652+0000 7ffa1be00640 1 --2- 172.21.15.28:0/1588925140 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff9fc06c8f0 0x7ff9fc06edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:50.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.652+0000 7ffa1be00640 1 --2- 172.21.15.28:0/1588925140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa1c1468e0 0x7ffa1c15e290 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:50.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.652+0000 7ffa1be00640 1 --2- 172.21.15.28:0/1588925140 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa1c06efd0 0x7ffa1c13a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:50.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.652+0000 7ffa1be00640 1 --2- 172.21.15.28:0/1588925140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa1c06eb40 0x7ffa1c0b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:50.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.652+0000 7ffa1be00640 1 -- 172.21.15.28:0/1588925140 >> 172.21.15.28:0/1588925140 conn(0x7ffa1c0b2820 msgr2=0x7ffa1c1425a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:50.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.652+0000 7ffa1be00640 1 -- 172.21.15.28:0/1588925140 shutdown_connections 2024-04-26T20:56:50.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.653+0000 7ffa1be00640 1 -- 172.21.15.28:0/1588925140 wait complete. 2024-04-26T20:56:50.691 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T20:56:50.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.873+0000 7fa40d600640 1 Processor -- start 2024-04-26T20:56:50.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.874+0000 7fa40d600640 1 -- start start 2024-04-26T20:56:50.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.875+0000 7fa40d600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa40806ed20 0x7fa408169cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:50.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.875+0000 7fa40d600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa40806f2d0 con 0x7fa4080b4f90 2024-04-26T20:56:50.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.875+0000 7fa40d600640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa40806f5b0 con 0x7fa4080731f0 2024-04-26T20:56:50.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.875+0000 7fa40d600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa40806f890 con 0x7fa40806ed20 2024-04-26T20:56:50.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.875+0000 7fa3ffe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa4080b4f90 0x7fa4080735d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57642/0 (socket says 172.21.15.28:57642) 2024-04-26T20:56:50.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.875+0000 7fa3ffe00640 1 -- 172.21.15.28:0/1968472697 learned_addr learned my addr 172.21.15.28:0/1968472697 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:50.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.876+0000 7fa407400640 1 --2- 172.21.15.28:0/1968472697 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa40806ed20 0x7fa408169cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:50.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.876+0000 7fa406000640 1 -- 172.21.15.28:0/1968472697 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 476212192 0 0) 0x7fa40806f2d0 con 0x7fa4080b4f90 2024-04-26T20:56:50.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.876+0000 7fa406000640 1 -- 172.21.15.28:0/1968472697 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3ec003520 con 0x7fa4080b4f90 2024-04-26T20:56:50.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.876+0000 7fa406000640 1 -- 172.21.15.28:0/1968472697 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2297976830 0 0) 0x7fa3ec003520 con 0x7fa4080b4f90 2024-04-26T20:56:50.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.876+0000 7fa406000640 1 -- 172.21.15.28:0/1968472697 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa40806ed20 msgr2=0x7fa408169cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:50.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.876+0000 7fa406000640 1 --2- 172.21.15.28:0/1968472697 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa40806ed20 0x7fa408169cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:50.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.876+0000 7fa406000640 1 -- 172.21.15.28:0/1968472697 >> v1:172.21.15.103:6789/0 conn(0x7fa4080731f0 legacy=0x7fa40806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:50.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.876+0000 7fa406000640 1 -- 172.21.15.28:0/1968472697 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40806fb70 con 0x7fa4080b4f90 2024-04-26T20:56:50.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.876+0000 7fa406000640 1 -- 172.21.15.28:0/1968472697 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fa3f0001670 con 0x7fa4080b4f90 2024-04-26T20:56:50.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.876+0000 7fa406000640 1 -- 172.21.15.28:0/1968472697 >> v1:172.21.15.28:6789/0 conn(0x7fa4080b4f90 legacy=0x7fa4080735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:50.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.876+0000 7fa406000640 1 --2- 172.21.15.28:0/1968472697 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa40806ed20 0x7fa3ec0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:50.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.876+0000 7fa406000640 1 --2- 172.21.15.28:0/1968472697 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ec003f60 0x7fa3ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:50.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.876+0000 7fa406000640 1 --2- 172.21.15.28:0/1968472697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3ec004d40 0x7fa3ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:50.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.876+0000 7fa406000640 1 -- 172.21.15.28:0/1968472697 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa40806f890 con 0x7fa3ec003f60 2024-04-26T20:56:50.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.876+0000 7fa406000640 1 -- 172.21.15.28:0/1968472697 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec00bb50 con 0x7fa3ec004d40 2024-04-26T20:56:50.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.876+0000 7fa406000640 1 -- 172.21.15.28:0/1968472697 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa3ec00bd50 con 0x7fa40806ed20 2024-04-26T20:56:50.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.877+0000 7fa3ffe00640 1 --2- 172.21.15.28:0/1968472697 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ec003f60 0x7fa3ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:50.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.877+0000 7fa406a00640 1 --2- 172.21.15.28:0/1968472697 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa40806ed20 0x7fa3ec0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:50.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.877+0000 7fa407400640 1 --2- 172.21.15.28:0/1968472697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3ec004d40 0x7fa3ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:50.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.877+0000 7fa3ffe00640 1 -- 172.21.15.28:0/1968472697 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa40806ed20 msgr2=0x7fa3ec0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:50.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.877+0000 7fa3ffe00640 1 --2- 172.21.15.28:0/1968472697 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa40806ed20 0x7fa3ec0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:50.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.877+0000 7fa3ffe00640 1 -- 172.21.15.28:0/1968472697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3ec004d40 msgr2=0x7fa3ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:50.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.877+0000 7fa3ffe00640 1 --2- 172.21.15.28:0/1968472697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3ec004d40 0x7fa3ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:50.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.877+0000 7fa3ffe00640 1 -- 172.21.15.28:0/1968472697 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ec003520 con 0x7fa3ec003f60 2024-04-26T20:56:50.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.877+0000 7fa407400640 1 --2- 172.21.15.28:0/1968472697 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3ec004d40 0x7fa3ec0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:56:50.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.877+0000 7fa406a00640 1 --2- 172.21.15.28:0/1968472697 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa40806ed20 0x7fa3ec0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:56:50.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.878+0000 7fa3ffe00640 1 --2- 172.21.15.28:0/1968472697 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ec003f60 0x7fa3ec004800 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c210 tx=0x7fa3ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d74958f2c529af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:50.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.878+0000 7fa406000640 1 -- 172.21.15.28:0/1968472697 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0007cf0 con 0x7fa3ec003f60 2024-04-26T20:56:50.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.879+0000 7fa406000640 1 -- 172.21.15.28:0/1968472697 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3f0005af0 con 0x7fa3ec003f60 2024-04-26T20:56:50.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.879+0000 7fa406000640 1 -- 172.21.15.28:0/1968472697 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0007070 con 0x7fa3ec003f60 2024-04-26T20:56:50.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.879+0000 7fa40d600640 1 -- 172.21.15.28:0/1968472697 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ec003f60 msgr2=0x7fa3ec004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:50.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.879+0000 7fa40d600640 1 --2- 172.21.15.28:0/1968472697 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ec003f60 0x7fa3ec004800 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000c210 tx=0x7fa3ec005830 comp rx=0 tx=0).stop 2024-04-26T20:56:50.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.879+0000 7fa407400640 1 -- 172.21.15.28:0/1968472697 reap_dead start 2024-04-26T20:56:50.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.879+0000 7fa40d600640 1 -- 172.21.15.28:0/1968472697 shutdown_connections 2024-04-26T20:56:50.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.879+0000 7fa40d600640 1 -- 172.21.15.28:0/1968472697 >> 172.21.15.28:0/1968472697 conn(0x7fa40806c770 msgr2=0x7fa40806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:50.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.879+0000 7fa40d600640 1 -- 172.21.15.28:0/1968472697 shutdown_connections 2024-04-26T20:56:50.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.879+0000 7fa40d600640 1 -- 172.21.15.28:0/1968472697 wait complete. 2024-04-26T20:56:50.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.881+0000 7fa40d600640 1 Processor -- start 2024-04-26T20:56:50.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.881+0000 7fa40d600640 1 -- start start 2024-04-26T20:56:50.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.882+0000 7fa40d600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3ec003f60 0x7fa408142d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:50.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.882+0000 7fa40d600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa40813ac90 0x7fa408137f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:50.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.882+0000 7fa40d600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa408139e40 0x7fa4080b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:50.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.882+0000 7fa40d600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa408073e60 con 0x7fa408139e40 2024-04-26T20:56:50.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.882+0000 7fa40d600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa408073ce0 con 0x7fa3ec003f60 2024-04-26T20:56:50.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.882+0000 7fa40d600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa408073fe0 con 0x7fa40813ac90 2024-04-26T20:56:50.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.883+0000 7fa407400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa408139e40 0x7fa4080b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:50.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.883+0000 7fa407400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa408139e40 0x7fa4080b16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41956/0 (socket says 172.21.15.28:41956) 2024-04-26T20:56:50.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.883+0000 7fa407400640 1 -- 172.21.15.28:0/342682056 learned_addr learned my addr 172.21.15.28:0/342682056 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:50.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.883+0000 7fa406a00640 1 --2- 172.21.15.28:0/342682056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3ec003f60 0x7fa408142d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:50.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.883+0000 7fa3ffe00640 1 --2- 172.21.15.28:0/342682056 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa40813ac90 0x7fa408137f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:50.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.883+0000 7fa407400640 1 -- 172.21.15.28:0/342682056 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa40813ac90 msgr2=0x7fa408137f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:50.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.883+0000 7fa407400640 1 --2- 172.21.15.28:0/342682056 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa40813ac90 0x7fa408137f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:50.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.883+0000 7fa407400640 1 -- 172.21.15.28:0/342682056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3ec003f60 msgr2=0x7fa408142d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:50.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.883+0000 7fa407400640 1 --2- 172.21.15.28:0/342682056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3ec003f60 0x7fa408142d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:50.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.883+0000 7fa407400640 1 -- 172.21.15.28:0/342682056 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4081432c0 con 0x7fa408139e40 2024-04-26T20:56:50.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.883+0000 7fa407400640 1 --2- 172.21.15.28:0/342682056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa408139e40 0x7fa4080b16e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa4080b5370 tx=0x7fa40000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:50.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.884+0000 7fa3fea00640 1 -- 172.21.15.28:0/342682056 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40001c070 con 0x7fa408139e40 2024-04-26T20:56:50.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.884+0000 7fa3fea00640 1 -- 172.21.15.28:0/342682056 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa400017400 con 0x7fa408139e40 2024-04-26T20:56:50.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.884+0000 7fa3fea00640 1 -- 172.21.15.28:0/342682056 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4000177d0 con 0x7fa408139e40 2024-04-26T20:56:50.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.884+0000 7fa40d600640 1 -- 172.21.15.28:0/342682056 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4080aa630 con 0x7fa408139e40 2024-04-26T20:56:50.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.884+0000 7fa40d600640 1 -- 172.21.15.28:0/342682056 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa408141660 con 0x7fa408139e40 2024-04-26T20:56:50.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.885+0000 7fa40d600640 1 -- 172.21.15.28:0/342682056 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4080aab50 con 0x7fa408139e40 2024-04-26T20:56:50.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.886+0000 7fa3fea00640 1 -- 172.21.15.28:0/342682056 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa40001f070 con 0x7fa408139e40 2024-04-26T20:56:50.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.887+0000 7fa3fea00640 1 --2- 172.21.15.28:0/342682056 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa3e406c920 0x7fa3e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:50.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.887+0000 7fa3fea00640 1 -- 172.21.15.28:0/342682056 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7fa40008bc70 con 0x7fa408139e40 2024-04-26T20:56:50.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.887+0000 7fa406a00640 1 --2- 172.21.15.28:0/342682056 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa3e406c920 0x7fa3e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:50.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.888+0000 7fa406a00640 1 --2- 172.21.15.28:0/342682056 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa3e406c920 0x7fa3e406ede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8001750 tx=0x7fa3f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:50.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:50.893+0000 7fa3fea00640 1 -- 172.21.15.28:0/342682056 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa400012070 con 0x7fa408139e40 2024-04-26T20:56:51.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.141+0000 7fa40d600640 1 -- 172.21.15.28:0/342682056 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa40806a360 con 0x7fa408139e40 2024-04-26T20:56:51.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.700+0000 7fa3fea00640 1 -- 172.21.15.28:0/342682056 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fa40005a060 con 0x7fa408139e40 2024-04-26T20:56:51.704 INFO:teuthology.orchestra.run.smithi028.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T20:56:51.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.706+0000 7fa40d600640 1 -- 172.21.15.28:0/342682056 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa3e406c920 msgr2=0x7fa3e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:51.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.706+0000 7fa40d600640 1 --2- 172.21.15.28:0/342682056 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa3e406c920 0x7fa3e406ede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8001750 tx=0x7fa3f8009040 comp rx=0 tx=0).stop 2024-04-26T20:56:51.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.706+0000 7fa40d600640 1 -- 172.21.15.28:0/342682056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa408139e40 msgr2=0x7fa4080b16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:51.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.706+0000 7fa40d600640 1 --2- 172.21.15.28:0/342682056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa408139e40 0x7fa4080b16e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa4080b5370 tx=0x7fa40000f4a0 comp rx=0 tx=0).stop 2024-04-26T20:56:51.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.706+0000 7fa40d600640 1 -- 172.21.15.28:0/342682056 shutdown_connections 2024-04-26T20:56:51.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.707+0000 7fa40d600640 1 --2- 172.21.15.28:0/342682056 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa3e406c920 0x7fa3e406ede0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:51.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.707+0000 7fa40d600640 1 --2- 172.21.15.28:0/342682056 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa408139e40 0x7fa4080b16e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:51.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.707+0000 7fa40d600640 1 --2- 172.21.15.28:0/342682056 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa40813ac90 0x7fa408137f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:51.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.707+0000 7fa40d600640 1 --2- 172.21.15.28:0/342682056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3ec003f60 0x7fa408142d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:51.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.707+0000 7fa40d600640 1 -- 172.21.15.28:0/342682056 >> 172.21.15.28:0/342682056 conn(0x7fa40806c770 msgr2=0x7fa4080735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:51.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.708+0000 7fa40d600640 1 -- 172.21.15.28:0/342682056 shutdown_connections 2024-04-26T20:56:51.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.708+0000 7fa40d600640 1 -- 172.21.15.28:0/342682056 wait complete. 2024-04-26T20:56:51.746 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:56:51.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.920+0000 7f6fbd400640 1 Processor -- start 2024-04-26T20:56:51.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.920+0000 7f6fbd400640 1 -- start start 2024-04-26T20:56:51.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.922+0000 7f6fbd400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6fb815f1b0 0x7f6fb80d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:51.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.922+0000 7f6fbd400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6fb80d5540 0x7f6fb80b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:51.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.922+0000 7f6fbd400640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fb80d0710 con 0x7f6fb80cf700 2024-04-26T20:56:51.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.922+0000 7f6fbd400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f6fb80cff00 con 0x7f6fb815f1b0 2024-04-26T20:56:51.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.922+0000 7f6fbd400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f6fb80d4270 con 0x7f6fb80d5540 2024-04-26T20:56:51.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.922+0000 7f6fb7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6fb815f1b0 0x7f6fb80d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:51.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.922+0000 7f6fb7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6fb815f1b0 0x7f6fb80d4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41964/0 (socket says 172.21.15.28:41964) 2024-04-26T20:56:51.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.922+0000 7f6fb7e00640 1 -- 172.21.15.28:0/807195829 learned_addr learned my addr 172.21.15.28:0/807195829 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:51.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.923+0000 7f6fbca00640 1 --2- 172.21.15.28:0/807195829 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6fb80d5540 0x7f6fb80b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:51.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.923+0000 7f6fb7e00640 1 -- 172.21.15.28:0/807195829 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6fb80d5540 msgr2=0x7f6fb80b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:51.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.923+0000 7f6fb7e00640 1 --2- 172.21.15.28:0/807195829 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6fb80d5540 0x7f6fb80b80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:51.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.923+0000 7f6fb7e00640 1 -- 172.21.15.28:0/807195829 >> v1:172.21.15.103:6789/0 conn(0x7f6fb80cf700 legacy=0x7f6fb80d4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:51.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.923+0000 7f6fb7e00640 1 -- 172.21.15.28:0/807195829 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb80d0150 con 0x7f6fb815f1b0 2024-04-26T20:56:51.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.924+0000 7f6fb7e00640 1 --2- 172.21.15.28:0/807195829 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fb815f1b0 0x7f6fb80d4700 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009870 tx=0x7f6fa802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15b3213aeffdb5f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:51.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.924+0000 7f6fb7400640 1 -- 172.21.15.28:0/807195829 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8037070 con 0x7f6fb815f1b0 2024-04-26T20:56:51.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.924+0000 7f6fb7400640 1 -- 172.21.15.28:0/807195829 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fa8032440 con 0x7f6fb815f1b0 2024-04-26T20:56:51.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.924+0000 7f6fb7400640 1 -- 172.21.15.28:0/807195829 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8032810 con 0x7f6fb815f1b0 2024-04-26T20:56:51.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.925+0000 7f6fbd400640 1 -- 172.21.15.28:0/807195829 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fb815f1b0 msgr2=0x7f6fb80d4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:51.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.925+0000 7f6fbd400640 1 --2- 172.21.15.28:0/807195829 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fb815f1b0 0x7f6fb80d4700 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009870 tx=0x7f6fa802a2a0 comp rx=0 tx=0).stop 2024-04-26T20:56:51.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.925+0000 7f6fbd400640 1 -- 172.21.15.28:0/807195829 shutdown_connections 2024-04-26T20:56:51.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.925+0000 7f6fbd400640 1 --2- 172.21.15.28:0/807195829 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6fb80d5540 0x7f6fb80b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:51.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.925+0000 7f6fbd400640 1 --2- 172.21.15.28:0/807195829 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fb815f1b0 0x7f6fb80d4700 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:51.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.925+0000 7f6fbd400640 1 -- 172.21.15.28:0/807195829 >> 172.21.15.28:0/807195829 conn(0x7f6fb806c770 msgr2=0x7f6fb806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:51.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.925+0000 7f6fbd400640 1 -- 172.21.15.28:0/807195829 shutdown_connections 2024-04-26T20:56:51.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.925+0000 7f6fbd400640 1 -- 172.21.15.28:0/807195829 wait complete. 2024-04-26T20:56:51.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.927+0000 7f6fbd400640 1 Processor -- start 2024-04-26T20:56:51.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.927+0000 7f6fbd400640 1 -- start start 2024-04-26T20:56:51.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.928+0000 7f6fbd400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6fb80cf700 0x7f6fb814ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:51.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.928+0000 7f6fbd400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fb80d5540 0x7f6fb81601f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:51.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.928+0000 7f6fbd400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6fb815f1b0 0x7f6fb80caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:51.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.928+0000 7f6fbd400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb815f590 con 0x7f6fb80d5540 2024-04-26T20:56:51.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.928+0000 7f6fbd400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6fb80d3f00 con 0x7f6fb80cf700 2024-04-26T20:56:51.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.928+0000 7f6fbd400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f6fb80b4410 con 0x7f6fb815f1b0 2024-04-26T20:56:51.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.928+0000 7f6faf400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fb80d5540 0x7f6fb81601f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:51.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.928+0000 7f6faf400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fb80d5540 0x7f6fb81601f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41966/0 (socket says 172.21.15.28:41966) 2024-04-26T20:56:51.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.928+0000 7f6faf400640 1 -- 172.21.15.28:0/2436048344 learned_addr learned my addr 172.21.15.28:0/2436048344 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:51.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.928+0000 7f6fb7e00640 1 --2- 172.21.15.28:0/2436048344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6fb80cf700 0x7f6fb814ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:51.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.928+0000 7f6fbca00640 1 --2- 172.21.15.28:0/2436048344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6fb815f1b0 0x7f6fb80caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:51.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.929+0000 7f6faf400640 1 -- 172.21.15.28:0/2436048344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6fb815f1b0 msgr2=0x7f6fb80caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:51.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.929+0000 7f6faf400640 1 --2- 172.21.15.28:0/2436048344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6fb815f1b0 0x7f6fb80caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:51.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.929+0000 7f6faf400640 1 -- 172.21.15.28:0/2436048344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6fb80cf700 msgr2=0x7f6fb814ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:51.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.929+0000 7f6faf400640 1 --2- 172.21.15.28:0/2436048344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6fb80cf700 0x7f6fb814ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:51.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.929+0000 7f6faf400640 1 -- 172.21.15.28:0/2436048344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fb80cb430 con 0x7f6fb80d5540 2024-04-26T20:56:51.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.929+0000 7f6fbca00640 1 --2- 172.21.15.28:0/2436048344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6fb815f1b0 0x7f6fb80caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:56:51.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.929+0000 7f6faf400640 1 --2- 172.21.15.28:0/2436048344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fb80d5540 0x7f6fb81601f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000c460 tx=0x7f6fa000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:51.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.929+0000 7f6fb4c00640 1 -- 172.21.15.28:0/2436048344 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0016020 con 0x7f6fb80d5540 2024-04-26T20:56:51.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.929+0000 7f6fb4c00640 1 -- 172.21.15.28:0/2436048344 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fa0012870 con 0x7f6fb80d5540 2024-04-26T20:56:51.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.929+0000 7f6fb4c00640 1 -- 172.21.15.28:0/2436048344 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0012c60 con 0x7f6fb80d5540 2024-04-26T20:56:51.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.929+0000 7f6fbd400640 1 -- 172.21.15.28:0/2436048344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fb80cb710 con 0x7f6fb80d5540 2024-04-26T20:56:51.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.929+0000 7f6fbd400640 1 -- 172.21.15.28:0/2436048344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fb815f9f0 con 0x7f6fb80d5540 2024-04-26T20:56:51.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.930+0000 7f6fbd400640 1 -- 172.21.15.28:0/2436048344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fb806a360 con 0x7f6fb80d5540 2024-04-26T20:56:51.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.931+0000 7f6fb4c00640 1 -- 172.21.15.28:0/2436048344 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6fa0019020 con 0x7f6fb80d5540 2024-04-26T20:56:51.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.932+0000 7f6fb4c00640 1 --2- 172.21.15.28:0/2436048344 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6f9406c8f0 0x7f6f9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:51.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.932+0000 7f6fb4c00640 1 -- 172.21.15.28:0/2436048344 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6fa008b8a0 con 0x7f6fb80d5540 2024-04-26T20:56:51.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.933+0000 7f6fb7e00640 1 --2- 172.21.15.28:0/2436048344 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6f9406c8f0 0x7f6f9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:51.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.933+0000 7f6fb7e00640 1 --2- 172.21.15.28:0/2436048344 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6f9406c8f0 0x7f6f9406edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8034000 tx=0x7f6fa802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:51.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:51.937+0000 7f6fb4c00640 1 -- 172.21.15.28:0/2436048344 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6fa0020030 con 0x7f6fb80d5540 2024-04-26T20:56:52.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.141+0000 7f6fbd400640 1 -- 172.21.15.28:0/2436048344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fb806cc60 con 0x7f6fb80d5540 2024-04-26T20:56:52.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.143+0000 7f6fb4c00640 1 -- 172.21.15.28:0/2436048344 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16594 (secure 0 0 0) 0x7f6fa0059cc0 con 0x7f6fb80d5540 2024-04-26T20:56:52.145 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:56:52.145 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":20,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:56:51.143762+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:56:52.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.146+0000 7f6fbd400640 1 -- 172.21.15.28:0/2436048344 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6f9406c8f0 msgr2=0x7f6f9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:52.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.146+0000 7f6fbd400640 1 --2- 172.21.15.28:0/2436048344 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6f9406c8f0 0x7f6f9406edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8034000 tx=0x7f6fa802e000 comp rx=0 tx=0).stop 2024-04-26T20:56:52.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.146+0000 7f6fbd400640 1 -- 172.21.15.28:0/2436048344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fb80d5540 msgr2=0x7f6fb81601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:52.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.146+0000 7f6fbd400640 1 --2- 172.21.15.28:0/2436048344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fb80d5540 0x7f6fb81601f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000c460 tx=0x7f6fa000c920 comp rx=0 tx=0).stop 2024-04-26T20:56:52.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.146+0000 7f6fbd400640 1 -- 172.21.15.28:0/2436048344 shutdown_connections 2024-04-26T20:56:52.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.146+0000 7f6fbd400640 1 --2- 172.21.15.28:0/2436048344 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6f9406c8f0 0x7f6f9406edb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:52.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.146+0000 7f6fbd400640 1 --2- 172.21.15.28:0/2436048344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6fb815f1b0 0x7f6fb80caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:52.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.146+0000 7f6fbd400640 1 --2- 172.21.15.28:0/2436048344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fb80d5540 0x7f6fb81601f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:52.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.146+0000 7f6fbd400640 1 --2- 172.21.15.28:0/2436048344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6fb80cf700 0x7f6fb814ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:52.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.146+0000 7f6fbd400640 1 -- 172.21.15.28:0/2436048344 >> 172.21.15.28:0/2436048344 conn(0x7f6fb806c770 msgr2=0x7f6fb815a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:52.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.146+0000 7f6fbd400640 1 -- 172.21.15.28:0/2436048344 shutdown_connections 2024-04-26T20:56:52.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.146+0000 7f6fbd400640 1 -- 172.21.15.28:0/2436048344 wait complete. 2024-04-26T20:56:52.166 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:56:52.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.370+0000 7f539b800640 1 Processor -- start 2024-04-26T20:56:52.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.371+0000 7f539b800640 1 -- start start 2024-04-26T20:56:52.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.372+0000 7f539b800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5394146980 0x7f53940b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:52.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.372+0000 7f539b800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53940d0a30 con 0x7f539415dcc0 2024-04-26T20:56:52.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.372+0000 7f539b800640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f539415e230 con 0x7f53940740e0 2024-04-26T20:56:52.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.372+0000 7f539b800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f539415e510 con 0x7f5394146980 2024-04-26T20:56:52.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.373+0000 7f5399800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f539415dcc0 0x7f5394071ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57658/0 (socket says 172.21.15.28:57658) 2024-04-26T20:56:52.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.373+0000 7f5399800640 1 -- 172.21.15.28:0/3752771405 learned_addr learned my addr 172.21.15.28:0/3752771405 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:52.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.373+0000 7f5398e00640 1 --2- 172.21.15.28:0/3752771405 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5394146980 0x7f53940b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:52.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.373+0000 7f5393400640 1 -- 172.21.15.28:0/3752771405 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1918992787 0 0) 0x7f538c001490 con 0x7f539415dcc0 2024-04-26T20:56:52.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.373+0000 7f5393400640 1 -- 172.21.15.28:0/3752771405 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5378003520 con 0x7f539415dcc0 2024-04-26T20:56:52.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.373+0000 7f5393400640 1 -- 172.21.15.28:0/3752771405 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2822951829 0 0) 0x7f5378003520 con 0x7f539415dcc0 2024-04-26T20:56:52.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.374+0000 7f5393400640 1 -- 172.21.15.28:0/3752771405 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5394146980 msgr2=0x7f53940b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:52.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.374+0000 7f5393400640 1 --2- 172.21.15.28:0/3752771405 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5394146980 0x7f53940b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:52.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.374+0000 7f5393400640 1 -- 172.21.15.28:0/3752771405 >> v1:172.21.15.103:6789/0 conn(0x7f53940740e0 legacy=0x7f539415d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:52.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.374+0000 7f5393400640 1 -- 172.21.15.28:0/3752771405 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5394073600 con 0x7f539415dcc0 2024-04-26T20:56:52.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.374+0000 7f5393400640 1 -- 172.21.15.28:0/3752771405 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f538c001bc0 con 0x7f539415dcc0 2024-04-26T20:56:52.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.374+0000 7f5393400640 1 -- 172.21.15.28:0/3752771405 >> v1:172.21.15.28:6789/0 conn(0x7f539415dcc0 legacy=0x7f5394071ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:52.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.374+0000 7f5393400640 1 --2- 172.21.15.28:0/3752771405 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f53780039e0 0x7f5378003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:52.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.374+0000 7f5393400640 1 --2- 172.21.15.28:0/3752771405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5378004800 0x7f5378004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:52.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.374+0000 7f5393400640 1 --2- 172.21.15.28:0/3752771405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53780051c0 0x7f537800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:52.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.374+0000 7f5393400640 1 -- 172.21.15.28:0/3752771405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f539415e510 con 0x7f53780051c0 2024-04-26T20:56:52.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.374+0000 7f5393400640 1 -- 172.21.15.28:0/3752771405 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f537800c0c0 con 0x7f5378004800 2024-04-26T20:56:52.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.374+0000 7f5393400640 1 -- 172.21.15.28:0/3752771405 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f537800c2c0 con 0x7f53780039e0 2024-04-26T20:56:52.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.374+0000 7f5399800640 1 --2- 172.21.15.28:0/3752771405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53780051c0 0x7f537800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:52.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.374+0000 7f5398e00640 1 --2- 172.21.15.28:0/3752771405 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f53780039e0 0x7f5378003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:52.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.374+0000 7f5399800640 1 -- 172.21.15.28:0/3752771405 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f53780039e0 msgr2=0x7f5378003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:52.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.374+0000 7f5399800640 1 --2- 172.21.15.28:0/3752771405 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f53780039e0 0x7f5378003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:52.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.374+0000 7f5393e00640 1 --2- 172.21.15.28:0/3752771405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5378004800 0x7f5378004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:52.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.374+0000 7f5399800640 1 -- 172.21.15.28:0/3752771405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5378004800 msgr2=0x7f5378004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:52.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.374+0000 7f5399800640 1 --2- 172.21.15.28:0/3752771405 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5378004800 0x7f5378004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:52.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.374+0000 7f5399800640 1 -- 172.21.15.28:0/3752771405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5378003520 con 0x7f53780051c0 2024-04-26T20:56:52.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.375+0000 7f5399800640 1 --2- 172.21.15.28:0/3752771405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53780051c0 0x7f537800b9c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f538c00a950 tx=0x7f5378005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a98c6955dc94b3e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:52.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.376+0000 7f5393400640 1 -- 172.21.15.28:0/3752771405 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c00a5a0 con 0x7f53780051c0 2024-04-26T20:56:52.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.376+0000 7f5393400640 1 -- 172.21.15.28:0/3752771405 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f538c02f070 con 0x7f53780051c0 2024-04-26T20:56:52.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.376+0000 7f5393400640 1 -- 172.21.15.28:0/3752771405 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c02be10 con 0x7f53780051c0 2024-04-26T20:56:52.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.376+0000 7f539b800640 1 -- 172.21.15.28:0/3752771405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53780051c0 msgr2=0x7f537800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:52.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.376+0000 7f539b800640 1 --2- 172.21.15.28:0/3752771405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53780051c0 0x7f537800b9c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f538c00a950 tx=0x7f5378005830 comp rx=0 tx=0).stop 2024-04-26T20:56:52.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.376+0000 7f5399800640 1 -- 172.21.15.28:0/3752771405 reap_dead start 2024-04-26T20:56:52.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.376+0000 7f539b800640 1 -- 172.21.15.28:0/3752771405 shutdown_connections 2024-04-26T20:56:52.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.376+0000 7f539b800640 1 -- 172.21.15.28:0/3752771405 >> 172.21.15.28:0/3752771405 conn(0x7f539406e220 msgr2=0x7f539406e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:52.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.376+0000 7f539b800640 1 -- 172.21.15.28:0/3752771405 shutdown_connections 2024-04-26T20:56:52.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.377+0000 7f539b800640 1 -- 172.21.15.28:0/3752771405 wait complete. 2024-04-26T20:56:52.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.378+0000 7f539b800640 1 Processor -- start 2024-04-26T20:56:52.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.378+0000 7f539b800640 1 -- start start 2024-04-26T20:56:52.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.379+0000 7f539b800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f53780051c0 0x7f5394152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:52.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.379+0000 7f539b800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53941f4bf0 0x7f53941542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:52.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.379+0000 7f539b800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53941547f0 0x7f53940afc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:52.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.379+0000 7f539b800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5394072700 con 0x7f53941547f0 2024-04-26T20:56:52.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.379+0000 7f539b800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5394073bc0 con 0x7f53941f4bf0 2024-04-26T20:56:52.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.379+0000 7f539b800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f5394072880 con 0x7f53780051c0 2024-04-26T20:56:52.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.380+0000 7f5399800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53941547f0 0x7f53940afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:52.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.380+0000 7f5399800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53941547f0 0x7f53940afc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41976/0 (socket says 172.21.15.28:41976) 2024-04-26T20:56:52.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.380+0000 7f5399800640 1 -- 172.21.15.28:0/2192275908 learned_addr learned my addr 172.21.15.28:0/2192275908 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:52.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.380+0000 7f5393e00640 1 --2- 172.21.15.28:0/2192275908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53941f4bf0 0x7f53941542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:52.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.380+0000 7f5398e00640 1 --2- 172.21.15.28:0/2192275908 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f53780051c0 0x7f5394152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:52.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.380+0000 7f5399800640 1 -- 172.21.15.28:0/2192275908 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f53780051c0 msgr2=0x7f5394152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:52.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.380+0000 7f5399800640 1 --2- 172.21.15.28:0/2192275908 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f53780051c0 0x7f5394152260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:52.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.380+0000 7f5399800640 1 -- 172.21.15.28:0/2192275908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53941f4bf0 msgr2=0x7f53941542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:52.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.380+0000 7f5399800640 1 --2- 172.21.15.28:0/2192275908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53941f4bf0 0x7f53941542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:52.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.380+0000 7f5399800640 1 -- 172.21.15.28:0/2192275908 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53941527a0 con 0x7f53941547f0 2024-04-26T20:56:52.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.380+0000 7f5399800640 1 --2- 172.21.15.28:0/2192275908 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53941547f0 0x7f53940afc70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f538c02b620 tx=0x7f538c001ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:52.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.381+0000 7f5390c00640 1 -- 172.21.15.28:0/2192275908 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c02bd80 con 0x7f53941547f0 2024-04-26T20:56:52.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.381+0000 7f5390c00640 1 -- 172.21.15.28:0/2192275908 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f538c00a580 con 0x7f53941547f0 2024-04-26T20:56:52.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.381+0000 7f5390c00640 1 -- 172.21.15.28:0/2192275908 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c0346b0 con 0x7f53941547f0 2024-04-26T20:56:52.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.381+0000 7f539b800640 1 -- 172.21.15.28:0/2192275908 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5394154d70 con 0x7f53941547f0 2024-04-26T20:56:52.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.381+0000 7f539b800640 1 -- 172.21.15.28:0/2192275908 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539413b870 con 0x7f53941547f0 2024-04-26T20:56:52.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.382+0000 7f539b800640 1 -- 172.21.15.28:0/2192275908 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f539406a400 con 0x7f53941547f0 2024-04-26T20:56:52.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.383+0000 7f5390c00640 1 -- 172.21.15.28:0/2192275908 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f538c046020 con 0x7f53941547f0 2024-04-26T20:56:52.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.384+0000 7f5390c00640 1 --2- 172.21.15.28:0/2192275908 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f537006c920 0x7f537006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:52.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.384+0000 7f5390c00640 1 -- 172.21.15.28:0/2192275908 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f538c0abb90 con 0x7f53941547f0 2024-04-26T20:56:52.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.384+0000 7f5398e00640 1 --2- 172.21.15.28:0/2192275908 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f537006c920 0x7f537006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:52.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.385+0000 7f5398e00640 1 --2- 172.21.15.28:0/2192275908 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f537006c920 0x7f537006ede0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f53940b10f0 tx=0x7f5384008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:52.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.389+0000 7f5390c00640 1 -- 172.21.15.28:0/2192275908 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f53940d0a30 con 0x7f53941547f0 2024-04-26T20:56:52.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.594+0000 7f539b800640 1 -- 172.21.15.28:0/2192275908 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f539406ff00 con 0x7f53941547f0 2024-04-26T20:56:52.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.597+0000 7f5390c00640 1 -- 172.21.15.28:0/2192275908 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16594 (secure 0 0 0) 0x7f538c079fb0 con 0x7f53941547f0 2024-04-26T20:56:52.599 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:56:52.599 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":20,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:56:51.143762+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:56:52.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.599+0000 7f539b800640 1 -- 172.21.15.28:0/2192275908 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f537006c920 msgr2=0x7f537006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:52.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.599+0000 7f539b800640 1 --2- 172.21.15.28:0/2192275908 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f537006c920 0x7f537006ede0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f53940b10f0 tx=0x7f5384008040 comp rx=0 tx=0).stop 2024-04-26T20:56:52.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.600+0000 7f539b800640 1 -- 172.21.15.28:0/2192275908 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53941547f0 msgr2=0x7f53940afc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:52.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.600+0000 7f539b800640 1 --2- 172.21.15.28:0/2192275908 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53941547f0 0x7f53940afc70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f538c02b620 tx=0x7f538c001ec0 comp rx=0 tx=0).stop 2024-04-26T20:56:52.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.600+0000 7f539b800640 1 -- 172.21.15.28:0/2192275908 shutdown_connections 2024-04-26T20:56:52.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.600+0000 7f539b800640 1 --2- 172.21.15.28:0/2192275908 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f537006c920 0x7f537006ede0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:52.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.600+0000 7f539b800640 1 --2- 172.21.15.28:0/2192275908 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53941547f0 0x7f53940afc70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:52.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.600+0000 7f539b800640 1 --2- 172.21.15.28:0/2192275908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53941f4bf0 0x7f53941542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:52.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.600+0000 7f539b800640 1 --2- 172.21.15.28:0/2192275908 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f53780051c0 0x7f5394152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:52.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.600+0000 7f539b800640 1 -- 172.21.15.28:0/2192275908 >> 172.21.15.28:0/2192275908 conn(0x7f539406e220 msgr2=0x7f53940b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:52.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.600+0000 7f539b800640 1 -- 172.21.15.28:0/2192275908 shutdown_connections 2024-04-26T20:56:52.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.600+0000 7f539b800640 1 -- 172.21.15.28:0/2192275908 wait complete. 2024-04-26T20:56:52.618 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T20:56:52.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.822+0000 7fda70800640 1 Processor -- start 2024-04-26T20:56:52.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.822+0000 7fda70800640 1 -- start start 2024-04-26T20:56:52.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.823+0000 7fda70800640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fda680ceaa0 0x7fda6806e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:52.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.824+0000 7fda70800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda68074400 con 0x7fda6806fc10 2024-04-26T20:56:52.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.824+0000 7fda70800640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda6815e310 con 0x7fda68073f50 2024-04-26T20:56:52.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.824+0000 7fda70800640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fda68072aa0 con 0x7fda680ceaa0 2024-04-26T20:56:52.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.824+0000 7fda6de00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fda6806fc10 0x7fda68073580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57666/0 (socket says 172.21.15.28:57666) 2024-04-26T20:56:52.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.824+0000 7fda6de00640 1 -- 172.21.15.28:0/3110231978 learned_addr learned my addr 172.21.15.28:0/3110231978 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:52.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.824+0000 7fda6ca00640 1 -- 172.21.15.28:0/3110231978 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3017629560 0 0) 0x7fda68074400 con 0x7fda6806fc10 2024-04-26T20:56:52.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.824+0000 7fda6e800640 1 --2- 172.21.15.28:0/3110231978 >> v2:172.21.15.103:3300/0 conn(0x7fda680ceaa0 0x7fda6806e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:52.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.824+0000 7fda6ca00640 1 -- 172.21.15.28:0/3110231978 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda50003520 con 0x7fda6806fc10 2024-04-26T20:56:52.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6ca00640 1 -- 172.21.15.28:0/3110231978 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2999126554 0 0) 0x7fda50003520 con 0x7fda6806fc10 2024-04-26T20:56:52.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6ca00640 1 -- 172.21.15.28:0/3110231978 >> v2:172.21.15.103:3300/0 conn(0x7fda680ceaa0 msgr2=0x7fda6806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:52.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6ca00640 1 --2- 172.21.15.28:0/3110231978 >> v2:172.21.15.103:3300/0 conn(0x7fda680ceaa0 0x7fda6806e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:52.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6ca00640 1 -- 172.21.15.28:0/3110231978 >> v1:172.21.15.103:6789/0 conn(0x7fda68073f50 legacy=0x7fda680ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:52.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6ca00640 1 -- 172.21.15.28:0/3110231978 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda68072d80 con 0x7fda6806fc10 2024-04-26T20:56:52.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6ca00640 1 -- 172.21.15.28:0/3110231978 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fda5c001670 con 0x7fda6806fc10 2024-04-26T20:56:52.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6ca00640 1 -- 172.21.15.28:0/3110231978 >> v1:172.21.15.28:6789/0 conn(0x7fda6806fc10 legacy=0x7fda68073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:52.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6ca00640 1 --2- 172.21.15.28:0/3110231978 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda50003a00 0x7fda50003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:52.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6ca00640 1 --2- 172.21.15.28:0/3110231978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda50004800 0x7fda50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:52.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6ca00640 1 --2- 172.21.15.28:0/3110231978 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fda50005290 0x7fda5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:52.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6ca00640 1 -- 172.21.15.28:0/3110231978 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fda68072aa0 con 0x7fda50003a00 2024-04-26T20:56:52.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6ca00640 1 -- 172.21.15.28:0/3110231978 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fda5000c1e0 con 0x7fda50004800 2024-04-26T20:56:52.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6ca00640 1 -- 172.21.15.28:0/3110231978 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fda5000c3e0 con 0x7fda50005290 2024-04-26T20:56:52.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6e800640 1 --2- 172.21.15.28:0/3110231978 >> v2:172.21.15.103:3300/0 conn(0x7fda680ceaa0 0x7fda6806e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:56:52.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6de00640 1 --2- 172.21.15.28:0/3110231978 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda50003a00 0x7fda50003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:52.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6de00640 1 -- 172.21.15.28:0/3110231978 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fda50005290 msgr2=0x7fda5000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:52.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6de00640 1 --2- 172.21.15.28:0/3110231978 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fda50005290 0x7fda5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:52.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6de00640 1 -- 172.21.15.28:0/3110231978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda50004800 msgr2=0x7fda50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:52.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6de00640 1 --2- 172.21.15.28:0/3110231978 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda50004800 0x7fda50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:52.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6de00640 1 -- 172.21.15.28:0/3110231978 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda50003520 con 0x7fda50003a00 2024-04-26T20:56:52.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.825+0000 7fda6e800640 1 -- 172.21.15.28:0/3110231978 reap_dead start 2024-04-26T20:56:52.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.826+0000 7fda6de00640 1 --2- 172.21.15.28:0/3110231978 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda50003a00 0x7fda50003e00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00c110 tx=0x7fda50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef66dd74571268e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:52.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.827+0000 7fda6ca00640 1 -- 172.21.15.28:0/3110231978 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c007c40 con 0x7fda50003a00 2024-04-26T20:56:52.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.827+0000 7fda6ca00640 1 -- 172.21.15.28:0/3110231978 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda5c005ad0 con 0x7fda50003a00 2024-04-26T20:56:52.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.827+0000 7fda6ca00640 1 -- 172.21.15.28:0/3110231978 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c007070 con 0x7fda50003a00 2024-04-26T20:56:52.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.827+0000 7fda70800640 1 -- 172.21.15.28:0/3110231978 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda50003a00 msgr2=0x7fda50003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:52.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.827+0000 7fda70800640 1 --2- 172.21.15.28:0/3110231978 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda50003a00 0x7fda50003e00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00c110 tx=0x7fda50005830 comp rx=0 tx=0).stop 2024-04-26T20:56:52.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.827+0000 7fda70800640 1 -- 172.21.15.28:0/3110231978 shutdown_connections 2024-04-26T20:56:52.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.827+0000 7fda70800640 1 --2- 172.21.15.28:0/3110231978 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda50003a00 0x7fda50003e00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:52.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.827+0000 7fda70800640 1 -- 172.21.15.28:0/3110231978 >> 172.21.15.28:0/3110231978 conn(0x7fda6806c770 msgr2=0x7fda6806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:52.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.827+0000 7fda70800640 1 -- 172.21.15.28:0/3110231978 shutdown_connections 2024-04-26T20:56:52.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.827+0000 7fda70800640 1 -- 172.21.15.28:0/3110231978 wait complete. 2024-04-26T20:56:52.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.829+0000 7fda70800640 1 Processor -- start 2024-04-26T20:56:52.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.829+0000 7fda70800640 1 -- start start 2024-04-26T20:56:52.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.830+0000 7fda70800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fda50003a00 0x7fda68154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:52.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.830+0000 7fda70800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda6813f6a0 0x7fda680b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:52.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.830+0000 7fda70800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda680b2fa0 0x7fda681420a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:52.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.831+0000 7fda70800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fda680d07d0 con 0x7fda680b2fa0 2024-04-26T20:56:52.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.831+0000 7fda70800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fda6815dfa0 con 0x7fda6813f6a0 2024-04-26T20:56:52.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.831+0000 7fda70800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fda680d0950 con 0x7fda50003a00 2024-04-26T20:56:52.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.831+0000 7fda6e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda680b2fa0 0x7fda681420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:52.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.831+0000 7fda6e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda680b2fa0 0x7fda681420a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:41992/0 (socket says 172.21.15.28:41992) 2024-04-26T20:56:52.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.831+0000 7fda6e800640 1 -- 172.21.15.28:0/1178427884 learned_addr learned my addr 172.21.15.28:0/1178427884 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:52.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.831+0000 7fda6de00640 1 --2- 172.21.15.28:0/1178427884 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fda50003a00 0x7fda68154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:52.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.831+0000 7fda6e800640 1 -- 172.21.15.28:0/1178427884 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fda50003a00 msgr2=0x7fda68154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:52.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.831+0000 7fda6e800640 1 --2- 172.21.15.28:0/1178427884 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fda50003a00 0x7fda68154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:52.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.831+0000 7fda6e800640 1 -- 172.21.15.28:0/1178427884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda6813f6a0 msgr2=0x7fda680b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:52.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.831+0000 7fda6e800640 1 --2- 172.21.15.28:0/1178427884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda6813f6a0 0x7fda680b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:52.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.831+0000 7fda6e800640 1 -- 172.21.15.28:0/1178427884 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6813fb20 con 0x7fda680b2fa0 2024-04-26T20:56:52.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.831+0000 7fda6de00640 1 --2- 172.21.15.28:0/1178427884 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fda50003a00 0x7fda68154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:56:52.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.831+0000 7fda6e800640 1 --2- 172.21.15.28:0/1178427884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda680b2fa0 0x7fda681420a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fda680b3b10 tx=0x7fda6400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:52.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.832+0000 7fda62000640 1 -- 172.21.15.28:0/1178427884 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6401a070 con 0x7fda680b2fa0 2024-04-26T20:56:52.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.832+0000 7fda62000640 1 -- 172.21.15.28:0/1178427884 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda64007e20 con 0x7fda680b2fa0 2024-04-26T20:56:52.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.832+0000 7fda70800640 1 -- 172.21.15.28:0/1178427884 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda68142dd0 con 0x7fda680b2fa0 2024-04-26T20:56:52.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.832+0000 7fda62000640 1 -- 172.21.15.28:0/1178427884 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda64003bd0 con 0x7fda680b2fa0 2024-04-26T20:56:52.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.832+0000 7fda70800640 1 -- 172.21.15.28:0/1178427884 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda68153750 con 0x7fda680b2fa0 2024-04-26T20:56:52.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.833+0000 7fda70800640 1 -- 172.21.15.28:0/1178427884 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda6806a360 con 0x7fda680b2fa0 2024-04-26T20:56:52.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.834+0000 7fda62000640 1 -- 172.21.15.28:0/1178427884 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda6401d070 con 0x7fda680b2fa0 2024-04-26T20:56:52.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.835+0000 7fda62000640 1 --2- 172.21.15.28:0/1178427884 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fda4c06c970 0x7fda4c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:52.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.835+0000 7fda62000640 1 -- 172.21.15.28:0/1178427884 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fda6408b950 con 0x7fda680b2fa0 2024-04-26T20:56:52.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.835+0000 7fda6de00640 1 --2- 172.21.15.28:0/1178427884 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fda4c06c970 0x7fda4c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:52.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.835+0000 7fda6de00640 1 --2- 172.21.15.28:0/1178427884 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fda4c06c970 0x7fda4c06ee30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fda5c032000 tx=0x7fda5c034040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:52.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:52.841+0000 7fda62000640 1 -- 172.21.15.28:0/1178427884 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda64010070 con 0x7fda680b2fa0 2024-04-26T20:56:53.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.084+0000 7fda70800640 1 -- 172.21.15.28:0/1178427884 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fda6814eff0 con 0x7fda680b2fa0 2024-04-26T20:56:53.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.719+0000 7fda62000640 1 -- 172.21.15.28:0/1178427884 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7fda64059d60 con 0x7fda680b2fa0 2024-04-26T20:56:53.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.723+0000 7fda70800640 1 -- 172.21.15.28:0/1178427884 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fda4c06c970 msgr2=0x7fda4c06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:53.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.723+0000 7fda70800640 1 --2- 172.21.15.28:0/1178427884 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fda4c06c970 0x7fda4c06ee30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fda5c032000 tx=0x7fda5c034040 comp rx=0 tx=0).stop 2024-04-26T20:56:53.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.723+0000 7fda70800640 1 -- 172.21.15.28:0/1178427884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda680b2fa0 msgr2=0x7fda681420a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:53.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.723+0000 7fda70800640 1 --2- 172.21.15.28:0/1178427884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda680b2fa0 0x7fda681420a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fda680b3b10 tx=0x7fda6400d4a0 comp rx=0 tx=0).stop 2024-04-26T20:56:53.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.724+0000 7fda70800640 1 -- 172.21.15.28:0/1178427884 shutdown_connections 2024-04-26T20:56:53.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.724+0000 7fda70800640 1 --2- 172.21.15.28:0/1178427884 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fda4c06c970 0x7fda4c06ee30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:53.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.724+0000 7fda70800640 1 --2- 172.21.15.28:0/1178427884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda680b2fa0 0x7fda681420a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:53.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.724+0000 7fda70800640 1 --2- 172.21.15.28:0/1178427884 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fda6813f6a0 0x7fda680b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:53.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.724+0000 7fda70800640 1 --2- 172.21.15.28:0/1178427884 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fda50003a00 0x7fda68154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:53.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.724+0000 7fda70800640 1 -- 172.21.15.28:0/1178427884 >> 172.21.15.28:0/1178427884 conn(0x7fda6806c770 msgr2=0x7fda680cdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:53.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.724+0000 7fda70800640 1 -- 172.21.15.28:0/1178427884 shutdown_connections 2024-04-26T20:56:53.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.724+0000 7fda70800640 1 -- 172.21.15.28:0/1178427884 wait complete. 2024-04-26T20:56:53.749 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T20:56:53.749 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T20:56:53.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.964+0000 7f0560800640 1 Processor -- start 2024-04-26T20:56:53.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.965+0000 7f0560800640 1 -- start start 2024-04-26T20:56:53.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.966+0000 7f0560800640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f055815f680 0x7f05580d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:53.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.966+0000 7f0560800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05580d3af0 0x7f05580b8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:53.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.966+0000 7f0560800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05580d4110 con 0x7f05580d1f90 2024-04-26T20:56:53.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.966+0000 7f0560800640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f05580d23f0 con 0x7f055815f680 2024-04-26T20:56:53.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.966+0000 7f0560800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f05580d43f0 con 0x7f05580d3af0 2024-04-26T20:56:53.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.967+0000 7f055d400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f05580d1f90 0x7f05580d33e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57670/0 (socket says 172.21.15.28:57670) 2024-04-26T20:56:53.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.967+0000 7f055d400640 1 -- 172.21.15.28:0/3112606716 learned_addr learned my addr 172.21.15.28:0/3112606716 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:53.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.967+0000 7f055e800640 1 --2- 172.21.15.28:0/3112606716 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05580d3af0 0x7f05580b8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:53.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.967+0000 7f055de00640 1 --2- 172.21.15.28:0/3112606716 >> v2:172.21.15.103:3300/0 conn(0x7f055815f680 0x7f05580d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:53.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.967+0000 7f055ca00640 1 -- 172.21.15.28:0/3112606716 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2292027984 0 0) 0x7f05580d4110 con 0x7f05580d1f90 2024-04-26T20:56:53.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.967+0000 7f055ca00640 1 -- 172.21.15.28:0/3112606716 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0540003520 con 0x7f05580d1f90 2024-04-26T20:56:53.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.967+0000 7f055ca00640 1 -- 172.21.15.28:0/3112606716 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 495964255 0 0) 0x7f0540003520 con 0x7f05580d1f90 2024-04-26T20:56:53.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.967+0000 7f055ca00640 1 -- 172.21.15.28:0/3112606716 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05580d3af0 msgr2=0x7f05580b8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:53.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.967+0000 7f055ca00640 1 --2- 172.21.15.28:0/3112606716 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05580d3af0 0x7f05580b8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:53.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.968+0000 7f055ca00640 1 -- 172.21.15.28:0/3112606716 >> v2:172.21.15.103:3300/0 conn(0x7f055815f680 msgr2=0x7f05580d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:53.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.968+0000 7f055ca00640 1 --2- 172.21.15.28:0/3112606716 >> v2:172.21.15.103:3300/0 conn(0x7f055815f680 0x7f05580d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:53.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.968+0000 7f055ca00640 1 -- 172.21.15.28:0/3112606716 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05580d46d0 con 0x7f05580d1f90 2024-04-26T20:56:53.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.968+0000 7f055ca00640 1 -- 172.21.15.28:0/3112606716 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f05480017f0 con 0x7f05580d1f90 2024-04-26T20:56:53.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.968+0000 7f055ca00640 1 -- 172.21.15.28:0/3112606716 >> v1:172.21.15.28:6789/0 conn(0x7f05580d1f90 legacy=0x7f05580d33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:53.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.968+0000 7f055ca00640 1 --2- 172.21.15.28:0/3112606716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05400039b0 0x7f0540003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:53.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.968+0000 7f055ca00640 1 --2- 172.21.15.28:0/3112606716 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05580d3af0 0x7f0540004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:53.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.968+0000 7f055ca00640 1 --2- 172.21.15.28:0/3112606716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0540004d40 0x7f05400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:53.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.968+0000 7f055ca00640 1 -- 172.21.15.28:0/3112606716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f05580d43f0 con 0x7f05400039b0 2024-04-26T20:56:53.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.968+0000 7f055ca00640 1 -- 172.21.15.28:0/3112606716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f054000b9c0 con 0x7f0540004d40 2024-04-26T20:56:53.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.968+0000 7f055ca00640 1 -- 172.21.15.28:0/3112606716 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f054000bb40 con 0x7f05580d3af0 2024-04-26T20:56:53.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.968+0000 7f055de00640 1 --2- 172.21.15.28:0/3112606716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05400039b0 0x7f0540003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:53.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.968+0000 7f055d400640 1 --2- 172.21.15.28:0/3112606716 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05580d3af0 0x7f0540004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:53.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.968+0000 7f055e800640 1 --2- 172.21.15.28:0/3112606716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0540004d40 0x7f05400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:53.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.968+0000 7f055de00640 1 -- 172.21.15.28:0/3112606716 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05580d3af0 msgr2=0x7f0540004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:53.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.968+0000 7f055de00640 1 --2- 172.21.15.28:0/3112606716 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05580d3af0 0x7f0540004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:53.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.968+0000 7f055de00640 1 -- 172.21.15.28:0/3112606716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0540004d40 msgr2=0x7f05400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:53.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.968+0000 7f055de00640 1 --2- 172.21.15.28:0/3112606716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0540004d40 0x7f05400051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:53.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.968+0000 7f055de00640 1 -- 172.21.15.28:0/3112606716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0540003520 con 0x7f05400039b0 2024-04-26T20:56:53.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.969+0000 7f055de00640 1 --2- 172.21.15.28:0/3112606716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05400039b0 0x7f0540003dd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f054c009ac0 tx=0x7f0540005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e008b545374d3f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:53.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.970+0000 7f055ca00640 1 -- 172.21.15.28:0/3112606716 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c036070 con 0x7f05400039b0 2024-04-26T20:56:53.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.970+0000 7f055ca00640 1 -- 172.21.15.28:0/3112606716 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f054c031400 con 0x7f05400039b0 2024-04-26T20:56:53.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.970+0000 7f055ca00640 1 -- 172.21.15.28:0/3112606716 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c009690 con 0x7f05400039b0 2024-04-26T20:56:53.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.970+0000 7f0560800640 1 -- 172.21.15.28:0/3112606716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05400039b0 msgr2=0x7f0540003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:53.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.970+0000 7f0560800640 1 --2- 172.21.15.28:0/3112606716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05400039b0 0x7f0540003dd0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f054c009ac0 tx=0x7f0540005830 comp rx=0 tx=0).stop 2024-04-26T20:56:53.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.970+0000 7f055e800640 1 -- 172.21.15.28:0/3112606716 reap_dead start 2024-04-26T20:56:53.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.970+0000 7f0560800640 1 -- 172.21.15.28:0/3112606716 shutdown_connections 2024-04-26T20:56:53.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.970+0000 7f0560800640 1 -- 172.21.15.28:0/3112606716 >> 172.21.15.28:0/3112606716 conn(0x7f055806ca10 msgr2=0x7f0558072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:53.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.971+0000 7f0560800640 1 -- 172.21.15.28:0/3112606716 shutdown_connections 2024-04-26T20:56:53.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.971+0000 7f0560800640 1 -- 172.21.15.28:0/3112606716 wait complete. 2024-04-26T20:56:53.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.972+0000 7f0560800640 1 Processor -- start 2024-04-26T20:56:53.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.972+0000 7f0560800640 1 -- start start 2024-04-26T20:56:53.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.973+0000 7f0560800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05400039b0 0x7f05581580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:53.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.973+0000 7f0560800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0558153390 0x7f0558159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:53.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.973+0000 7f0560800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05581566e0 0x7f0558156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:53.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.973+0000 7f0560800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f05580b8e60 con 0x7f05581566e0 2024-04-26T20:56:53.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.973+0000 7f0560800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f05580b8940 con 0x7f0558153390 2024-04-26T20:56:53.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.973+0000 7f0560800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f05580b8fe0 con 0x7f05400039b0 2024-04-26T20:56:53.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.974+0000 7f055e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05581566e0 0x7f0558156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:53.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.974+0000 7f055e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05581566e0 0x7f0558156b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42008/0 (socket says 172.21.15.28:42008) 2024-04-26T20:56:53.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.974+0000 7f055e800640 1 -- 172.21.15.28:0/2524419539 learned_addr learned my addr 172.21.15.28:0/2524419539 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:53.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.974+0000 7f055e800640 1 -- 172.21.15.28:0/2524419539 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05400039b0 msgr2=0x7f05581580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:53.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.974+0000 7f055de00640 1 --2- 172.21.15.28:0/2524419539 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05400039b0 0x7f05581580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:53.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.974+0000 7f055d400640 1 --2- 172.21.15.28:0/2524419539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0558153390 0x7f0558159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:53.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.974+0000 7f055e800640 1 --2- 172.21.15.28:0/2524419539 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05400039b0 0x7f05581580a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:53.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.974+0000 7f055e800640 1 -- 172.21.15.28:0/2524419539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0558153390 msgr2=0x7f0558159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:53.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.974+0000 7f055e800640 1 --2- 172.21.15.28:0/2524419539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0558153390 0x7f0558159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:53.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.974+0000 7f055e800640 1 -- 172.21.15.28:0/2524419539 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055815a2d0 con 0x7f05581566e0 2024-04-26T20:56:53.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.974+0000 7f055de00640 1 --2- 172.21.15.28:0/2524419539 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05400039b0 0x7f05581580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:56:53.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.975+0000 7f055e800640 1 --2- 172.21.15.28:0/2524419539 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05581566e0 0x7f0558156b60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f05580cf640 tx=0x7f055400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:53.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.975+0000 7f0552000640 1 -- 172.21.15.28:0/2524419539 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055401c070 con 0x7f05581566e0 2024-04-26T20:56:53.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.975+0000 7f0552000640 1 -- 172.21.15.28:0/2524419539 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0554017400 con 0x7f05581566e0 2024-04-26T20:56:53.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.975+0000 7f0552000640 1 -- 172.21.15.28:0/2524419539 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05540177f0 con 0x7f05581566e0 2024-04-26T20:56:53.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.975+0000 7f0560800640 1 -- 172.21.15.28:0/2524419539 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055815a5b0 con 0x7f05581566e0 2024-04-26T20:56:53.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.975+0000 7f0560800640 1 -- 172.21.15.28:0/2524419539 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0558160e00 con 0x7f05581566e0 2024-04-26T20:56:53.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.976+0000 7f0560800640 1 -- 172.21.15.28:0/2524419539 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f055806a360 con 0x7f05581566e0 2024-04-26T20:56:53.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.977+0000 7f0552000640 1 -- 172.21.15.28:0/2524419539 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f055401f070 con 0x7f05581566e0 2024-04-26T20:56:53.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.978+0000 7f0552000640 1 --2- 172.21.15.28:0/2524419539 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f053c06c920 0x7f053c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:53.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.978+0000 7f0552000640 1 -- 172.21.15.28:0/2524419539 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f055408bc00 con 0x7f05581566e0 2024-04-26T20:56:53.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.979+0000 7f055de00640 1 --2- 172.21.15.28:0/2524419539 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f053c06c920 0x7f053c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:53.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.979+0000 7f055de00640 1 --2- 172.21.15.28:0/2524419539 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f053c06c920 0x7f053c06ede0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f054c033000 tx=0x7f054c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:53.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:53.983+0000 7f0552000640 1 -- 172.21.15.28:0/2524419539 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0554012070 con 0x7f05581566e0 2024-04-26T20:56:54.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.226+0000 7f0560800640 1 -- 172.21.15.28:0/2524419539 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0558160fd0 con 0x7f05581566e0 2024-04-26T20:56:54.609 INFO:tasks.ceph.mds.d.smithi103.stderr:2024-04-26T20:56:54.606+0000 7f3d2ce00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:56:54.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.722+0000 7f0552000640 1 -- 172.21.15.28:0/2524419539 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f055405a020 con 0x7f05581566e0 2024-04-26T20:56:54.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.726+0000 7f0560800640 1 -- 172.21.15.28:0/2524419539 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f053c06c920 msgr2=0x7f053c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:54.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.726+0000 7f0560800640 1 --2- 172.21.15.28:0/2524419539 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f053c06c920 0x7f053c06ede0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f054c033000 tx=0x7f054c008030 comp rx=0 tx=0).stop 2024-04-26T20:56:54.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.726+0000 7f0560800640 1 -- 172.21.15.28:0/2524419539 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05581566e0 msgr2=0x7f0558156b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:54.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.726+0000 7f0560800640 1 --2- 172.21.15.28:0/2524419539 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05581566e0 0x7f0558156b60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f05580cf640 tx=0x7f055400f4a0 comp rx=0 tx=0).stop 2024-04-26T20:56:54.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.726+0000 7f0560800640 1 -- 172.21.15.28:0/2524419539 shutdown_connections 2024-04-26T20:56:54.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.726+0000 7f0560800640 1 --2- 172.21.15.28:0/2524419539 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f053c06c920 0x7f053c06ede0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:54.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.726+0000 7f0560800640 1 --2- 172.21.15.28:0/2524419539 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05581566e0 0x7f0558156b60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:54.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.726+0000 7f0560800640 1 --2- 172.21.15.28:0/2524419539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0558153390 0x7f0558159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:54.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.726+0000 7f0560800640 1 --2- 172.21.15.28:0/2524419539 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05400039b0 0x7f05581580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:54.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.726+0000 7f0560800640 1 -- 172.21.15.28:0/2524419539 >> 172.21.15.28:0/2524419539 conn(0x7f055806ca10 msgr2=0x7f05580d4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:54.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.726+0000 7f0560800640 1 -- 172.21.15.28:0/2524419539 shutdown_connections 2024-04-26T20:56:54.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.726+0000 7f0560800640 1 -- 172.21.15.28:0/2524419539 wait complete. 2024-04-26T20:56:54.750 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T20:56:54.750 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:56:54.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.954+0000 7f242cc00640 1 Processor -- start 2024-04-26T20:56:54.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.954+0000 7f242cc00640 1 -- start start 2024-04-26T20:56:54.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.956+0000 7f242cc00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f2428073990 0x7f2428073d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:54.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.956+0000 7f242cc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f242815c430 0x7f24280b5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:54.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.956+0000 7f242cc00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24280b41b0 con 0x7f24280b5790 2024-04-26T20:56:54.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.956+0000 7f242cc00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f24280b4490 con 0x7f242815c430 2024-04-26T20:56:54.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.956+0000 7f242cc00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f24280b4770 con 0x7f2428073990 2024-04-26T20:56:54.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.956+0000 7f241fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f242815c430 0x7f24280b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:54.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.956+0000 7f241fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f242815c430 0x7f24280b5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42014/0 (socket says 172.21.15.28:42014) 2024-04-26T20:56:54.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.956+0000 7f241fe00640 1 -- 172.21.15.28:0/870697333 learned_addr learned my addr 172.21.15.28:0/870697333 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:54.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.956+0000 7f2426800640 1 --1- 172.21.15.28:0/870697333 >> v1:172.21.15.28:6789/0 conn(0x7f24280b5790 0x7f24280b5c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57686/0 (socket says 172.21.15.28:57686) 2024-04-26T20:56:54.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.956+0000 7f2425e00640 1 --2- 172.21.15.28:0/870697333 >> v2:172.21.15.103:3300/0 conn(0x7f2428073990 0x7f2428073d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:54.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.957+0000 7f241fe00640 1 -- 172.21.15.28:0/870697333 >> v2:172.21.15.103:3300/0 conn(0x7f2428073990 msgr2=0x7f2428073d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:54.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.957+0000 7f241fe00640 1 --2- 172.21.15.28:0/870697333 >> v2:172.21.15.103:3300/0 conn(0x7f2428073990 0x7f2428073d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:54.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.957+0000 7f241fe00640 1 -- 172.21.15.28:0/870697333 >> v1:172.21.15.28:6789/0 conn(0x7f24280b5790 legacy=0x7f24280b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:54.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.957+0000 7f241fe00640 1 -- 172.21.15.28:0/870697333 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24280b4a50 con 0x7f242815c430 2024-04-26T20:56:54.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.958+0000 7f241fe00640 1 --2- 172.21.15.28:0/870697333 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f242815c430 0x7f24280b5250 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2428074510 tx=0x7f241002a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74da548d8093d61 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:54.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.958+0000 7f2425400640 1 -- 172.21.15.28:0/870697333 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410037070 con 0x7f242815c430 2024-04-26T20:56:54.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.958+0000 7f2425400640 1 -- 172.21.15.28:0/870697333 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2410032440 con 0x7f242815c430 2024-04-26T20:56:54.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.958+0000 7f2425400640 1 -- 172.21.15.28:0/870697333 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410032810 con 0x7f242815c430 2024-04-26T20:56:54.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.958+0000 7f242cc00640 1 -- 172.21.15.28:0/870697333 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f242815c430 msgr2=0x7f24280b5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:54.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.958+0000 7f242cc00640 1 --2- 172.21.15.28:0/870697333 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f242815c430 0x7f24280b5250 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2428074510 tx=0x7f241002a160 comp rx=0 tx=0).stop 2024-04-26T20:56:54.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.959+0000 7f242cc00640 1 -- 172.21.15.28:0/870697333 shutdown_connections 2024-04-26T20:56:54.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.959+0000 7f242cc00640 1 --2- 172.21.15.28:0/870697333 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f242815c430 0x7f24280b5250 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:54.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.959+0000 7f242cc00640 1 --2- 172.21.15.28:0/870697333 >> v2:172.21.15.103:3300/0 conn(0x7f2428073990 0x7f2428073d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:54.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.959+0000 7f242cc00640 1 -- 172.21.15.28:0/870697333 >> 172.21.15.28:0/870697333 conn(0x7f242806e220 msgr2=0x7f242806e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:54.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.959+0000 7f242cc00640 1 -- 172.21.15.28:0/870697333 shutdown_connections 2024-04-26T20:56:54.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.959+0000 7f242cc00640 1 -- 172.21.15.28:0/870697333 wait complete. 2024-04-26T20:56:54.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.960+0000 7f242cc00640 1 Processor -- start 2024-04-26T20:56:54.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.960+0000 7f242cc00640 1 -- start start 2024-04-26T20:56:54.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.961+0000 7f242cc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2428073990 0x7f2428139740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:54.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.961+0000 7f242cc00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24280b5790 0x7f242815b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:54.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.961+0000 7f242cc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f242815b7e0 0x7f24281f5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:54.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.961+0000 7f242cc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2428072220 con 0x7f2428073990 2024-04-26T20:56:54.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.961+0000 7f242cc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f24280742b0 con 0x7f242815b7e0 2024-04-26T20:56:54.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.961+0000 7f242cc00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f24280723a0 con 0x7f24280b5790 2024-04-26T20:56:54.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.962+0000 7f2425e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2428073990 0x7f2428139740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:54.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.962+0000 7f2425e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2428073990 0x7f2428139740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42018/0 (socket says 172.21.15.28:42018) 2024-04-26T20:56:54.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.962+0000 7f2425e00640 1 -- 172.21.15.28:0/648946383 learned_addr learned my addr 172.21.15.28:0/648946383 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:54.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.962+0000 7f2426800640 1 --2- 172.21.15.28:0/648946383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f242815b7e0 0x7f24281f5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:54.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.962+0000 7f241fe00640 1 --2- 172.21.15.28:0/648946383 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24280b5790 0x7f242815b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:54.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.962+0000 7f2425e00640 1 -- 172.21.15.28:0/648946383 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24280b5790 msgr2=0x7f242815b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:54.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.962+0000 7f2425e00640 1 --2- 172.21.15.28:0/648946383 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24280b5790 0x7f242815b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:54.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.962+0000 7f2425e00640 1 -- 172.21.15.28:0/648946383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f242815b7e0 msgr2=0x7f24281f5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:54.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.962+0000 7f2425e00640 1 --2- 172.21.15.28:0/648946383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f242815b7e0 0x7f24281f5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:54.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.962+0000 7f2425e00640 1 -- 172.21.15.28:0/648946383 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2428142530 con 0x7f2428073990 2024-04-26T20:56:54.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.962+0000 7f2426800640 1 --2- 172.21.15.28:0/648946383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f242815b7e0 0x7f24281f5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:56:54.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.962+0000 7f2425e00640 1 --2- 172.21.15.28:0/648946383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2428073990 0x7f2428139740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f241800d800 tx=0x7f241800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:54.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.963+0000 7f241e000640 1 -- 172.21.15.28:0/648946383 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241800b6c0 con 0x7f2428073990 2024-04-26T20:56:54.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.963+0000 7f241e000640 1 -- 172.21.15.28:0/648946383 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2418019070 con 0x7f2428073990 2024-04-26T20:56:54.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.963+0000 7f241e000640 1 -- 172.21.15.28:0/648946383 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241800ba80 con 0x7f2428073990 2024-04-26T20:56:54.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.963+0000 7f242cc00640 1 -- 172.21.15.28:0/648946383 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2428142810 con 0x7f2428073990 2024-04-26T20:56:54.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.963+0000 7f242cc00640 1 -- 172.21.15.28:0/648946383 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24280af3c0 con 0x7f2428073990 2024-04-26T20:56:54.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.964+0000 7f242cc00640 1 -- 172.21.15.28:0/648946383 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f242806a400 con 0x7f2428073990 2024-04-26T20:56:54.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.965+0000 7f241e000640 1 -- 172.21.15.28:0/648946383 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f241800bc20 con 0x7f2428073990 2024-04-26T20:56:54.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.966+0000 7f241e000640 1 --2- 172.21.15.28:0/648946383 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f240406c8f0 0x7f240406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:54.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.966+0000 7f241e000640 1 -- 172.21.15.28:0/648946383 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f241808c0a0 con 0x7f2428073990 2024-04-26T20:56:54.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.966+0000 7f241fe00640 1 --2- 172.21.15.28:0/648946383 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f240406c8f0 0x7f240406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:54.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.967+0000 7f241fe00640 1 --2- 172.21.15.28:0/648946383 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f240406c8f0 0x7f240406edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2410034000 tx=0x7f241002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:54.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:54.970+0000 7f241e000640 1 -- 172.21.15.28:0/648946383 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2418011340 con 0x7f2428073990 2024-04-26T20:56:55.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.210+0000 7f242cc00640 1 -- 172.21.15.28:0/648946383 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24280b5e30 con 0x7f2428073990 2024-04-26T20:56:55.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.211+0000 7f241e000640 1 -- 172.21.15.28:0/648946383 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7f241805a430 con 0x7f2428073990 2024-04-26T20:56:55.213 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:56:55.213 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/2818824721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2818824721},{"type":"v1","addr":"172.21.15.103:6834","nonce":2818824721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/259712913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":259712913},{"type":"v1","addr":"172.21.15.28:6836","nonce":259712913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/989575415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":989575415},{"type":"v1","addr":"172.21.15.28:6837","nonce":989575415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:56:51.143688+0000","modified":"2024-04-26T20:56:54.228571+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6835/2361950169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":2361950169},{"type":"v1","addr":"172.21.15.103:6835","nonce":2361950169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-26T20:56:55.213 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 8 2024-04-26T20:56:55.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.214+0000 7f242cc00640 1 -- 172.21.15.28:0/648946383 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f240406c8f0 msgr2=0x7f240406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:55.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.214+0000 7f242cc00640 1 --2- 172.21.15.28:0/648946383 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f240406c8f0 0x7f240406edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2410034000 tx=0x7f241002e000 comp rx=0 tx=0).stop 2024-04-26T20:56:55.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.214+0000 7f242cc00640 1 -- 172.21.15.28:0/648946383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2428073990 msgr2=0x7f2428139740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:55.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.214+0000 7f242cc00640 1 --2- 172.21.15.28:0/648946383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2428073990 0x7f2428139740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f241800d800 tx=0x7f241800dcc0 comp rx=0 tx=0).stop 2024-04-26T20:56:55.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.214+0000 7f242cc00640 1 -- 172.21.15.28:0/648946383 shutdown_connections 2024-04-26T20:56:55.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.214+0000 7f242cc00640 1 --2- 172.21.15.28:0/648946383 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f240406c8f0 0x7f240406edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:55.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.214+0000 7f242cc00640 1 --2- 172.21.15.28:0/648946383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f242815b7e0 0x7f24281f5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:55.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.214+0000 7f242cc00640 1 --2- 172.21.15.28:0/648946383 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24280b5790 0x7f242815b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:55.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.214+0000 7f242cc00640 1 --2- 172.21.15.28:0/648946383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2428073990 0x7f2428139740 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:55.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.214+0000 7f242cc00640 1 -- 172.21.15.28:0/648946383 >> 172.21.15.28:0/648946383 conn(0x7f242806e220 msgr2=0x7f24280b6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:55.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.215+0000 7f242cc00640 1 -- 172.21.15.28:0/648946383 shutdown_connections 2024-04-26T20:56:55.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.215+0000 7f242cc00640 1 -- 172.21.15.28:0/648946383 wait complete. 2024-04-26T20:56:55.233 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:56:55.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.427+0000 7fbc8e800640 1 Processor -- start 2024-04-26T20:56:55.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.427+0000 7fbc8e800640 1 -- start start 2024-04-26T20:56:55.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.429+0000 7fbc8e800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbc88146670 0x7fbc88071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:55.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.429+0000 7fbc8e800640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fbc881610c0 0x7fbc880cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:55.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.429+0000 7fbc8e800640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc88161530 con 0x7fbc88146060 2024-04-26T20:56:55.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.429+0000 7fbc8e800640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fbc8806f510 con 0x7fbc881610c0 2024-04-26T20:56:55.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.429+0000 7fbc8e800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fbc88146440 con 0x7fbc88146670 2024-04-26T20:56:55.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.429+0000 7fbc8ca00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbc88146670 0x7fbc88071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:55.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.429+0000 7fbc8de00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fbc881610c0 0x7fbc880cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:55.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.430+0000 7fbc8ca00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbc88146670 0x7fbc88071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.28:59012/0 (socket says 172.21.15.28:59012) 2024-04-26T20:56:55.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.430+0000 7fbc8ca00640 1 -- 172.21.15.28:0/3371794778 learned_addr learned my addr 172.21.15.28:0/3371794778 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:55.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.430+0000 7fbc8ca00640 1 -- 172.21.15.28:0/3371794778 >> v2:172.21.15.103:3300/0 conn(0x7fbc881610c0 msgr2=0x7fbc880cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:55.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.430+0000 7fbc8ca00640 1 --2- 172.21.15.28:0/3371794778 >> v2:172.21.15.103:3300/0 conn(0x7fbc881610c0 0x7fbc880cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:55.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.430+0000 7fbc8ca00640 1 -- 172.21.15.28:0/3371794778 >> v1:172.21.15.103:6789/0 conn(0x7fbc88146060 legacy=0x7fbc881609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:55.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.430+0000 7fbc8ca00640 1 -- 172.21.15.28:0/3371794778 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8806f010 con 0x7fbc88146670 2024-04-26T20:56:55.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.431+0000 7fbc8ca00640 1 --2- 172.21.15.28:0/3371794778 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbc88146670 0x7fbc88071e20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009870 tx=0x7fbc7402a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b02e13ec818b692 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:55.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.432+0000 7fbc87e00640 1 -- 172.21.15.28:0/3371794778 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74037070 con 0x7fbc88146670 2024-04-26T20:56:55.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.432+0000 7fbc87e00640 1 -- 172.21.15.28:0/3371794778 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc74032440 con 0x7fbc88146670 2024-04-26T20:56:55.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.432+0000 7fbc87e00640 1 -- 172.21.15.28:0/3371794778 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74032810 con 0x7fbc88146670 2024-04-26T20:56:55.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.433+0000 7fbc8e800640 1 -- 172.21.15.28:0/3371794778 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbc88146670 msgr2=0x7fbc88071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:55.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.433+0000 7fbc8e800640 1 --2- 172.21.15.28:0/3371794778 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbc88146670 0x7fbc88071e20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009870 tx=0x7fbc7402a1d0 comp rx=0 tx=0).stop 2024-04-26T20:56:55.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.433+0000 7fbc8e800640 1 -- 172.21.15.28:0/3371794778 shutdown_connections 2024-04-26T20:56:55.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.433+0000 7fbc8e800640 1 --2- 172.21.15.28:0/3371794778 >> v2:172.21.15.103:3300/0 conn(0x7fbc881610c0 0x7fbc880cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:55.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.433+0000 7fbc8e800640 1 --2- 172.21.15.28:0/3371794778 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbc88146670 0x7fbc88071e20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:55.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.433+0000 7fbc8e800640 1 -- 172.21.15.28:0/3371794778 >> 172.21.15.28:0/3371794778 conn(0x7fbc880d0a50 msgr2=0x7fbc880d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:55.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.433+0000 7fbc8e800640 1 -- 172.21.15.28:0/3371794778 shutdown_connections 2024-04-26T20:56:55.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.433+0000 7fbc8e800640 1 -- 172.21.15.28:0/3371794778 wait complete. 2024-04-26T20:56:55.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.435+0000 7fbc8e800640 1 Processor -- start 2024-04-26T20:56:55.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.436+0000 7fbc8e800640 1 -- start start 2024-04-26T20:56:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.437+0000 7fbc8e800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc88146060 0x7fbc880b4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.437+0000 7fbc8e800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbc88146670 0x7fbc8815d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.437+0000 7fbc8e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc881610c0 0x7fbc88143e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.437+0000 7fbc8e800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8806ff60 con 0x7fbc881610c0 2024-04-26T20:56:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.437+0000 7fbc8e800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8806fc80 con 0x7fbc88146060 2024-04-26T20:56:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.437+0000 7fbc8e800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fbc880cb5e0 con 0x7fbc88146670 2024-04-26T20:56:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.437+0000 7fbc8de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc881610c0 0x7fbc88143e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.437+0000 7fbc8de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc881610c0 0x7fbc88143e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42034/0 (socket says 172.21.15.28:42034) 2024-04-26T20:56:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.437+0000 7fbc8de00640 1 -- 172.21.15.28:0/3256734934 learned_addr learned my addr 172.21.15.28:0/3256734934 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.437+0000 7fbc8ca00640 1 --2- 172.21.15.28:0/3256734934 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbc88146670 0x7fbc8815d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.437+0000 7fbc8d400640 1 --2- 172.21.15.28:0/3256734934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc88146060 0x7fbc880b4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.438+0000 7fbc8de00640 1 -- 172.21.15.28:0/3256734934 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbc88146670 msgr2=0x7fbc8815d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:55.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.438+0000 7fbc8de00640 1 --2- 172.21.15.28:0/3256734934 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbc88146670 0x7fbc8815d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:55.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.438+0000 7fbc8de00640 1 -- 172.21.15.28:0/3256734934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc88146060 msgr2=0x7fbc880b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:55.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.438+0000 7fbc8de00640 1 --2- 172.21.15.28:0/3256734934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc88146060 0x7fbc880b4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:55.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.438+0000 7fbc8de00640 1 -- 172.21.15.28:0/3256734934 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc880b3b80 con 0x7fbc881610c0 2024-04-26T20:56:55.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.438+0000 7fbc8de00640 1 --2- 172.21.15.28:0/3256734934 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc881610c0 0x7fbc88143e20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009d30 tx=0x7fbc8000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:55.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.438+0000 7fbc85600640 1 -- 172.21.15.28:0/3256734934 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc80018020 con 0x7fbc881610c0 2024-04-26T20:56:55.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.438+0000 7fbc85600640 1 -- 172.21.15.28:0/3256734934 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc80014480 con 0x7fbc881610c0 2024-04-26T20:56:55.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.438+0000 7fbc8e800640 1 -- 172.21.15.28:0/3256734934 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc880b3e60 con 0x7fbc881610c0 2024-04-26T20:56:55.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.438+0000 7fbc85600640 1 -- 172.21.15.28:0/3256734934 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc80014870 con 0x7fbc881610c0 2024-04-26T20:56:55.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.438+0000 7fbc8e800640 1 -- 172.21.15.28:0/3256734934 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc880b4420 con 0x7fbc881610c0 2024-04-26T20:56:55.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.440+0000 7fbc8e800640 1 -- 172.21.15.28:0/3256734934 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc8806a360 con 0x7fbc881610c0 2024-04-26T20:56:55.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.441+0000 7fbc85600640 1 -- 172.21.15.28:0/3256734934 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc80005d10 con 0x7fbc881610c0 2024-04-26T20:56:55.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.442+0000 7fbc85600640 1 --2- 172.21.15.28:0/3256734934 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbc6806c8f0 0x7fbc6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:55.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.442+0000 7fbc85600640 1 -- 172.21.15.28:0/3256734934 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbc8008c710 con 0x7fbc881610c0 2024-04-26T20:56:55.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.442+0000 7fbc8d400640 1 --2- 172.21.15.28:0/3256734934 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbc6806c8f0 0x7fbc6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:55.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.442+0000 7fbc8d400640 1 --2- 172.21.15.28:0/3256734934 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbc6806c8f0 0x7fbc6806edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c001640 tx=0x7fbc7c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:55.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.446+0000 7fbc85600640 1 -- 172.21.15.28:0/3256734934 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbc80022070 con 0x7fbc881610c0 2024-04-26T20:56:55.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.648+0000 7fbc8e800640 1 -- 172.21.15.28:0/3256734934 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc8815fd00 con 0x7fbc881610c0 2024-04-26T20:56:55.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.650+0000 7fbc85600640 1 -- 172.21.15.28:0/3256734934 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7fbc80010040 con 0x7fbc881610c0 2024-04-26T20:56:55.652 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:56:55.652 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":21,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:56:52.699663+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:56:55.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.653+0000 7fbc8e800640 1 -- 172.21.15.28:0/3256734934 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbc6806c8f0 msgr2=0x7fbc6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:55.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.653+0000 7fbc8e800640 1 --2- 172.21.15.28:0/3256734934 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbc6806c8f0 0x7fbc6806edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c001640 tx=0x7fbc7c009040 comp rx=0 tx=0).stop 2024-04-26T20:56:55.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.653+0000 7fbc8e800640 1 -- 172.21.15.28:0/3256734934 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc881610c0 msgr2=0x7fbc88143e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:55.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.653+0000 7fbc8e800640 1 --2- 172.21.15.28:0/3256734934 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc881610c0 0x7fbc88143e20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbc80009d30 tx=0x7fbc8000c490 comp rx=0 tx=0).stop 2024-04-26T20:56:55.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.653+0000 7fbc8e800640 1 -- 172.21.15.28:0/3256734934 shutdown_connections 2024-04-26T20:56:55.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.653+0000 7fbc8e800640 1 --2- 172.21.15.28:0/3256734934 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbc6806c8f0 0x7fbc6806edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:55.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.653+0000 7fbc8e800640 1 --2- 172.21.15.28:0/3256734934 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbc881610c0 0x7fbc88143e20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:55.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.653+0000 7fbc8e800640 1 --2- 172.21.15.28:0/3256734934 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbc88146670 0x7fbc8815d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:55.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.653+0000 7fbc8e800640 1 --2- 172.21.15.28:0/3256734934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc88146060 0x7fbc880b4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:55.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.653+0000 7fbc8e800640 1 -- 172.21.15.28:0/3256734934 >> 172.21.15.28:0/3256734934 conn(0x7fbc880d0a50 msgr2=0x7fbc8806e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:55.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.654+0000 7fbc8e800640 1 -- 172.21.15.28:0/3256734934 shutdown_connections 2024-04-26T20:56:55.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.654+0000 7fbc8e800640 1 -- 172.21.15.28:0/3256734934 wait complete. 2024-04-26T20:56:55.672 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T20:56:55.672 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:56:55.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.872+0000 7f6861e00640 1 Processor -- start 2024-04-26T20:56:55.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.873+0000 7f6861e00640 1 -- start start 2024-04-26T20:56:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.874+0000 7f6861e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f685c146670 0x7f685c071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:55.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.874+0000 7f6861e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f685c161530 con 0x7f685c146060 2024-04-26T20:56:55.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.874+0000 7f6861e00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f685c146440 con 0x7f685c1610c0 2024-04-26T20:56:55.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.874+0000 7f6861e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f685c06f510 con 0x7f685c146670 2024-04-26T20:56:55.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.875+0000 7f6860a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f685c146060 0x7f685c1609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57690/0 (socket says 172.21.15.28:57690) 2024-04-26T20:56:55.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.875+0000 7f6860a00640 1 -- 172.21.15.28:0/2117527934 learned_addr learned my addr 172.21.15.28:0/2117527934 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:55.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.875+0000 7f685be00640 1 --2- 172.21.15.28:0/2117527934 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f685c146670 0x7f685c071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:55.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.875+0000 7f685b400640 1 -- 172.21.15.28:0/2117527934 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4290864728 0 0) 0x7f685c161530 con 0x7f685c146060 2024-04-26T20:56:55.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.875+0000 7f685b400640 1 -- 172.21.15.28:0/2117527934 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6840003520 con 0x7f685c146060 2024-04-26T20:56:55.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.876+0000 7f685b400640 1 -- 172.21.15.28:0/2117527934 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4149033169 0 0) 0x7f6840003520 con 0x7f685c146060 2024-04-26T20:56:55.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.876+0000 7f685b400640 1 -- 172.21.15.28:0/2117527934 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f685c146670 msgr2=0x7f685c071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:55.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.876+0000 7f685b400640 1 --2- 172.21.15.28:0/2117527934 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f685c146670 0x7f685c071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:55.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.876+0000 7f685b400640 1 -- 172.21.15.28:0/2117527934 >> v1:172.21.15.103:6789/0 conn(0x7f685c1610c0 legacy=0x7f685c0cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:55.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.876+0000 7f685b400640 1 -- 172.21.15.28:0/2117527934 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685c06f010 con 0x7f685c146060 2024-04-26T20:56:55.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.876+0000 7f685b400640 1 -- 172.21.15.28:0/2117527934 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f684c001670 con 0x7f685c146060 2024-04-26T20:56:55.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.876+0000 7f685b400640 1 -- 172.21.15.28:0/2117527934 >> v1:172.21.15.28:6789/0 conn(0x7f685c146060 legacy=0x7f685c1609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:55.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.876+0000 7f685b400640 1 --2- 172.21.15.28:0/2117527934 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f685c146670 0x7f68400039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:55.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.876+0000 7f685b400640 1 --2- 172.21.15.28:0/2117527934 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6840003f20 0x7f6840004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:55.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.876+0000 7f685b400640 1 --2- 172.21.15.28:0/2117527934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6840004d40 0x7f68400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:55.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.876+0000 7f685b400640 1 -- 172.21.15.28:0/2117527934 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f685c06f510 con 0x7f6840003f20 2024-04-26T20:56:55.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.876+0000 7f685b400640 1 -- 172.21.15.28:0/2117527934 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f684000bb00 con 0x7f6840004d40 2024-04-26T20:56:55.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.876+0000 7f685b400640 1 -- 172.21.15.28:0/2117527934 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f684000bd00 con 0x7f685c146670 2024-04-26T20:56:55.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.876+0000 7f685be00640 1 --2- 172.21.15.28:0/2117527934 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6840003f20 0x7f6840004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:55.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.876+0000 7f6861400640 1 --2- 172.21.15.28:0/2117527934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6840004d40 0x7f68400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:55.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.876+0000 7f6860a00640 1 --2- 172.21.15.28:0/2117527934 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f685c146670 0x7f68400039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:55.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.876+0000 7f685be00640 1 -- 172.21.15.28:0/2117527934 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f685c146670 msgr2=0x7f68400039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:55.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.877+0000 7f685be00640 1 --2- 172.21.15.28:0/2117527934 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f685c146670 0x7f68400039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:55.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.877+0000 7f685be00640 1 -- 172.21.15.28:0/2117527934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6840004d40 msgr2=0x7f68400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:55.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.877+0000 7f685be00640 1 --2- 172.21.15.28:0/2117527934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6840004d40 0x7f68400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:55.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.877+0000 7f685be00640 1 -- 172.21.15.28:0/2117527934 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6840003520 con 0x7f6840003f20 2024-04-26T20:56:55.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.877+0000 7f6860a00640 1 --2- 172.21.15.28:0/2117527934 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f685c146670 0x7f68400039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:56:55.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.877+0000 7f6861400640 1 --2- 172.21.15.28:0/2117527934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6840004d40 0x7f68400051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:56:55.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.878+0000 7f685be00640 1 --2- 172.21.15.28:0/2117527934 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6840003f20 0x7f6840004800 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6848009ac0 tx=0x7f6840005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac810285e1c31fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:55.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.878+0000 7f685b400640 1 -- 172.21.15.28:0/2117527934 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848036070 con 0x7f6840003f20 2024-04-26T20:56:55.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.878+0000 7f685b400640 1 -- 172.21.15.28:0/2117527934 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6848031400 con 0x7f6840003f20 2024-04-26T20:56:55.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.878+0000 7f685b400640 1 -- 172.21.15.28:0/2117527934 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848009690 con 0x7f6840003f20 2024-04-26T20:56:55.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.878+0000 7f6861e00640 1 -- 172.21.15.28:0/2117527934 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6840003f20 msgr2=0x7f6840004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:55.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.878+0000 7f6861e00640 1 --2- 172.21.15.28:0/2117527934 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6840003f20 0x7f6840004800 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6848009ac0 tx=0x7f6840005830 comp rx=0 tx=0).stop 2024-04-26T20:56:55.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.878+0000 7f6861400640 1 -- 172.21.15.28:0/2117527934 reap_dead start 2024-04-26T20:56:55.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.879+0000 7f6861e00640 1 -- 172.21.15.28:0/2117527934 shutdown_connections 2024-04-26T20:56:55.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.879+0000 7f6861e00640 1 -- 172.21.15.28:0/2117527934 >> 172.21.15.28:0/2117527934 conn(0x7f685c0d0a50 msgr2=0x7f685c0d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:55.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.879+0000 7f6861e00640 1 -- 172.21.15.28:0/2117527934 shutdown_connections 2024-04-26T20:56:55.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.879+0000 7f6861e00640 1 -- 172.21.15.28:0/2117527934 wait complete. 2024-04-26T20:56:55.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.880+0000 7f6861e00640 1 Processor -- start 2024-04-26T20:56:55.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.881+0000 7f6861e00640 1 -- start start 2024-04-26T20:56:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.882+0000 7f6861e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6840003f20 0x7f685c13bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.882+0000 7f6861e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f685c15e450 0x7f685c15e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.882+0000 7f6861e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f685c0cee60 0x7f685c0cf310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.882+0000 7f6861e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f685c06ff60 con 0x7f6840003f20 2024-04-26T20:56:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.882+0000 7f6861e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f685c06fc80 con 0x7f685c15e450 2024-04-26T20:56:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.882+0000 7f6861e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f685c071e20 con 0x7f685c0cee60 2024-04-26T20:56:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.882+0000 7f6860a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6840003f20 0x7f685c13bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.882+0000 7f6860a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6840003f20 0x7f685c13bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42042/0 (socket says 172.21.15.28:42042) 2024-04-26T20:56:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.882+0000 7f6860a00640 1 -- 172.21.15.28:0/1203099995 learned_addr learned my addr 172.21.15.28:0/1203099995 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.882+0000 7f685be00640 1 --2- 172.21.15.28:0/1203099995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f685c15e450 0x7f685c15e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.882+0000 7f6861400640 1 --2- 172.21.15.28:0/1203099995 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f685c0cee60 0x7f685c0cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.882+0000 7f6860a00640 1 -- 172.21.15.28:0/1203099995 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f685c0cee60 msgr2=0x7f685c0cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.882+0000 7f6860a00640 1 --2- 172.21.15.28:0/1203099995 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f685c0cee60 0x7f685c0cf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.882+0000 7f6860a00640 1 -- 172.21.15.28:0/1203099995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f685c15e450 msgr2=0x7f685c15e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.882+0000 7f6860a00640 1 --2- 172.21.15.28:0/1203099995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f685c15e450 0x7f685c15e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:55.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.882+0000 7f6860a00640 1 -- 172.21.15.28:0/1203099995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685c0b3ad0 con 0x7f6840003f20 2024-04-26T20:56:55.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.883+0000 7f6860a00640 1 --2- 172.21.15.28:0/1203099995 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6840003f20 0x7f685c13bfe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f684c001200 tx=0x7f684c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:55.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.883+0000 7f6858c00640 1 -- 172.21.15.28:0/1203099995 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c019070 con 0x7f6840003f20 2024-04-26T20:56:55.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.883+0000 7f6858c00640 1 -- 172.21.15.28:0/1203099995 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f684c0074e0 con 0x7f6840003f20 2024-04-26T20:56:55.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.883+0000 7f6858c00640 1 -- 172.21.15.28:0/1203099995 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c0049e0 con 0x7f6840003f20 2024-04-26T20:56:55.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.883+0000 7f6861e00640 1 -- 172.21.15.28:0/1203099995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685c0b3db0 con 0x7f6840003f20 2024-04-26T20:56:55.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.883+0000 7f6861e00640 1 -- 172.21.15.28:0/1203099995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f685c0b4370 con 0x7f6840003f20 2024-04-26T20:56:55.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.884+0000 7f6861e00640 1 -- 172.21.15.28:0/1203099995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f685c06a360 con 0x7f6840003f20 2024-04-26T20:56:55.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.885+0000 7f6858c00640 1 -- 172.21.15.28:0/1203099995 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f684c004b80 con 0x7f6840003f20 2024-04-26T20:56:55.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.886+0000 7f6858c00640 1 --2- 172.21.15.28:0/1203099995 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f683c06c8f0 0x7f683c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:55.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.886+0000 7f6858c00640 1 -- 172.21.15.28:0/1203099995 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f684c08b600 con 0x7f6840003f20 2024-04-26T20:56:55.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.886+0000 7f685be00640 1 --2- 172.21.15.28:0/1203099995 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f683c06c8f0 0x7f683c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:55.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.887+0000 7f685be00640 1 --2- 172.21.15.28:0/1203099995 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f683c06c8f0 0x7f683c06edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6848033000 tx=0x7f6848008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:55.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:55.891+0000 7f6858c00640 1 -- 172.21.15.28:0/1203099995 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f684c059a20 con 0x7f6840003f20 2024-04-26T20:56:56.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.089+0000 7f6861e00640 1 -- 172.21.15.28:0/1203099995 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f685c0ccb70 con 0x7f683c06c8f0 2024-04-26T20:56:56.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.112+0000 7f6858c00640 1 -- 172.21.15.28:0/1203099995 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f685c0ccb70 con 0x7f683c06c8f0 2024-04-26T20:56:56.116 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:56:56.117 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T20:56:56.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.117+0000 7f6861e00640 1 -- 172.21.15.28:0/1203099995 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f683c06c8f0 msgr2=0x7f683c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:56.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.117+0000 7f6861e00640 1 --2- 172.21.15.28:0/1203099995 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f683c06c8f0 0x7f683c06edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6848033000 tx=0x7f6848008030 comp rx=0 tx=0).stop 2024-04-26T20:56:56.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.117+0000 7f6861e00640 1 -- 172.21.15.28:0/1203099995 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6840003f20 msgr2=0x7f685c13bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:56.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.117+0000 7f6861e00640 1 --2- 172.21.15.28:0/1203099995 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6840003f20 0x7f685c13bfe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f684c001200 tx=0x7f684c00dc90 comp rx=0 tx=0).stop 2024-04-26T20:56:56.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.117+0000 7f6861e00640 1 -- 172.21.15.28:0/1203099995 shutdown_connections 2024-04-26T20:56:56.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.117+0000 7f6861e00640 1 --2- 172.21.15.28:0/1203099995 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f683c06c8f0 0x7f683c06edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:56.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.117+0000 7f6861e00640 1 --2- 172.21.15.28:0/1203099995 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f685c0cee60 0x7f685c0cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:56.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.117+0000 7f6861e00640 1 --2- 172.21.15.28:0/1203099995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f685c15e450 0x7f685c15e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:56.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.117+0000 7f6861e00640 1 --2- 172.21.15.28:0/1203099995 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6840003f20 0x7f685c13bfe0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:56.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.117+0000 7f6861e00640 1 -- 172.21.15.28:0/1203099995 >> 172.21.15.28:0/1203099995 conn(0x7f685c0d0a50 msgr2=0x7f685c068090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:56.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.117+0000 7f6861e00640 1 -- 172.21.15.28:0/1203099995 shutdown_connections 2024-04-26T20:56:56.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.117+0000 7f6861e00640 1 -- 172.21.15.28:0/1203099995 wait complete. 2024-04-26T20:56:56.136 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-26T20:56:54.701536+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235928,"kb_used_data":2896,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738792,"statfs":{"total":773094113280,"available":772852523008,"internally_reserved":0,"allocated":2965504,"data_stored":1777796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.059007"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153718+0000","last_change":"2024-04-26T20:56:51.731207+0000","last_active":"2024-04-26T20:56:53.153718+0000","last_peered":"2024-04-26T20:56:53.153718+0000","last_clean":"2024-04-26T20:56:53.153718+0000","last_became_active":"2024-04-26T20:56:51.730930+0000","last_became_peered":"2024-04-26T20:56:51.730930+0000","last_unstale":"2024-04-26T20:56:53.153718+0000","last_undegraded":"2024-04-26T20:56:53.153718+0000","last_fullsized":"2024-04-26T20:56:53.153718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:06.270681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.714391+0000","last_change":"2024-04-26T20:56:51.719125+0000","last_active":"2024-04-26T20:56:52.714391+0000","last_peered":"2024-04-26T20:56:52.714391+0000","last_clean":"2024-04-26T20:56:52.714391+0000","last_became_active":"2024-04-26T20:56:51.717685+0000","last_became_peered":"2024-04-26T20:56:51.717685+0000","last_unstale":"2024-04-26T20:56:52.714391+0000","last_undegraded":"2024-04-26T20:56:52.714391+0000","last_fullsized":"2024-04-26T20:56:52.714391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.428432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144729+0000","last_change":"2024-04-26T20:56:51.717024+0000","last_active":"2024-04-26T20:56:53.144729+0000","last_peered":"2024-04-26T20:56:53.144729+0000","last_clean":"2024-04-26T20:56:53.144729+0000","last_became_active":"2024-04-26T20:56:51.716486+0000","last_became_peered":"2024-04-26T20:56:51.716486+0000","last_unstale":"2024-04-26T20:56:53.144729+0000","last_undegraded":"2024-04-26T20:56:53.144729+0000","last_fullsized":"2024-04-26T20:56:53.144729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:01.617412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716816+0000","last_change":"2024-04-26T20:56:51.716770+0000","last_active":"2024-04-26T20:56:52.716816+0000","last_peered":"2024-04-26T20:56:52.716816+0000","last_clean":"2024-04-26T20:56:52.716816+0000","last_became_active":"2024-04-26T20:56:51.716443+0000","last_became_peered":"2024-04-26T20:56:51.716443+0000","last_unstale":"2024-04-26T20:56:52.716816+0000","last_undegraded":"2024-04-26T20:56:52.716816+0000","last_fullsized":"2024-04-26T20:56:52.716816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:08.812637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153115+0000","last_change":"2024-04-26T20:56:51.735130+0000","last_active":"2024-04-26T20:56:53.153115+0000","last_peered":"2024-04-26T20:56:53.153115+0000","last_clean":"2024-04-26T20:56:53.153115+0000","last_became_active":"2024-04-26T20:56:51.734775+0000","last_became_peered":"2024-04-26T20:56:51.734775+0000","last_unstale":"2024-04-26T20:56:53.153115+0000","last_undegraded":"2024-04-26T20:56:53.153115+0000","last_fullsized":"2024-04-26T20:56:53.153115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:08.812637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716491+0000","last_change":"2024-04-26T20:56:51.719621+0000","last_active":"2024-04-26T20:56:52.716491+0000","last_peered":"2024-04-26T20:56:52.716491+0000","last_clean":"2024-04-26T20:56:52.716491+0000","last_became_active":"2024-04-26T20:56:51.719312+0000","last_became_peered":"2024-04-26T20:56:51.719312+0000","last_unstale":"2024-04-26T20:56:52.716491+0000","last_undegraded":"2024-04-26T20:56:52.716491+0000","last_fullsized":"2024-04-26T20:56:52.716491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:23.268496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.152560+0000","last_change":"2024-04-26T20:56:51.730495+0000","last_active":"2024-04-26T20:56:53.152560+0000","last_peered":"2024-04-26T20:56:53.152560+0000","last_clean":"2024-04-26T20:56:53.152560+0000","last_became_active":"2024-04-26T20:56:51.729796+0000","last_became_peered":"2024-04-26T20:56:51.729796+0000","last_unstale":"2024-04-26T20:56:53.152560+0000","last_undegraded":"2024-04-26T20:56:53.152560+0000","last_fullsized":"2024-04-26T20:56:53.152560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:52.912874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225874+0000","last_change":"2024-04-26T20:56:51.729824+0000","last_active":"2024-04-26T20:56:53.225874+0000","last_peered":"2024-04-26T20:56:53.225874+0000","last_clean":"2024-04-26T20:56:53.225874+0000","last_became_active":"2024-04-26T20:56:51.729546+0000","last_became_peered":"2024-04-26T20:56:51.729546+0000","last_unstale":"2024-04-26T20:56:53.225874+0000","last_undegraded":"2024-04-26T20:56:53.225874+0000","last_fullsized":"2024-04-26T20:56:53.225874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:18.631055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153639+0000","last_change":"2024-04-26T20:56:51.732556+0000","last_active":"2024-04-26T20:56:53.153639+0000","last_peered":"2024-04-26T20:56:53.153639+0000","last_clean":"2024-04-26T20:56:53.153639+0000","last_became_active":"2024-04-26T20:56:51.732298+0000","last_became_peered":"2024-04-26T20:56:51.732298+0000","last_unstale":"2024-04-26T20:56:53.153639+0000","last_undegraded":"2024-04-26T20:56:53.153639+0000","last_fullsized":"2024-04-26T20:56:53.153639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:07.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730718+0000","last_change":"2024-04-26T20:56:51.724747+0000","last_active":"2024-04-26T20:56:52.730718+0000","last_peered":"2024-04-26T20:56:52.730718+0000","last_clean":"2024-04-26T20:56:52.730718+0000","last_became_active":"2024-04-26T20:56:51.724452+0000","last_became_peered":"2024-04-26T20:56:51.724452+0000","last_unstale":"2024-04-26T20:56:52.730718+0000","last_undegraded":"2024-04-26T20:56:52.730718+0000","last_fullsized":"2024-04-26T20:56:52.730718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.428432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730341+0000","last_change":"2024-04-26T20:56:51.729732+0000","last_active":"2024-04-26T20:56:52.730341+0000","last_peered":"2024-04-26T20:56:52.730341+0000","last_clean":"2024-04-26T20:56:52.730341+0000","last_became_active":"2024-04-26T20:56:51.729464+0000","last_became_peered":"2024-04-26T20:56:51.729464+0000","last_unstale":"2024-04-26T20:56:52.730341+0000","last_undegraded":"2024-04-26T20:56:52.730341+0000","last_fullsized":"2024-04-26T20:56:52.730341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:37.778325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.715425+0000","last_change":"2024-04-26T20:56:51.723870+0000","last_active":"2024-04-26T20:56:52.715425+0000","last_peered":"2024-04-26T20:56:52.715425+0000","last_clean":"2024-04-26T20:56:52.715425+0000","last_became_active":"2024-04-26T20:56:51.722037+0000","last_became_peered":"2024-04-26T20:56:51.722037+0000","last_unstale":"2024-04-26T20:56:52.715425+0000","last_undegraded":"2024-04-26T20:56:52.715425+0000","last_fullsized":"2024-04-26T20:56:52.715425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:42.676745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226737+0000","last_change":"2024-04-26T20:56:51.719947+0000","last_active":"2024-04-26T20:56:53.226737+0000","last_peered":"2024-04-26T20:56:53.226737+0000","last_clean":"2024-04-26T20:56:53.226737+0000","last_became_active":"2024-04-26T20:56:51.719565+0000","last_became_peered":"2024-04-26T20:56:51.719565+0000","last_unstale":"2024-04-26T20:56:53.226737+0000","last_undegraded":"2024-04-26T20:56:53.226737+0000","last_fullsized":"2024-04-26T20:56:53.226737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:36.354765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144673+0000","last_change":"2024-04-26T20:56:51.725826+0000","last_active":"2024-04-26T20:56:53.144673+0000","last_peered":"2024-04-26T20:56:53.144673+0000","last_clean":"2024-04-26T20:56:53.144673+0000","last_became_active":"2024-04-26T20:56:51.723315+0000","last_became_peered":"2024-04-26T20:56:51.723315+0000","last_unstale":"2024-04-26T20:56:53.144673+0000","last_undegraded":"2024-04-26T20:56:53.144673+0000","last_fullsized":"2024-04-26T20:56:53.144673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:38:55.851891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713041+0000","last_change":"2024-04-26T20:56:51.704000+0000","last_active":"2024-04-26T20:56:52.713041+0000","last_peered":"2024-04-26T20:56:52.713041+0000","last_clean":"2024-04-26T20:56:52.713041+0000","last_became_active":"2024-04-26T20:56:51.703007+0000","last_became_peered":"2024-04-26T20:56:51.703007+0000","last_unstale":"2024-04-26T20:56:52.713041+0000","last_undegraded":"2024-04-26T20:56:52.713041+0000","last_fullsized":"2024-04-26T20:56:52.713041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:36.354765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716554+0000","last_change":"2024-04-26T20:56:51.724622+0000","last_active":"2024-04-26T20:56:52.716554+0000","last_peered":"2024-04-26T20:56:52.716554+0000","last_clean":"2024-04-26T20:56:52.716554+0000","last_became_active":"2024-04-26T20:56:51.723198+0000","last_became_peered":"2024-04-26T20:56:51.723198+0000","last_unstale":"2024-04-26T20:56:52.716554+0000","last_undegraded":"2024-04-26T20:56:52.716554+0000","last_fullsized":"2024-04-26T20:56:52.716554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:52.912874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.718883+0000","last_change":"2024-04-26T20:56:51.736813+0000","last_active":"2024-04-26T20:56:52.718883+0000","last_peered":"2024-04-26T20:56:52.718883+0000","last_clean":"2024-04-26T20:56:52.718883+0000","last_became_active":"2024-04-26T20:56:51.736532+0000","last_became_peered":"2024-04-26T20:56:51.736532+0000","last_unstale":"2024-04-26T20:56:52.718883+0000","last_undegraded":"2024-04-26T20:56:52.718883+0000","last_fullsized":"2024-04-26T20:56:52.718883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:08.679353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225324+0000","last_change":"2024-04-26T20:56:51.726234+0000","last_active":"2024-04-26T20:56:53.225324+0000","last_peered":"2024-04-26T20:56:53.225324+0000","last_clean":"2024-04-26T20:56:53.225324+0000","last_became_active":"2024-04-26T20:56:51.723289+0000","last_became_peered":"2024-04-26T20:56:51.723289+0000","last_unstale":"2024-04-26T20:56:53.225324+0000","last_undegraded":"2024-04-26T20:56:53.225324+0000","last_fullsized":"2024-04-26T20:56:53.225324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:12.708963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153878+0000","last_change":"2024-04-26T20:56:51.735058+0000","last_active":"2024-04-26T20:56:53.153878+0000","last_peered":"2024-04-26T20:56:53.153878+0000","last_clean":"2024-04-26T20:56:53.153878+0000","last_became_active":"2024-04-26T20:56:51.734516+0000","last_became_peered":"2024-04-26T20:56:51.734516+0000","last_unstale":"2024-04-26T20:56:53.153878+0000","last_undegraded":"2024-04-26T20:56:53.153878+0000","last_fullsized":"2024-04-26T20:56:53.153878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:33.950624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730309+0000","last_change":"2024-04-26T20:56:51.715023+0000","last_active":"2024-04-26T20:56:52.730309+0000","last_peered":"2024-04-26T20:56:52.730309+0000","last_clean":"2024-04-26T20:56:52.730309+0000","last_became_active":"2024-04-26T20:56:51.714250+0000","last_became_peered":"2024-04-26T20:56:51.714250+0000","last_unstale":"2024-04-26T20:56:52.730309+0000","last_undegraded":"2024-04-26T20:56:52.730309+0000","last_fullsized":"2024-04-26T20:56:52.730309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:24.679612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144998+0000","last_change":"2024-04-26T20:56:51.737492+0000","last_active":"2024-04-26T20:56:53.144998+0000","last_peered":"2024-04-26T20:56:53.144998+0000","last_clean":"2024-04-26T20:56:53.144998+0000","last_became_active":"2024-04-26T20:56:51.736774+0000","last_became_peered":"2024-04-26T20:56:51.736774+0000","last_unstale":"2024-04-26T20:56:53.144998+0000","last_undegraded":"2024-04-26T20:56:53.144998+0000","last_fullsized":"2024-04-26T20:56:53.144998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:36.354765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716674+0000","last_change":"2024-04-26T20:56:51.731749+0000","last_active":"2024-04-26T20:56:52.716674+0000","last_peered":"2024-04-26T20:56:52.716674+0000","last_clean":"2024-04-26T20:56:52.716674+0000","last_became_active":"2024-04-26T20:56:51.731460+0000","last_became_peered":"2024-04-26T20:56:51.731460+0000","last_unstale":"2024-04-26T20:56:52.716674+0000","last_undegraded":"2024-04-26T20:56:52.716674+0000","last_fullsized":"2024-04-26T20:56:52.716674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:47.662176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226704+0000","last_change":"2024-04-26T20:56:51.721856+0000","last_active":"2024-04-26T20:56:53.226704+0000","last_peered":"2024-04-26T20:56:53.226704+0000","last_clean":"2024-04-26T20:56:53.226704+0000","last_became_active":"2024-04-26T20:56:51.721531+0000","last_became_peered":"2024-04-26T20:56:51.721531+0000","last_unstale":"2024-04-26T20:56:53.226704+0000","last_undegraded":"2024-04-26T20:56:53.226704+0000","last_fullsized":"2024-04-26T20:56:53.226704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:53.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716438+0000","last_change":"2024-04-26T20:56:51.718492+0000","last_active":"2024-04-26T20:56:52.716438+0000","last_peered":"2024-04-26T20:56:52.716438+0000","last_clean":"2024-04-26T20:56:52.716438+0000","last_became_active":"2024-04-26T20:56:51.718127+0000","last_became_peered":"2024-04-26T20:56:51.718127+0000","last_unstale":"2024-04-26T20:56:52.716438+0000","last_undegraded":"2024-04-26T20:56:52.716438+0000","last_fullsized":"2024-04-26T20:56:52.716438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.428432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.227580+0000","last_change":"2024-04-26T20:56:51.722240+0000","last_active":"2024-04-26T20:56:53.227580+0000","last_peered":"2024-04-26T20:56:53.227580+0000","last_clean":"2024-04-26T20:56:53.227580+0000","last_became_active":"2024-04-26T20:56:51.721601+0000","last_became_peered":"2024-04-26T20:56:51.721601+0000","last_unstale":"2024-04-26T20:56:53.227580+0000","last_undegraded":"2024-04-26T20:56:53.227580+0000","last_fullsized":"2024-04-26T20:56:53.227580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:52.912874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730730+0000","last_change":"2024-04-26T20:56:51.722608+0000","last_active":"2024-04-26T20:56:52.730730+0000","last_peered":"2024-04-26T20:56:52.730730+0000","last_clean":"2024-04-26T20:56:52.730730+0000","last_became_active":"2024-04-26T20:56:51.722321+0000","last_became_peered":"2024-04-26T20:56:51.722321+0000","last_unstale":"2024-04-26T20:56:52.730730+0000","last_undegraded":"2024-04-26T20:56:52.730730+0000","last_fullsized":"2024-04-26T20:56:52.730730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:23.268496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713738+0000","last_change":"2024-04-26T20:56:51.715230+0000","last_active":"2024-04-26T20:56:52.713738+0000","last_peered":"2024-04-26T20:56:52.713738+0000","last_clean":"2024-04-26T20:56:52.713738+0000","last_became_active":"2024-04-26T20:56:51.714911+0000","last_became_peered":"2024-04-26T20:56:51.714911+0000","last_unstale":"2024-04-26T20:56:52.713738+0000","last_undegraded":"2024-04-26T20:56:52.713738+0000","last_fullsized":"2024-04-26T20:56:52.713738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:37.778325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.227039+0000","last_change":"2024-04-26T20:56:51.712010+0000","last_active":"2024-04-26T20:56:53.227039+0000","last_peered":"2024-04-26T20:56:53.227039+0000","last_clean":"2024-04-26T20:56:53.227039+0000","last_became_active":"2024-04-26T20:56:51.711583+0000","last_became_peered":"2024-04-26T20:56:51.711583+0000","last_unstale":"2024-04-26T20:56:53.227039+0000","last_undegraded":"2024-04-26T20:56:53.227039+0000","last_fullsized":"2024-04-26T20:56:53.227039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.428432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.145333+0000","last_change":"2024-04-26T20:56:51.722575+0000","last_active":"2024-04-26T20:56:53.145333+0000","last_peered":"2024-04-26T20:56:53.145333+0000","last_clean":"2024-04-26T20:56:53.145333+0000","last_became_active":"2024-04-26T20:56:51.722274+0000","last_became_peered":"2024-04-26T20:56:51.722274+0000","last_unstale":"2024-04-26T20:56:53.145333+0000","last_undegraded":"2024-04-26T20:56:53.145333+0000","last_fullsized":"2024-04-26T20:56:53.145333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.428432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716895+0000","last_change":"2024-04-26T20:56:51.712244+0000","last_active":"2024-04-26T20:56:52.716895+0000","last_peered":"2024-04-26T20:56:52.716895+0000","last_clean":"2024-04-26T20:56:52.716895+0000","last_became_active":"2024-04-26T20:56:51.711957+0000","last_became_peered":"2024-04-26T20:56:51.711957+0000","last_unstale":"2024-04-26T20:56:52.716895+0000","last_undegraded":"2024-04-26T20:56:52.716895+0000","last_fullsized":"2024-04-26T20:56:52.716895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:22.643252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225198+0000","last_change":"2024-04-26T20:56:51.721797+0000","last_active":"2024-04-26T20:56:53.225198+0000","last_peered":"2024-04-26T20:56:53.225198+0000","last_clean":"2024-04-26T20:56:53.225198+0000","last_became_active":"2024-04-26T20:56:51.721465+0000","last_became_peered":"2024-04-26T20:56:51.721465+0000","last_unstale":"2024-04-26T20:56:53.225198+0000","last_undegraded":"2024-04-26T20:56:53.225198+0000","last_fullsized":"2024-04-26T20:56:53.225198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:26.791119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.729925+0000","last_change":"2024-04-26T20:56:51.716527+0000","last_active":"2024-04-26T20:56:52.729925+0000","last_peered":"2024-04-26T20:56:52.729925+0000","last_clean":"2024-04-26T20:56:52.729925+0000","last_became_active":"2024-04-26T20:56:51.716103+0000","last_became_peered":"2024-04-26T20:56:51.716103+0000","last_unstale":"2024-04-26T20:56:52.729925+0000","last_undegraded":"2024-04-26T20:56:52.729925+0000","last_fullsized":"2024-04-26T20:56:52.729925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:53.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.714884+0000","last_change":"2024-04-26T20:56:51.713515+0000","last_active":"2024-04-26T20:56:52.714884+0000","last_peered":"2024-04-26T20:56:52.714884+0000","last_clean":"2024-04-26T20:56:52.714884+0000","last_became_active":"2024-04-26T20:56:51.710208+0000","last_became_peered":"2024-04-26T20:56:51.710208+0000","last_unstale":"2024-04-26T20:56:52.714884+0000","last_undegraded":"2024-04-26T20:56:52.714884+0000","last_fullsized":"2024-04-26T20:56:52.714884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:51.540923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226969+0000","last_change":"2024-04-26T20:56:50.692216+0000","last_active":"2024-04-26T20:56:53.226969+0000","last_peered":"2024-04-26T20:56:53.226969+0000","last_clean":"2024-04-26T20:56:53.226969+0000","last_became_active":"2024-04-26T20:56:50.691457+0000","last_became_peered":"2024-04-26T20:56:50.691457+0000","last_unstale":"2024-04-26T20:56:53.226969+0000","last_undegraded":"2024-04-26T20:56:53.226969+0000","last_fullsized":"2024-04-26T20:56:53.226969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:50.477216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.145297+0000","last_change":"2024-04-26T20:56:51.730925+0000","last_active":"2024-04-26T20:56:53.145297+0000","last_peered":"2024-04-26T20:56:53.145297+0000","last_clean":"2024-04-26T20:56:53.145297+0000","last_became_active":"2024-04-26T20:56:51.730497+0000","last_became_peered":"2024-04-26T20:56:51.730497+0000","last_unstale":"2024-04-26T20:56:53.145297+0000","last_undegraded":"2024-04-26T20:56:53.145297+0000","last_fullsized":"2024-04-26T20:56:53.145297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:37.778325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.729866+0000","last_change":"2024-04-26T20:56:50.684303+0000","last_active":"2024-04-26T20:56:52.729866+0000","last_peered":"2024-04-26T20:56:52.729866+0000","last_clean":"2024-04-26T20:56:52.729866+0000","last_became_active":"2024-04-26T20:56:50.684006+0000","last_became_peered":"2024-04-26T20:56:50.684006+0000","last_unstale":"2024-04-26T20:56:52.729866+0000","last_undegraded":"2024-04-26T20:56:52.729866+0000","last_fullsized":"2024-04-26T20:56:52.729866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:50.526509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226111+0000","last_change":"2024-04-26T20:56:50.688365+0000","last_active":"2024-04-26T20:56:53.226111+0000","last_peered":"2024-04-26T20:56:53.226111+0000","last_clean":"2024-04-26T20:56:53.226111+0000","last_became_active":"2024-04-26T20:56:50.688034+0000","last_became_peered":"2024-04-26T20:56:50.688034+0000","last_unstale":"2024-04-26T20:56:53.226111+0000","last_undegraded":"2024-04-26T20:56:53.226111+0000","last_fullsized":"2024-04-26T20:56:53.226111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:14.594594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226190+0000","last_change":"2024-04-26T20:56:51.721954+0000","last_active":"2024-04-26T20:56:53.226190+0000","last_peered":"2024-04-26T20:56:53.226190+0000","last_clean":"2024-04-26T20:56:53.226190+0000","last_became_active":"2024-04-26T20:56:51.721650+0000","last_became_peered":"2024-04-26T20:56:51.721650+0000","last_unstale":"2024-04-26T20:56:53.226190+0000","last_undegraded":"2024-04-26T20:56:53.226190+0000","last_fullsized":"2024-04-26T20:56:53.226190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:01.617412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716559+0000","last_change":"2024-04-26T20:56:50.680262+0000","last_active":"2024-04-26T20:56:52.716559+0000","last_peered":"2024-04-26T20:56:52.716559+0000","last_clean":"2024-04-26T20:56:52.716559+0000","last_became_active":"2024-04-26T20:56:50.675501+0000","last_became_peered":"2024-04-26T20:56:50.675501+0000","last_unstale":"2024-04-26T20:56:52.716559+0000","last_undegraded":"2024-04-26T20:56:52.716559+0000","last_fullsized":"2024-04-26T20:56:52.716559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:31.999414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713009+0000","last_change":"2024-04-26T20:56:51.714659+0000","last_active":"2024-04-26T20:56:52.713009+0000","last_peered":"2024-04-26T20:56:52.713009+0000","last_clean":"2024-04-26T20:56:52.713009+0000","last_became_active":"2024-04-26T20:56:51.714340+0000","last_became_peered":"2024-04-26T20:56:51.714340+0000","last_unstale":"2024-04-26T20:56:52.713009+0000","last_undegraded":"2024-04-26T20:56:52.713009+0000","last_fullsized":"2024-04-26T20:56:52.713009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:53.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730521+0000","last_change":"2024-04-26T20:56:50.689246+0000","last_active":"2024-04-26T20:56:52.730521+0000","last_peered":"2024-04-26T20:56:52.730521+0000","last_clean":"2024-04-26T20:56:52.730521+0000","last_became_active":"2024-04-26T20:56:50.688882+0000","last_became_peered":"2024-04-26T20:56:50.688882+0000","last_unstale":"2024-04-26T20:56:52.730521+0000","last_undegraded":"2024-04-26T20:56:52.730521+0000","last_fullsized":"2024-04-26T20:56:52.730521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:39.328210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.718004+0000","last_change":"2024-04-26T20:56:51.737626+0000","last_active":"2024-04-26T20:56:52.718004+0000","last_peered":"2024-04-26T20:56:52.718004+0000","last_clean":"2024-04-26T20:56:52.718004+0000","last_became_active":"2024-04-26T20:56:51.737273+0000","last_became_peered":"2024-04-26T20:56:51.737273+0000","last_unstale":"2024-04-26T20:56:52.718004+0000","last_undegraded":"2024-04-26T20:56:52.718004+0000","last_fullsized":"2024-04-26T20:56:52.718004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:33.950624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.718371+0000","last_change":"2024-04-26T20:56:50.690573+0000","last_active":"2024-04-26T20:56:52.718371+0000","last_peered":"2024-04-26T20:56:52.718371+0000","last_clean":"2024-04-26T20:56:52.718371+0000","last_became_active":"2024-04-26T20:56:50.690198+0000","last_became_peered":"2024-04-26T20:56:50.690198+0000","last_unstale":"2024-04-26T20:56:52.718371+0000","last_undegraded":"2024-04-26T20:56:52.718371+0000","last_fullsized":"2024-04-26T20:56:52.718371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:25.776704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144843+0000","last_change":"2024-04-26T20:56:51.733136+0000","last_active":"2024-04-26T20:56:53.144843+0000","last_peered":"2024-04-26T20:56:53.144843+0000","last_clean":"2024-04-26T20:56:53.144843+0000","last_became_active":"2024-04-26T20:56:51.732782+0000","last_became_peered":"2024-04-26T20:56:51.732782+0000","last_unstale":"2024-04-26T20:56:53.144843+0000","last_undegraded":"2024-04-26T20:56:53.144843+0000","last_fullsized":"2024-04-26T20:56:53.144843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:16.275518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713411+0000","last_change":"2024-04-26T20:56:50.689210+0000","last_active":"2024-04-26T20:56:52.713411+0000","last_peered":"2024-04-26T20:56:52.713411+0000","last_clean":"2024-04-26T20:56:52.713411+0000","last_became_active":"2024-04-26T20:56:50.688655+0000","last_became_peered":"2024-04-26T20:56:50.688655+0000","last_unstale":"2024-04-26T20:56:52.713411+0000","last_undegraded":"2024-04-26T20:56:52.713411+0000","last_fullsized":"2024-04-26T20:56:52.713411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:25.776704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.715628+0000","last_change":"2024-04-26T20:56:51.737157+0000","last_active":"2024-04-26T20:56:52.715628+0000","last_peered":"2024-04-26T20:56:52.715628+0000","last_clean":"2024-04-26T20:56:52.715628+0000","last_became_active":"2024-04-26T20:56:51.736082+0000","last_became_peered":"2024-04-26T20:56:51.736082+0000","last_unstale":"2024-04-26T20:56:52.715628+0000","last_undegraded":"2024-04-26T20:56:52.715628+0000","last_fullsized":"2024-04-26T20:56:52.715628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:40.342624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730295+0000","last_change":"2024-04-26T20:56:50.695225+0000","last_active":"2024-04-26T20:56:52.730295+0000","last_peered":"2024-04-26T20:56:52.730295+0000","last_clean":"2024-04-26T20:56:52.730295+0000","last_became_active":"2024-04-26T20:56:50.694892+0000","last_became_peered":"2024-04-26T20:56:50.694892+0000","last_unstale":"2024-04-26T20:56:52.730295+0000","last_undegraded":"2024-04-26T20:56:52.730295+0000","last_fullsized":"2024-04-26T20:56:52.730295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:41.662331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153191+0000","last_change":"2024-04-26T20:56:51.722030+0000","last_active":"2024-04-26T20:56:53.153191+0000","last_peered":"2024-04-26T20:56:53.153191+0000","last_clean":"2024-04-26T20:56:53.153191+0000","last_became_active":"2024-04-26T20:56:51.721724+0000","last_became_peered":"2024-04-26T20:56:51.721724+0000","last_unstale":"2024-04-26T20:56:53.153191+0000","last_undegraded":"2024-04-26T20:56:53.153191+0000","last_fullsized":"2024-04-26T20:56:53.153191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:52.330678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.145449+0000","last_change":"2024-04-26T20:56:50.682167+0000","last_active":"2024-04-26T20:56:53.145449+0000","last_peered":"2024-04-26T20:56:53.145449+0000","last_clean":"2024-04-26T20:56:53.145449+0000","last_became_active":"2024-04-26T20:56:50.675090+0000","last_became_peered":"2024-04-26T20:56:50.675090+0000","last_unstale":"2024-04-26T20:56:53.145449+0000","last_undegraded":"2024-04-26T20:56:53.145449+0000","last_fullsized":"2024-04-26T20:56:53.145449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:03.566375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.145134+0000","last_change":"2024-04-26T20:56:51.736362+0000","last_active":"2024-04-26T20:56:53.145134+0000","last_peered":"2024-04-26T20:56:53.145134+0000","last_clean":"2024-04-26T20:56:53.145134+0000","last_became_active":"2024-04-26T20:56:51.736050+0000","last_became_peered":"2024-04-26T20:56:51.736050+0000","last_unstale":"2024-04-26T20:56:53.145134+0000","last_undegraded":"2024-04-26T20:56:53.145134+0000","last_fullsized":"2024-04-26T20:56:53.145134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:24.679612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717056+0000","last_change":"2024-04-26T20:56:50.683775+0000","last_active":"2024-04-26T20:56:52.717056+0000","last_peered":"2024-04-26T20:56:52.717056+0000","last_clean":"2024-04-26T20:56:52.717056+0000","last_became_active":"2024-04-26T20:56:50.683470+0000","last_became_peered":"2024-04-26T20:56:50.683470+0000","last_unstale":"2024-04-26T20:56:52.717056+0000","last_undegraded":"2024-04-26T20:56:52.717056+0000","last_fullsized":"2024-04-26T20:56:52.717056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:50.526509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153154+0000","last_change":"2024-04-26T20:56:51.730365+0000","last_active":"2024-04-26T20:56:53.153154+0000","last_peered":"2024-04-26T20:56:53.153154+0000","last_clean":"2024-04-26T20:56:53.153154+0000","last_became_active":"2024-04-26T20:56:51.730015+0000","last_became_peered":"2024-04-26T20:56:51.730015+0000","last_unstale":"2024-04-26T20:56:53.153154+0000","last_undegraded":"2024-04-26T20:56:53.153154+0000","last_fullsized":"2024-04-26T20:56:53.153154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:22.643252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.729839+0000","last_change":"2024-04-26T20:56:50.686055+0000","last_active":"2024-04-26T20:56:52.729839+0000","last_peered":"2024-04-26T20:56:52.729839+0000","last_clean":"2024-04-26T20:56:52.729839+0000","last_became_active":"2024-04-26T20:56:50.685771+0000","last_became_peered":"2024-04-26T20:56:50.685771+0000","last_unstale":"2024-04-26T20:56:52.729839+0000","last_undegraded":"2024-04-26T20:56:52.729839+0000","last_fullsized":"2024-04-26T20:56:52.729839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:31.999414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.729780+0000","last_change":"2024-04-26T20:56:51.719273+0000","last_active":"2024-04-26T20:56:52.729780+0000","last_peered":"2024-04-26T20:56:52.729780+0000","last_clean":"2024-04-26T20:56:52.729780+0000","last_became_active":"2024-04-26T20:56:51.718659+0000","last_became_peered":"2024-04-26T20:56:51.718659+0000","last_unstale":"2024-04-26T20:56:52.729780+0000","last_undegraded":"2024-04-26T20:56:52.729780+0000","last_fullsized":"2024-04-26T20:56:52.729780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:16.275518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225397+0000","last_change":"2024-04-26T20:56:50.686714+0000","last_active":"2024-04-26T20:56:53.225397+0000","last_peered":"2024-04-26T20:56:53.225397+0000","last_clean":"2024-04-26T20:56:53.225397+0000","last_became_active":"2024-04-26T20:56:50.686348+0000","last_became_peered":"2024-04-26T20:56:50.686348+0000","last_unstale":"2024-04-26T20:56:53.225397+0000","last_undegraded":"2024-04-26T20:56:53.225397+0000","last_fullsized":"2024-04-26T20:56:53.225397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:14.594594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.152873+0000","last_change":"2024-04-26T20:56:51.730639+0000","last_active":"2024-04-26T20:56:53.152873+0000","last_peered":"2024-04-26T20:56:53.152873+0000","last_clean":"2024-04-26T20:56:53.152873+0000","last_became_active":"2024-04-26T20:56:51.730196+0000","last_became_peered":"2024-04-26T20:56:51.730196+0000","last_unstale":"2024-04-26T20:56:53.152873+0000","last_undegraded":"2024-04-26T20:56:53.152873+0000","last_fullsized":"2024-04-26T20:56:53.152873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:47.662176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.715683+0000","last_change":"2024-04-26T20:56:50.685016+0000","last_active":"2024-04-26T20:56:52.715683+0000","last_peered":"2024-04-26T20:56:52.715683+0000","last_clean":"2024-04-26T20:56:52.715683+0000","last_became_active":"2024-04-26T20:56:50.684679+0000","last_became_peered":"2024-04-26T20:56:50.684679+0000","last_unstale":"2024-04-26T20:56:52.715683+0000","last_undegraded":"2024-04-26T20:56:52.715683+0000","last_fullsized":"2024-04-26T20:56:52.715683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:34.811589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153972+0000","last_change":"2024-04-26T20:56:51.728717+0000","last_active":"2024-04-26T20:56:53.153972+0000","last_peered":"2024-04-26T20:56:53.153972+0000","last_clean":"2024-04-26T20:56:53.153972+0000","last_became_active":"2024-04-26T20:56:51.728261+0000","last_became_peered":"2024-04-26T20:56:51.728261+0000","last_unstale":"2024-04-26T20:56:53.153972+0000","last_undegraded":"2024-04-26T20:56:53.153972+0000","last_fullsized":"2024-04-26T20:56:53.153972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:02:31.238922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225598+0000","last_change":"2024-04-26T20:56:50.695251+0000","last_active":"2024-04-26T20:56:53.225598+0000","last_peered":"2024-04-26T20:56:53.225598+0000","last_clean":"2024-04-26T20:56:53.225598+0000","last_became_active":"2024-04-26T20:56:50.694956+0000","last_became_peered":"2024-04-26T20:56:50.694956+0000","last_unstale":"2024-04-26T20:56:53.225598+0000","last_undegraded":"2024-04-26T20:56:53.225598+0000","last_fullsized":"2024-04-26T20:56:53.225598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:22.304242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153932+0000","last_change":"2024-04-26T20:56:51.736963+0000","last_active":"2024-04-26T20:56:53.153932+0000","last_peered":"2024-04-26T20:56:53.153932+0000","last_clean":"2024-04-26T20:56:53.153932+0000","last_became_active":"2024-04-26T20:56:51.734437+0000","last_became_peered":"2024-04-26T20:56:51.734437+0000","last_unstale":"2024-04-26T20:56:53.153932+0000","last_undegraded":"2024-04-26T20:56:53.153932+0000","last_fullsized":"2024-04-26T20:56:53.153932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:08.679353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713517+0000","last_change":"2024-04-26T20:56:50.684034+0000","last_active":"2024-04-26T20:56:52.713517+0000","last_peered":"2024-04-26T20:56:52.713517+0000","last_clean":"2024-04-26T20:56:52.713517+0000","last_became_active":"2024-04-26T20:56:50.683720+0000","last_became_peered":"2024-04-26T20:56:50.683720+0000","last_unstale":"2024-04-26T20:56:52.713517+0000","last_undegraded":"2024-04-26T20:56:52.713517+0000","last_fullsized":"2024-04-26T20:56:52.713517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:46.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717068+0000","last_change":"2024-04-26T20:56:51.725602+0000","last_active":"2024-04-26T20:56:52.717068+0000","last_peered":"2024-04-26T20:56:52.717068+0000","last_clean":"2024-04-26T20:56:52.717068+0000","last_became_active":"2024-04-26T20:56:51.725308+0000","last_became_peered":"2024-04-26T20:56:51.725308+0000","last_unstale":"2024-04-26T20:56:52.717068+0000","last_undegraded":"2024-04-26T20:56:52.717068+0000","last_fullsized":"2024-04-26T20:56:52.717068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:52.330678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225216+0000","last_change":"2024-04-26T20:56:50.688464+0000","last_active":"2024-04-26T20:56:53.225216+0000","last_peered":"2024-04-26T20:56:53.225216+0000","last_clean":"2024-04-26T20:56:53.225216+0000","last_became_active":"2024-04-26T20:56:50.688101+0000","last_became_peered":"2024-04-26T20:56:50.688101+0000","last_unstale":"2024-04-26T20:56:53.225216+0000","last_undegraded":"2024-04-26T20:56:53.225216+0000","last_fullsized":"2024-04-26T20:56:53.225216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:50.526509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153386+0000","last_change":"2024-04-26T20:56:51.734773+0000","last_active":"2024-04-26T20:56:53.153386+0000","last_peered":"2024-04-26T20:56:53.153386+0000","last_clean":"2024-04-26T20:56:53.153386+0000","last_became_active":"2024-04-26T20:56:51.733903+0000","last_became_peered":"2024-04-26T20:56:51.733903+0000","last_unstale":"2024-04-26T20:56:53.153386+0000","last_undegraded":"2024-04-26T20:56:53.153386+0000","last_fullsized":"2024-04-26T20:56:53.153386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:23.450962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.152300+0000","last_change":"2024-04-26T20:56:46.848603+0000","last_active":"2024-04-26T20:56:53.152300+0000","last_peered":"2024-04-26T20:56:53.152300+0000","last_clean":"2024-04-26T20:56:53.152300+0000","last_became_active":"2024-04-26T20:56:46.846091+0000","last_became_peered":"2024-04-26T20:56:46.846091+0000","last_unstale":"2024-04-26T20:56:53.152300+0000","last_undegraded":"2024-04-26T20:56:53.152300+0000","last_fullsized":"2024-04-26T20:56:53.152300+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_clean_scrub_stamp":"2024-04-26T20:56:45.594041+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:00.232387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716618+0000","last_change":"2024-04-26T20:56:50.681942+0000","last_active":"2024-04-26T20:56:52.716618+0000","last_peered":"2024-04-26T20:56:52.716618+0000","last_clean":"2024-04-26T20:56:52.716618+0000","last_became_active":"2024-04-26T20:56:50.681645+0000","last_became_peered":"2024-04-26T20:56:50.681645+0000","last_unstale":"2024-04-26T20:56:52.716618+0000","last_undegraded":"2024-04-26T20:56:52.716618+0000","last_fullsized":"2024-04-26T20:56:52.716618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:39.328210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226904+0000","last_change":"2024-04-26T20:56:51.730048+0000","last_active":"2024-04-26T20:56:53.226904+0000","last_peered":"2024-04-26T20:56:53.226904+0000","last_clean":"2024-04-26T20:56:53.226904+0000","last_became_active":"2024-04-26T20:56:51.729733+0000","last_became_peered":"2024-04-26T20:56:51.729733+0000","last_unstale":"2024-04-26T20:56:53.226904+0000","last_undegraded":"2024-04-26T20:56:53.226904+0000","last_fullsized":"2024-04-26T20:56:53.226904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:24.679612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.152261+0000","last_change":"2024-04-26T20:56:50.699113+0000","last_active":"2024-04-26T20:56:53.152261+0000","last_peered":"2024-04-26T20:56:53.152261+0000","last_clean":"2024-04-26T20:56:53.152261+0000","last_became_active":"2024-04-26T20:56:50.698744+0000","last_became_peered":"2024-04-26T20:56:50.698744+0000","last_unstale":"2024-04-26T20:56:53.152261+0000","last_undegraded":"2024-04-26T20:56:53.152261+0000","last_fullsized":"2024-04-26T20:56:53.152261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:11.694549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144333+0000","last_change":"2024-04-26T20:56:51.732556+0000","last_active":"2024-04-26T20:56:53.144333+0000","last_peered":"2024-04-26T20:56:53.144333+0000","last_clean":"2024-04-26T20:56:53.144333+0000","last_became_active":"2024-04-26T20:56:51.732197+0000","last_became_peered":"2024-04-26T20:56:51.732197+0000","last_unstale":"2024-04-26T20:56:53.144333+0000","last_undegraded":"2024-04-26T20:56:53.144333+0000","last_fullsized":"2024-04-26T20:56:53.144333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:27:15.978359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225098+0000","last_change":"2024-04-26T20:56:50.694069+0000","last_active":"2024-04-26T20:56:53.225098+0000","last_peered":"2024-04-26T20:56:53.225098+0000","last_clean":"2024-04-26T20:56:53.225098+0000","last_became_active":"2024-04-26T20:56:50.693779+0000","last_became_peered":"2024-04-26T20:56:50.693779+0000","last_unstale":"2024-04-26T20:56:53.225098+0000","last_undegraded":"2024-04-26T20:56:53.225098+0000","last_fullsized":"2024-04-26T20:56:53.225098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:31.999414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.714833+0000","last_change":"2024-04-26T20:56:51.710866+0000","last_active":"2024-04-26T20:56:52.714833+0000","last_peered":"2024-04-26T20:56:52.714833+0000","last_clean":"2024-04-26T20:56:52.714833+0000","last_became_active":"2024-04-26T20:56:51.709325+0000","last_became_peered":"2024-04-26T20:56:51.709325+0000","last_unstale":"2024-04-26T20:56:52.714833+0000","last_undegraded":"2024-04-26T20:56:52.714833+0000","last_fullsized":"2024-04-26T20:56:52.714833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:23.268496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717176+0000","last_change":"2024-04-26T20:56:50.698231+0000","last_active":"2024-04-26T20:56:52.717176+0000","last_peered":"2024-04-26T20:56:52.717176+0000","last_clean":"2024-04-26T20:56:52.717176+0000","last_became_active":"2024-04-26T20:56:50.697872+0000","last_became_peered":"2024-04-26T20:56:50.697872+0000","last_unstale":"2024-04-26T20:56:52.717176+0000","last_undegraded":"2024-04-26T20:56:52.717176+0000","last_fullsized":"2024-04-26T20:56:52.717176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:41.662331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.715920+0000","last_change":"2024-04-26T20:56:51.720748+0000","last_active":"2024-04-26T20:56:52.715920+0000","last_peered":"2024-04-26T20:56:52.715920+0000","last_clean":"2024-04-26T20:56:52.715920+0000","last_became_active":"2024-04-26T20:56:51.720478+0000","last_became_peered":"2024-04-26T20:56:51.720478+0000","last_unstale":"2024-04-26T20:56:52.715920+0000","last_undegraded":"2024-04-26T20:56:52.715920+0000","last_fullsized":"2024-04-26T20:56:52.715920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:26.791119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713396+0000","last_change":"2024-04-26T20:56:50.685444+0000","last_active":"2024-04-26T20:56:52.713396+0000","last_peered":"2024-04-26T20:56:52.713396+0000","last_clean":"2024-04-26T20:56:52.713396+0000","last_became_active":"2024-04-26T20:56:50.684937+0000","last_became_peered":"2024-04-26T20:56:50.684937+0000","last_unstale":"2024-04-26T20:56:52.713396+0000","last_undegraded":"2024-04-26T20:56:52.713396+0000","last_fullsized":"2024-04-26T20:56:52.713396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:39.328210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717440+0000","last_change":"2024-04-26T20:56:51.727132+0000","last_active":"2024-04-26T20:56:52.717440+0000","last_peered":"2024-04-26T20:56:52.717440+0000","last_clean":"2024-04-26T20:56:52.717440+0000","last_became_active":"2024-04-26T20:56:51.720623+0000","last_became_peered":"2024-04-26T20:56:51.720623+0000","last_unstale":"2024-04-26T20:56:52.717440+0000","last_undegraded":"2024-04-26T20:56:52.717440+0000","last_fullsized":"2024-04-26T20:56:52.717440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:06.270681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153818+0000","last_change":"2024-04-26T20:56:50.693929+0000","last_active":"2024-04-26T20:56:53.153818+0000","last_peered":"2024-04-26T20:56:53.153818+0000","last_clean":"2024-04-26T20:56:53.153818+0000","last_became_active":"2024-04-26T20:56:50.693576+0000","last_became_peered":"2024-04-26T20:56:50.693576+0000","last_unstale":"2024-04-26T20:56:53.153818+0000","last_undegraded":"2024-04-26T20:56:53.153818+0000","last_fullsized":"2024-04-26T20:56:53.153818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:46.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713243+0000","last_change":"2024-04-26T20:56:51.714289+0000","last_active":"2024-04-26T20:56:52.713243+0000","last_peered":"2024-04-26T20:56:52.713243+0000","last_clean":"2024-04-26T20:56:52.713243+0000","last_became_active":"2024-04-26T20:56:51.713970+0000","last_became_peered":"2024-04-26T20:56:51.713970+0000","last_unstale":"2024-04-26T20:56:52.713243+0000","last_undegraded":"2024-04-26T20:56:52.713243+0000","last_fullsized":"2024-04-26T20:56:52.713243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:24.679612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.728669+0000","last_change":"2024-04-26T20:56:50.690572+0000","last_active":"2024-04-26T20:56:52.728669+0000","last_peered":"2024-04-26T20:56:52.728669+0000","last_clean":"2024-04-26T20:56:52.728669+0000","last_became_active":"2024-04-26T20:56:50.690221+0000","last_became_peered":"2024-04-26T20:56:50.690221+0000","last_unstale":"2024-04-26T20:56:52.728669+0000","last_undegraded":"2024-04-26T20:56:52.728669+0000","last_fullsized":"2024-04-26T20:56:52.728669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:11.694549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.227813+0000","last_change":"2024-04-26T20:56:51.740978+0000","last_active":"2024-04-26T20:56:53.227813+0000","last_peered":"2024-04-26T20:56:53.227813+0000","last_clean":"2024-04-26T20:56:53.227813+0000","last_became_active":"2024-04-26T20:56:51.739561+0000","last_became_peered":"2024-04-26T20:56:51.739561+0000","last_unstale":"2024-04-26T20:56:53.227813+0000","last_undegraded":"2024-04-26T20:56:53.227813+0000","last_fullsized":"2024-04-26T20:56:53.227813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:47.662176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730672+0000","last_change":"2024-04-26T20:56:50.694545+0000","last_active":"2024-04-26T20:56:52.730672+0000","last_peered":"2024-04-26T20:56:52.730672+0000","last_clean":"2024-04-26T20:56:52.730672+0000","last_became_active":"2024-04-26T20:56:50.694189+0000","last_became_peered":"2024-04-26T20:56:50.694189+0000","last_unstale":"2024-04-26T20:56:52.730672+0000","last_undegraded":"2024-04-26T20:56:52.730672+0000","last_fullsized":"2024-04-26T20:56:52.730672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:25.776704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144945+0000","last_change":"2024-04-26T20:56:51.724210+0000","last_active":"2024-04-26T20:56:53.144945+0000","last_peered":"2024-04-26T20:56:53.144945+0000","last_clean":"2024-04-26T20:56:53.144945+0000","last_became_active":"2024-04-26T20:56:51.723882+0000","last_became_peered":"2024-04-26T20:56:51.723882+0000","last_unstale":"2024-04-26T20:56:53.144945+0000","last_undegraded":"2024-04-26T20:56:53.144945+0000","last_fullsized":"2024-04-26T20:56:53.144945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:53.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.152108+0000","last_change":"2024-04-26T20:56:50.694039+0000","last_active":"2024-04-26T20:56:53.152108+0000","last_peered":"2024-04-26T20:56:53.152108+0000","last_clean":"2024-04-26T20:56:53.152108+0000","last_became_active":"2024-04-26T20:56:50.693569+0000","last_became_peered":"2024-04-26T20:56:50.693569+0000","last_unstale":"2024-04-26T20:56:53.152108+0000","last_undegraded":"2024-04-26T20:56:53.152108+0000","last_fullsized":"2024-04-26T20:56:53.152108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:41.662331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.714862+0000","last_change":"2024-04-26T20:56:51.719531+0000","last_active":"2024-04-26T20:56:52.714862+0000","last_peered":"2024-04-26T20:56:52.714862+0000","last_clean":"2024-04-26T20:56:52.714862+0000","last_became_active":"2024-04-26T20:56:51.714881+0000","last_became_peered":"2024-04-26T20:56:51.714881+0000","last_unstale":"2024-04-26T20:56:52.714862+0000","last_undegraded":"2024-04-26T20:56:52.714862+0000","last_fullsized":"2024-04-26T20:56:52.714862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:33.013829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713174+0000","last_change":"2024-04-26T20:56:50.689750+0000","last_active":"2024-04-26T20:56:52.713174+0000","last_peered":"2024-04-26T20:56:52.713174+0000","last_clean":"2024-04-26T20:56:52.713174+0000","last_became_active":"2024-04-26T20:56:50.689337+0000","last_became_peered":"2024-04-26T20:56:50.689337+0000","last_unstale":"2024-04-26T20:56:52.713174+0000","last_undegraded":"2024-04-26T20:56:52.713174+0000","last_fullsized":"2024-04-26T20:56:52.713174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:41.662331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226387+0000","last_change":"2024-04-26T20:56:51.712295+0000","last_active":"2024-04-26T20:56:53.226387+0000","last_peered":"2024-04-26T20:56:53.226387+0000","last_clean":"2024-04-26T20:56:53.226387+0000","last_became_active":"2024-04-26T20:56:51.711999+0000","last_became_peered":"2024-04-26T20:56:51.711999+0000","last_unstale":"2024-04-26T20:56:53.226387+0000","last_undegraded":"2024-04-26T20:56:53.226387+0000","last_fullsized":"2024-04-26T20:56:53.226387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:16.275518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713352+0000","last_change":"2024-04-26T20:56:50.697160+0000","last_active":"2024-04-26T20:56:52.713352+0000","last_peered":"2024-04-26T20:56:52.713352+0000","last_clean":"2024-04-26T20:56:52.713352+0000","last_became_active":"2024-04-26T20:56:50.695595+0000","last_became_peered":"2024-04-26T20:56:50.695595+0000","last_unstale":"2024-04-26T20:56:52.713352+0000","last_undegraded":"2024-04-26T20:56:52.713352+0000","last_fullsized":"2024-04-26T20:56:52.713352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:50.526509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226926+0000","last_change":"2024-04-26T20:56:51.717706+0000","last_active":"2024-04-26T20:56:53.226926+0000","last_peered":"2024-04-26T20:56:53.226926+0000","last_clean":"2024-04-26T20:56:53.226926+0000","last_became_active":"2024-04-26T20:56:51.717422+0000","last_became_peered":"2024-04-26T20:56:51.717422+0000","last_unstale":"2024-04-26T20:56:53.226926+0000","last_undegraded":"2024-04-26T20:56:53.226926+0000","last_fullsized":"2024-04-26T20:56:53.226926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:37.778325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153433+0000","last_change":"2024-04-26T20:56:50.695179+0000","last_active":"2024-04-26T20:56:53.153433+0000","last_peered":"2024-04-26T20:56:53.153433+0000","last_clean":"2024-04-26T20:56:53.153433+0000","last_became_active":"2024-04-26T20:56:50.694889+0000","last_became_peered":"2024-04-26T20:56:50.694889+0000","last_unstale":"2024-04-26T20:56:53.153433+0000","last_undegraded":"2024-04-26T20:56:53.153433+0000","last_fullsized":"2024-04-26T20:56:53.153433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:50.526509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730113+0000","last_change":"2024-04-26T20:56:51.713029+0000","last_active":"2024-04-26T20:56:52.730113+0000","last_peered":"2024-04-26T20:56:52.730113+0000","last_clean":"2024-04-26T20:56:52.730113+0000","last_became_active":"2024-04-26T20:56:51.712311+0000","last_became_peered":"2024-04-26T20:56:51.712311+0000","last_unstale":"2024-04-26T20:56:52.730113+0000","last_undegraded":"2024-04-26T20:56:52.730113+0000","last_fullsized":"2024-04-26T20:56:52.730113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:36.354765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717262+0000","last_change":"2024-04-26T20:56:51.730361+0000","last_active":"2024-04-26T20:56:52.717262+0000","last_peered":"2024-04-26T20:56:52.717262+0000","last_clean":"2024-04-26T20:56:52.717262+0000","last_became_active":"2024-04-26T20:56:51.730068+0000","last_became_peered":"2024-04-26T20:56:51.730068+0000","last_unstale":"2024-04-26T20:56:52.717262+0000","last_undegraded":"2024-04-26T20:56:52.717262+0000","last_fullsized":"2024-04-26T20:56:52.717262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:07.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153083+0000","last_change":"2024-04-26T20:56:50.696973+0000","last_active":"2024-04-26T20:56:53.153083+0000","last_peered":"2024-04-26T20:56:53.153083+0000","last_clean":"2024-04-26T20:56:53.153083+0000","last_became_active":"2024-04-26T20:56:50.696603+0000","last_became_peered":"2024-04-26T20:56:50.696603+0000","last_unstale":"2024-04-26T20:56:53.153083+0000","last_undegraded":"2024-04-26T20:56:53.153083+0000","last_fullsized":"2024-04-26T20:56:53.153083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:25.776704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717108+0000","last_change":"2024-04-26T20:56:51.728709+0000","last_active":"2024-04-26T20:56:52.717108+0000","last_peered":"2024-04-26T20:56:52.717108+0000","last_clean":"2024-04-26T20:56:52.717108+0000","last_became_active":"2024-04-26T20:56:51.727858+0000","last_became_peered":"2024-04-26T20:56:51.727858+0000","last_unstale":"2024-04-26T20:56:52.717108+0000","last_undegraded":"2024-04-26T20:56:52.717108+0000","last_fullsized":"2024-04-26T20:56:52.717108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:23.450962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153531+0000","last_change":"2024-04-26T20:56:50.696966+0000","last_active":"2024-04-26T20:56:53.153531+0000","last_peered":"2024-04-26T20:56:53.153531+0000","last_clean":"2024-04-26T20:56:53.153531+0000","last_became_active":"2024-04-26T20:56:50.696448+0000","last_became_peered":"2024-04-26T20:56:50.696448+0000","last_unstale":"2024-04-26T20:56:53.153531+0000","last_undegraded":"2024-04-26T20:56:53.153531+0000","last_fullsized":"2024-04-26T20:56:53.153531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:39.328210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144675+0000","last_change":"2024-04-26T20:56:51.731938+0000","last_active":"2024-04-26T20:56:53.144675+0000","last_peered":"2024-04-26T20:56:53.144675+0000","last_clean":"2024-04-26T20:56:53.144675+0000","last_became_active":"2024-04-26T20:56:51.731655+0000","last_became_peered":"2024-04-26T20:56:51.731655+0000","last_unstale":"2024-04-26T20:56:53.144675+0000","last_undegraded":"2024-04-26T20:56:53.144675+0000","last_fullsized":"2024-04-26T20:56:53.144675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:18.631055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.227678+0000","last_change":"2024-04-26T20:56:50.692059+0000","last_active":"2024-04-26T20:56:53.227678+0000","last_peered":"2024-04-26T20:56:53.227678+0000","last_clean":"2024-04-26T20:56:53.227678+0000","last_became_active":"2024-04-26T20:56:50.690628+0000","last_became_peered":"2024-04-26T20:56:50.690628+0000","last_unstale":"2024-04-26T20:56:53.227678+0000","last_undegraded":"2024-04-26T20:56:53.227678+0000","last_fullsized":"2024-04-26T20:56:53.227678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:00.125994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.227526+0000","last_change":"2024-04-26T20:56:51.719241+0000","last_active":"2024-04-26T20:56:53.227526+0000","last_peered":"2024-04-26T20:56:53.227526+0000","last_clean":"2024-04-26T20:56:53.227526+0000","last_became_active":"2024-04-26T20:56:51.718893+0000","last_became_peered":"2024-04-26T20:56:51.718893+0000","last_unstale":"2024-04-26T20:56:53.227526+0000","last_undegraded":"2024-04-26T20:56:53.227526+0000","last_fullsized":"2024-04-26T20:56:53.227526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:23.268496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144835+0000","last_change":"2024-04-26T20:56:50.691078+0000","last_active":"2024-04-26T20:56:53.144835+0000","last_peered":"2024-04-26T20:56:53.144835+0000","last_clean":"2024-04-26T20:56:53.144835+0000","last_became_active":"2024-04-26T20:56:50.689362+0000","last_became_peered":"2024-04-26T20:56:50.689362+0000","last_unstale":"2024-04-26T20:56:53.144835+0000","last_undegraded":"2024-04-26T20:56:53.144835+0000","last_fullsized":"2024-04-26T20:56:53.144835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:00.125994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":204800,"data_stored":69305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":811008,"data_stored":660311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:56:56.140 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:56:56.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.390+0000 7f1185800640 1 Processor -- start 2024-04-26T20:56:56.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.390+0000 7f1185800640 1 -- start start 2024-04-26T20:56:56.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.391+0000 7f1185800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1180146980 0x7f11800b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:56.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.392+0000 7f1185800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f11800740e0 0x7f11800b6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:56.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.392+0000 7f1185800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11800d0a30 con 0x7f118015d5b0 2024-04-26T20:56:56.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.392+0000 7f1185800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f118015e510 con 0x7f11800740e0 2024-04-26T20:56:56.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.392+0000 7f1185800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f118015e230 con 0x7f1180146980 2024-04-26T20:56:56.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.392+0000 7f1177e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f11800740e0 0x7f11800b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:56.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.392+0000 7f1177e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f11800740e0 0x7f11800b6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42046/0 (socket says 172.21.15.28:42046) 2024-04-26T20:56:56.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.392+0000 7f1177e00640 1 -- 172.21.15.28:0/4221843852 learned_addr learned my addr 172.21.15.28:0/4221843852 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:56.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.392+0000 7f117ea00640 1 --2- 172.21.15.28:0/4221843852 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1180146980 0x7f11800b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:56.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.393+0000 7f1177e00640 1 -- 172.21.15.28:0/4221843852 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1180146980 msgr2=0x7f11800b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:56.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.393+0000 7f1177e00640 1 --2- 172.21.15.28:0/4221843852 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1180146980 0x7f11800b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:56.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.393+0000 7f1177e00640 1 -- 172.21.15.28:0/4221843852 >> v1:172.21.15.28:6789/0 conn(0x7f118015d5b0 legacy=0x7f118015da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:56.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.393+0000 7f1177e00640 1 -- 172.21.15.28:0/4221843852 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1180073600 con 0x7f11800740e0 2024-04-26T20:56:56.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.393+0000 7f117ea00640 1 --2- 172.21.15.28:0/4221843852 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1180146980 0x7f11800b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:56:56.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.394+0000 7f1177e00640 1 --2- 172.21.15.28:0/4221843852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11800740e0 0x7f11800b6640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1168009870 tx=0x7f116802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6d2c51293ef21c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:56.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.394+0000 7f117e000640 1 -- 172.21.15.28:0/4221843852 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168037070 con 0x7f11800740e0 2024-04-26T20:56:56.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.394+0000 7f117e000640 1 -- 172.21.15.28:0/4221843852 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1168032440 con 0x7f11800740e0 2024-04-26T20:56:56.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.394+0000 7f117e000640 1 -- 172.21.15.28:0/4221843852 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1168032810 con 0x7f11800740e0 2024-04-26T20:56:56.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.395+0000 7f1185800640 1 -- 172.21.15.28:0/4221843852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11800740e0 msgr2=0x7f11800b6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:56.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.395+0000 7f1185800640 1 --2- 172.21.15.28:0/4221843852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11800740e0 0x7f11800b6640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1168009870 tx=0x7f116802a180 comp rx=0 tx=0).stop 2024-04-26T20:56:56.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.395+0000 7f1185800640 1 -- 172.21.15.28:0/4221843852 shutdown_connections 2024-04-26T20:56:56.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.395+0000 7f1185800640 1 --2- 172.21.15.28:0/4221843852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11800740e0 0x7f11800b6640 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:56.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.395+0000 7f1185800640 1 --2- 172.21.15.28:0/4221843852 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1180146980 0x7f11800b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:56.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.395+0000 7f1185800640 1 -- 172.21.15.28:0/4221843852 >> 172.21.15.28:0/4221843852 conn(0x7f118006e220 msgr2=0x7f118006e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:56.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.395+0000 7f1185800640 1 -- 172.21.15.28:0/4221843852 shutdown_connections 2024-04-26T20:56:56.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.395+0000 7f1185800640 1 -- 172.21.15.28:0/4221843852 wait complete. 2024-04-26T20:56:56.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.397+0000 7f1185800640 1 Processor -- start 2024-04-26T20:56:56.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.397+0000 7f1185800640 1 -- start start 2024-04-26T20:56:56.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.398+0000 7f1185800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11800740e0 0x7f118015a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:56.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.398+0000 7f1185800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1180146980 0x7f1180142140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:56.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.398+0000 7f1185800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f118015d5b0 0x7f1180142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:56.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.398+0000 7f1185800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1180071ff0 con 0x7f118015d5b0 2024-04-26T20:56:56.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.398+0000 7f1185800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1180073bc0 con 0x7f11800740e0 2024-04-26T20:56:56.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.398+0000 7f1185800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f1180072170 con 0x7f1180146980 2024-04-26T20:56:56.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.398+0000 7f117f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f118015d5b0 0x7f1180142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:56.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.398+0000 7f117f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f118015d5b0 0x7f1180142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42060/0 (socket says 172.21.15.28:42060) 2024-04-26T20:56:56.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.398+0000 7f117f400640 1 -- 172.21.15.28:0/2057063893 learned_addr learned my addr 172.21.15.28:0/2057063893 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:56.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.398+0000 7f117f400640 1 -- 172.21.15.28:0/2057063893 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1180146980 msgr2=0x7f1180142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:56.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.398+0000 7f117f400640 1 --2- 172.21.15.28:0/2057063893 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1180146980 0x7f1180142140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:56.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.398+0000 7f117f400640 1 -- 172.21.15.28:0/2057063893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11800740e0 msgr2=0x7f118015a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:56.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.398+0000 7f117f400640 1 --2- 172.21.15.28:0/2057063893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11800740e0 0x7f118015a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:56.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.398+0000 7f117f400640 1 -- 172.21.15.28:0/2057063893 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11801f3510 con 0x7f118015d5b0 2024-04-26T20:56:56.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.399+0000 7f117f400640 1 --2- 172.21.15.28:0/2057063893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f118015d5b0 0x7f1180142680 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f117800b970 tx=0x7f117800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:56.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.399+0000 7f1176a00640 1 -- 172.21.15.28:0/2057063893 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1178019070 con 0x7f118015d5b0 2024-04-26T20:56:56.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.399+0000 7f1176a00640 1 -- 172.21.15.28:0/2057063893 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f117800cb40 con 0x7f118015d5b0 2024-04-26T20:56:56.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.399+0000 7f1176a00640 1 -- 172.21.15.28:0/2057063893 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1178014440 con 0x7f118015d5b0 2024-04-26T20:56:56.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.399+0000 7f1185800640 1 -- 172.21.15.28:0/2057063893 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11801f37f0 con 0x7f118015d5b0 2024-04-26T20:56:56.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.399+0000 7f1185800640 1 -- 172.21.15.28:0/2057063893 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11801f3db0 con 0x7f118015d5b0 2024-04-26T20:56:56.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.400+0000 7f1185800640 1 -- 172.21.15.28:0/2057063893 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f118006a400 con 0x7f118015d5b0 2024-04-26T20:56:56.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.402+0000 7f1176a00640 1 -- 172.21.15.28:0/2057063893 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f117801c070 con 0x7f118015d5b0 2024-04-26T20:56:56.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.403+0000 7f1176a00640 1 --2- 172.21.15.28:0/2057063893 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f115c06c8f0 0x7f115c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:56.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.403+0000 7f1176a00640 1 -- 172.21.15.28:0/2057063893 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f117808b830 con 0x7f118015d5b0 2024-04-26T20:56:56.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.403+0000 7f117ea00640 1 --2- 172.21.15.28:0/2057063893 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f115c06c8f0 0x7f115c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:56.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.404+0000 7f117ea00640 1 --2- 172.21.15.28:0/2057063893 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f115c06c8f0 0x7f115c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1170006fd0 tx=0x7f1170008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:56.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.408+0000 7f1176a00640 1 -- 172.21.15.28:0/2057063893 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1178059bc0 con 0x7f118015d5b0 2024-04-26T20:56:56.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.606+0000 7f1185800640 1 -- 172.21.15.28:0/2057063893 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1180142ce0 con 0x7f115c06c8f0 2024-04-26T20:56:56.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.625+0000 7f1176a00640 1 -- 172.21.15.28:0/2057063893 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f1180142ce0 con 0x7f115c06c8f0 2024-04-26T20:56:56.629 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:56:56.631 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T20:56:56.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.632+0000 7f1185800640 1 -- 172.21.15.28:0/2057063893 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f115c06c8f0 msgr2=0x7f115c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:56.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.632+0000 7f1185800640 1 --2- 172.21.15.28:0/2057063893 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f115c06c8f0 0x7f115c06edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1170006fd0 tx=0x7f1170008040 comp rx=0 tx=0).stop 2024-04-26T20:56:56.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.632+0000 7f1185800640 1 -- 172.21.15.28:0/2057063893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f118015d5b0 msgr2=0x7f1180142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:56.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.632+0000 7f1185800640 1 --2- 172.21.15.28:0/2057063893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f118015d5b0 0x7f1180142680 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f117800b970 tx=0x7f117800be30 comp rx=0 tx=0).stop 2024-04-26T20:56:56.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.632+0000 7f1185800640 1 -- 172.21.15.28:0/2057063893 shutdown_connections 2024-04-26T20:56:56.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.632+0000 7f1185800640 1 --2- 172.21.15.28:0/2057063893 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f115c06c8f0 0x7f115c06edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:56.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.632+0000 7f1185800640 1 --2- 172.21.15.28:0/2057063893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f118015d5b0 0x7f1180142680 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:56.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.632+0000 7f1185800640 1 --2- 172.21.15.28:0/2057063893 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1180146980 0x7f1180142140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:56.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.632+0000 7f1185800640 1 --2- 172.21.15.28:0/2057063893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f11800740e0 0x7f118015a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:56.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.632+0000 7f1185800640 1 -- 172.21.15.28:0/2057063893 >> 172.21.15.28:0/2057063893 conn(0x7f118006e220 msgr2=0x7f118013f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:56.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.632+0000 7f1185800640 1 -- 172.21.15.28:0/2057063893 shutdown_connections 2024-04-26T20:56:56.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.632+0000 7f1185800640 1 -- 172.21.15.28:0/2057063893 wait complete. 2024-04-26T20:56:56.652 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-26T20:56:54.701536+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235928,"kb_used_data":2896,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738792,"statfs":{"total":773094113280,"available":772852523008,"internally_reserved":0,"allocated":2965504,"data_stored":1777796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.059007"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153718+0000","last_change":"2024-04-26T20:56:51.731207+0000","last_active":"2024-04-26T20:56:53.153718+0000","last_peered":"2024-04-26T20:56:53.153718+0000","last_clean":"2024-04-26T20:56:53.153718+0000","last_became_active":"2024-04-26T20:56:51.730930+0000","last_became_peered":"2024-04-26T20:56:51.730930+0000","last_unstale":"2024-04-26T20:56:53.153718+0000","last_undegraded":"2024-04-26T20:56:53.153718+0000","last_fullsized":"2024-04-26T20:56:53.153718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:06.270681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.714391+0000","last_change":"2024-04-26T20:56:51.719125+0000","last_active":"2024-04-26T20:56:52.714391+0000","last_peered":"2024-04-26T20:56:52.714391+0000","last_clean":"2024-04-26T20:56:52.714391+0000","last_became_active":"2024-04-26T20:56:51.717685+0000","last_became_peered":"2024-04-26T20:56:51.717685+0000","last_unstale":"2024-04-26T20:56:52.714391+0000","last_undegraded":"2024-04-26T20:56:52.714391+0000","last_fullsized":"2024-04-26T20:56:52.714391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.428432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144729+0000","last_change":"2024-04-26T20:56:51.717024+0000","last_active":"2024-04-26T20:56:53.144729+0000","last_peered":"2024-04-26T20:56:53.144729+0000","last_clean":"2024-04-26T20:56:53.144729+0000","last_became_active":"2024-04-26T20:56:51.716486+0000","last_became_peered":"2024-04-26T20:56:51.716486+0000","last_unstale":"2024-04-26T20:56:53.144729+0000","last_undegraded":"2024-04-26T20:56:53.144729+0000","last_fullsized":"2024-04-26T20:56:53.144729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:01.617412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716816+0000","last_change":"2024-04-26T20:56:51.716770+0000","last_active":"2024-04-26T20:56:52.716816+0000","last_peered":"2024-04-26T20:56:52.716816+0000","last_clean":"2024-04-26T20:56:52.716816+0000","last_became_active":"2024-04-26T20:56:51.716443+0000","last_became_peered":"2024-04-26T20:56:51.716443+0000","last_unstale":"2024-04-26T20:56:52.716816+0000","last_undegraded":"2024-04-26T20:56:52.716816+0000","last_fullsized":"2024-04-26T20:56:52.716816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:08.812637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153115+0000","last_change":"2024-04-26T20:56:51.735130+0000","last_active":"2024-04-26T20:56:53.153115+0000","last_peered":"2024-04-26T20:56:53.153115+0000","last_clean":"2024-04-26T20:56:53.153115+0000","last_became_active":"2024-04-26T20:56:51.734775+0000","last_became_peered":"2024-04-26T20:56:51.734775+0000","last_unstale":"2024-04-26T20:56:53.153115+0000","last_undegraded":"2024-04-26T20:56:53.153115+0000","last_fullsized":"2024-04-26T20:56:53.153115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:08.812637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716491+0000","last_change":"2024-04-26T20:56:51.719621+0000","last_active":"2024-04-26T20:56:52.716491+0000","last_peered":"2024-04-26T20:56:52.716491+0000","last_clean":"2024-04-26T20:56:52.716491+0000","last_became_active":"2024-04-26T20:56:51.719312+0000","last_became_peered":"2024-04-26T20:56:51.719312+0000","last_unstale":"2024-04-26T20:56:52.716491+0000","last_undegraded":"2024-04-26T20:56:52.716491+0000","last_fullsized":"2024-04-26T20:56:52.716491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:23.268496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.152560+0000","last_change":"2024-04-26T20:56:51.730495+0000","last_active":"2024-04-26T20:56:53.152560+0000","last_peered":"2024-04-26T20:56:53.152560+0000","last_clean":"2024-04-26T20:56:53.152560+0000","last_became_active":"2024-04-26T20:56:51.729796+0000","last_became_peered":"2024-04-26T20:56:51.729796+0000","last_unstale":"2024-04-26T20:56:53.152560+0000","last_undegraded":"2024-04-26T20:56:53.152560+0000","last_fullsized":"2024-04-26T20:56:53.152560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:52.912874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225874+0000","last_change":"2024-04-26T20:56:51.729824+0000","last_active":"2024-04-26T20:56:53.225874+0000","last_peered":"2024-04-26T20:56:53.225874+0000","last_clean":"2024-04-26T20:56:53.225874+0000","last_became_active":"2024-04-26T20:56:51.729546+0000","last_became_peered":"2024-04-26T20:56:51.729546+0000","last_unstale":"2024-04-26T20:56:53.225874+0000","last_undegraded":"2024-04-26T20:56:53.225874+0000","last_fullsized":"2024-04-26T20:56:53.225874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:18.631055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153639+0000","last_change":"2024-04-26T20:56:51.732556+0000","last_active":"2024-04-26T20:56:53.153639+0000","last_peered":"2024-04-26T20:56:53.153639+0000","last_clean":"2024-04-26T20:56:53.153639+0000","last_became_active":"2024-04-26T20:56:51.732298+0000","last_became_peered":"2024-04-26T20:56:51.732298+0000","last_unstale":"2024-04-26T20:56:53.153639+0000","last_undegraded":"2024-04-26T20:56:53.153639+0000","last_fullsized":"2024-04-26T20:56:53.153639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:07.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730718+0000","last_change":"2024-04-26T20:56:51.724747+0000","last_active":"2024-04-26T20:56:52.730718+0000","last_peered":"2024-04-26T20:56:52.730718+0000","last_clean":"2024-04-26T20:56:52.730718+0000","last_became_active":"2024-04-26T20:56:51.724452+0000","last_became_peered":"2024-04-26T20:56:51.724452+0000","last_unstale":"2024-04-26T20:56:52.730718+0000","last_undegraded":"2024-04-26T20:56:52.730718+0000","last_fullsized":"2024-04-26T20:56:52.730718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.428432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730341+0000","last_change":"2024-04-26T20:56:51.729732+0000","last_active":"2024-04-26T20:56:52.730341+0000","last_peered":"2024-04-26T20:56:52.730341+0000","last_clean":"2024-04-26T20:56:52.730341+0000","last_became_active":"2024-04-26T20:56:51.729464+0000","last_became_peered":"2024-04-26T20:56:51.729464+0000","last_unstale":"2024-04-26T20:56:52.730341+0000","last_undegraded":"2024-04-26T20:56:52.730341+0000","last_fullsized":"2024-04-26T20:56:52.730341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:37.778325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.715425+0000","last_change":"2024-04-26T20:56:51.723870+0000","last_active":"2024-04-26T20:56:52.715425+0000","last_peered":"2024-04-26T20:56:52.715425+0000","last_clean":"2024-04-26T20:56:52.715425+0000","last_became_active":"2024-04-26T20:56:51.722037+0000","last_became_peered":"2024-04-26T20:56:51.722037+0000","last_unstale":"2024-04-26T20:56:52.715425+0000","last_undegraded":"2024-04-26T20:56:52.715425+0000","last_fullsized":"2024-04-26T20:56:52.715425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:42.676745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226737+0000","last_change":"2024-04-26T20:56:51.719947+0000","last_active":"2024-04-26T20:56:53.226737+0000","last_peered":"2024-04-26T20:56:53.226737+0000","last_clean":"2024-04-26T20:56:53.226737+0000","last_became_active":"2024-04-26T20:56:51.719565+0000","last_became_peered":"2024-04-26T20:56:51.719565+0000","last_unstale":"2024-04-26T20:56:53.226737+0000","last_undegraded":"2024-04-26T20:56:53.226737+0000","last_fullsized":"2024-04-26T20:56:53.226737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:36.354765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144673+0000","last_change":"2024-04-26T20:56:51.725826+0000","last_active":"2024-04-26T20:56:53.144673+0000","last_peered":"2024-04-26T20:56:53.144673+0000","last_clean":"2024-04-26T20:56:53.144673+0000","last_became_active":"2024-04-26T20:56:51.723315+0000","last_became_peered":"2024-04-26T20:56:51.723315+0000","last_unstale":"2024-04-26T20:56:53.144673+0000","last_undegraded":"2024-04-26T20:56:53.144673+0000","last_fullsized":"2024-04-26T20:56:53.144673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:38:55.851891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713041+0000","last_change":"2024-04-26T20:56:51.704000+0000","last_active":"2024-04-26T20:56:52.713041+0000","last_peered":"2024-04-26T20:56:52.713041+0000","last_clean":"2024-04-26T20:56:52.713041+0000","last_became_active":"2024-04-26T20:56:51.703007+0000","last_became_peered":"2024-04-26T20:56:51.703007+0000","last_unstale":"2024-04-26T20:56:52.713041+0000","last_undegraded":"2024-04-26T20:56:52.713041+0000","last_fullsized":"2024-04-26T20:56:52.713041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:36.354765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716554+0000","last_change":"2024-04-26T20:56:51.724622+0000","last_active":"2024-04-26T20:56:52.716554+0000","last_peered":"2024-04-26T20:56:52.716554+0000","last_clean":"2024-04-26T20:56:52.716554+0000","last_became_active":"2024-04-26T20:56:51.723198+0000","last_became_peered":"2024-04-26T20:56:51.723198+0000","last_unstale":"2024-04-26T20:56:52.716554+0000","last_undegraded":"2024-04-26T20:56:52.716554+0000","last_fullsized":"2024-04-26T20:56:52.716554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:52.912874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.718883+0000","last_change":"2024-04-26T20:56:51.736813+0000","last_active":"2024-04-26T20:56:52.718883+0000","last_peered":"2024-04-26T20:56:52.718883+0000","last_clean":"2024-04-26T20:56:52.718883+0000","last_became_active":"2024-04-26T20:56:51.736532+0000","last_became_peered":"2024-04-26T20:56:51.736532+0000","last_unstale":"2024-04-26T20:56:52.718883+0000","last_undegraded":"2024-04-26T20:56:52.718883+0000","last_fullsized":"2024-04-26T20:56:52.718883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:08.679353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225324+0000","last_change":"2024-04-26T20:56:51.726234+0000","last_active":"2024-04-26T20:56:53.225324+0000","last_peered":"2024-04-26T20:56:53.225324+0000","last_clean":"2024-04-26T20:56:53.225324+0000","last_became_active":"2024-04-26T20:56:51.723289+0000","last_became_peered":"2024-04-26T20:56:51.723289+0000","last_unstale":"2024-04-26T20:56:53.225324+0000","last_undegraded":"2024-04-26T20:56:53.225324+0000","last_fullsized":"2024-04-26T20:56:53.225324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:12.708963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153878+0000","last_change":"2024-04-26T20:56:51.735058+0000","last_active":"2024-04-26T20:56:53.153878+0000","last_peered":"2024-04-26T20:56:53.153878+0000","last_clean":"2024-04-26T20:56:53.153878+0000","last_became_active":"2024-04-26T20:56:51.734516+0000","last_became_peered":"2024-04-26T20:56:51.734516+0000","last_unstale":"2024-04-26T20:56:53.153878+0000","last_undegraded":"2024-04-26T20:56:53.153878+0000","last_fullsized":"2024-04-26T20:56:53.153878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:33.950624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730309+0000","last_change":"2024-04-26T20:56:51.715023+0000","last_active":"2024-04-26T20:56:52.730309+0000","last_peered":"2024-04-26T20:56:52.730309+0000","last_clean":"2024-04-26T20:56:52.730309+0000","last_became_active":"2024-04-26T20:56:51.714250+0000","last_became_peered":"2024-04-26T20:56:51.714250+0000","last_unstale":"2024-04-26T20:56:52.730309+0000","last_undegraded":"2024-04-26T20:56:52.730309+0000","last_fullsized":"2024-04-26T20:56:52.730309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:24.679612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144998+0000","last_change":"2024-04-26T20:56:51.737492+0000","last_active":"2024-04-26T20:56:53.144998+0000","last_peered":"2024-04-26T20:56:53.144998+0000","last_clean":"2024-04-26T20:56:53.144998+0000","last_became_active":"2024-04-26T20:56:51.736774+0000","last_became_peered":"2024-04-26T20:56:51.736774+0000","last_unstale":"2024-04-26T20:56:53.144998+0000","last_undegraded":"2024-04-26T20:56:53.144998+0000","last_fullsized":"2024-04-26T20:56:53.144998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:36.354765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716674+0000","last_change":"2024-04-26T20:56:51.731749+0000","last_active":"2024-04-26T20:56:52.716674+0000","last_peered":"2024-04-26T20:56:52.716674+0000","last_clean":"2024-04-26T20:56:52.716674+0000","last_became_active":"2024-04-26T20:56:51.731460+0000","last_became_peered":"2024-04-26T20:56:51.731460+0000","last_unstale":"2024-04-26T20:56:52.716674+0000","last_undegraded":"2024-04-26T20:56:52.716674+0000","last_fullsized":"2024-04-26T20:56:52.716674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:47.662176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226704+0000","last_change":"2024-04-26T20:56:51.721856+0000","last_active":"2024-04-26T20:56:53.226704+0000","last_peered":"2024-04-26T20:56:53.226704+0000","last_clean":"2024-04-26T20:56:53.226704+0000","last_became_active":"2024-04-26T20:56:51.721531+0000","last_became_peered":"2024-04-26T20:56:51.721531+0000","last_unstale":"2024-04-26T20:56:53.226704+0000","last_undegraded":"2024-04-26T20:56:53.226704+0000","last_fullsized":"2024-04-26T20:56:53.226704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:53.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716438+0000","last_change":"2024-04-26T20:56:51.718492+0000","last_active":"2024-04-26T20:56:52.716438+0000","last_peered":"2024-04-26T20:56:52.716438+0000","last_clean":"2024-04-26T20:56:52.716438+0000","last_became_active":"2024-04-26T20:56:51.718127+0000","last_became_peered":"2024-04-26T20:56:51.718127+0000","last_unstale":"2024-04-26T20:56:52.716438+0000","last_undegraded":"2024-04-26T20:56:52.716438+0000","last_fullsized":"2024-04-26T20:56:52.716438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.428432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.227580+0000","last_change":"2024-04-26T20:56:51.722240+0000","last_active":"2024-04-26T20:56:53.227580+0000","last_peered":"2024-04-26T20:56:53.227580+0000","last_clean":"2024-04-26T20:56:53.227580+0000","last_became_active":"2024-04-26T20:56:51.721601+0000","last_became_peered":"2024-04-26T20:56:51.721601+0000","last_unstale":"2024-04-26T20:56:53.227580+0000","last_undegraded":"2024-04-26T20:56:53.227580+0000","last_fullsized":"2024-04-26T20:56:53.227580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:52.912874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730730+0000","last_change":"2024-04-26T20:56:51.722608+0000","last_active":"2024-04-26T20:56:52.730730+0000","last_peered":"2024-04-26T20:56:52.730730+0000","last_clean":"2024-04-26T20:56:52.730730+0000","last_became_active":"2024-04-26T20:56:51.722321+0000","last_became_peered":"2024-04-26T20:56:51.722321+0000","last_unstale":"2024-04-26T20:56:52.730730+0000","last_undegraded":"2024-04-26T20:56:52.730730+0000","last_fullsized":"2024-04-26T20:56:52.730730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:23.268496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713738+0000","last_change":"2024-04-26T20:56:51.715230+0000","last_active":"2024-04-26T20:56:52.713738+0000","last_peered":"2024-04-26T20:56:52.713738+0000","last_clean":"2024-04-26T20:56:52.713738+0000","last_became_active":"2024-04-26T20:56:51.714911+0000","last_became_peered":"2024-04-26T20:56:51.714911+0000","last_unstale":"2024-04-26T20:56:52.713738+0000","last_undegraded":"2024-04-26T20:56:52.713738+0000","last_fullsized":"2024-04-26T20:56:52.713738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:37.778325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.227039+0000","last_change":"2024-04-26T20:56:51.712010+0000","last_active":"2024-04-26T20:56:53.227039+0000","last_peered":"2024-04-26T20:56:53.227039+0000","last_clean":"2024-04-26T20:56:53.227039+0000","last_became_active":"2024-04-26T20:56:51.711583+0000","last_became_peered":"2024-04-26T20:56:51.711583+0000","last_unstale":"2024-04-26T20:56:53.227039+0000","last_undegraded":"2024-04-26T20:56:53.227039+0000","last_fullsized":"2024-04-26T20:56:53.227039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.428432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.145333+0000","last_change":"2024-04-26T20:56:51.722575+0000","last_active":"2024-04-26T20:56:53.145333+0000","last_peered":"2024-04-26T20:56:53.145333+0000","last_clean":"2024-04-26T20:56:53.145333+0000","last_became_active":"2024-04-26T20:56:51.722274+0000","last_became_peered":"2024-04-26T20:56:51.722274+0000","last_unstale":"2024-04-26T20:56:53.145333+0000","last_undegraded":"2024-04-26T20:56:53.145333+0000","last_fullsized":"2024-04-26T20:56:53.145333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.428432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716895+0000","last_change":"2024-04-26T20:56:51.712244+0000","last_active":"2024-04-26T20:56:52.716895+0000","last_peered":"2024-04-26T20:56:52.716895+0000","last_clean":"2024-04-26T20:56:52.716895+0000","last_became_active":"2024-04-26T20:56:51.711957+0000","last_became_peered":"2024-04-26T20:56:51.711957+0000","last_unstale":"2024-04-26T20:56:52.716895+0000","last_undegraded":"2024-04-26T20:56:52.716895+0000","last_fullsized":"2024-04-26T20:56:52.716895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:22.643252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225198+0000","last_change":"2024-04-26T20:56:51.721797+0000","last_active":"2024-04-26T20:56:53.225198+0000","last_peered":"2024-04-26T20:56:53.225198+0000","last_clean":"2024-04-26T20:56:53.225198+0000","last_became_active":"2024-04-26T20:56:51.721465+0000","last_became_peered":"2024-04-26T20:56:51.721465+0000","last_unstale":"2024-04-26T20:56:53.225198+0000","last_undegraded":"2024-04-26T20:56:53.225198+0000","last_fullsized":"2024-04-26T20:56:53.225198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:26.791119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.729925+0000","last_change":"2024-04-26T20:56:51.716527+0000","last_active":"2024-04-26T20:56:52.729925+0000","last_peered":"2024-04-26T20:56:52.729925+0000","last_clean":"2024-04-26T20:56:52.729925+0000","last_became_active":"2024-04-26T20:56:51.716103+0000","last_became_peered":"2024-04-26T20:56:51.716103+0000","last_unstale":"2024-04-26T20:56:52.729925+0000","last_undegraded":"2024-04-26T20:56:52.729925+0000","last_fullsized":"2024-04-26T20:56:52.729925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:53.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.714884+0000","last_change":"2024-04-26T20:56:51.713515+0000","last_active":"2024-04-26T20:56:52.714884+0000","last_peered":"2024-04-26T20:56:52.714884+0000","last_clean":"2024-04-26T20:56:52.714884+0000","last_became_active":"2024-04-26T20:56:51.710208+0000","last_became_peered":"2024-04-26T20:56:51.710208+0000","last_unstale":"2024-04-26T20:56:52.714884+0000","last_undegraded":"2024-04-26T20:56:52.714884+0000","last_fullsized":"2024-04-26T20:56:52.714884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:51.540923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226969+0000","last_change":"2024-04-26T20:56:50.692216+0000","last_active":"2024-04-26T20:56:53.226969+0000","last_peered":"2024-04-26T20:56:53.226969+0000","last_clean":"2024-04-26T20:56:53.226969+0000","last_became_active":"2024-04-26T20:56:50.691457+0000","last_became_peered":"2024-04-26T20:56:50.691457+0000","last_unstale":"2024-04-26T20:56:53.226969+0000","last_undegraded":"2024-04-26T20:56:53.226969+0000","last_fullsized":"2024-04-26T20:56:53.226969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:50.477216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.145297+0000","last_change":"2024-04-26T20:56:51.730925+0000","last_active":"2024-04-26T20:56:53.145297+0000","last_peered":"2024-04-26T20:56:53.145297+0000","last_clean":"2024-04-26T20:56:53.145297+0000","last_became_active":"2024-04-26T20:56:51.730497+0000","last_became_peered":"2024-04-26T20:56:51.730497+0000","last_unstale":"2024-04-26T20:56:53.145297+0000","last_undegraded":"2024-04-26T20:56:53.145297+0000","last_fullsized":"2024-04-26T20:56:53.145297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:37.778325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.729866+0000","last_change":"2024-04-26T20:56:50.684303+0000","last_active":"2024-04-26T20:56:52.729866+0000","last_peered":"2024-04-26T20:56:52.729866+0000","last_clean":"2024-04-26T20:56:52.729866+0000","last_became_active":"2024-04-26T20:56:50.684006+0000","last_became_peered":"2024-04-26T20:56:50.684006+0000","last_unstale":"2024-04-26T20:56:52.729866+0000","last_undegraded":"2024-04-26T20:56:52.729866+0000","last_fullsized":"2024-04-26T20:56:52.729866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:50.526509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226111+0000","last_change":"2024-04-26T20:56:50.688365+0000","last_active":"2024-04-26T20:56:53.226111+0000","last_peered":"2024-04-26T20:56:53.226111+0000","last_clean":"2024-04-26T20:56:53.226111+0000","last_became_active":"2024-04-26T20:56:50.688034+0000","last_became_peered":"2024-04-26T20:56:50.688034+0000","last_unstale":"2024-04-26T20:56:53.226111+0000","last_undegraded":"2024-04-26T20:56:53.226111+0000","last_fullsized":"2024-04-26T20:56:53.226111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:14.594594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226190+0000","last_change":"2024-04-26T20:56:51.721954+0000","last_active":"2024-04-26T20:56:53.226190+0000","last_peered":"2024-04-26T20:56:53.226190+0000","last_clean":"2024-04-26T20:56:53.226190+0000","last_became_active":"2024-04-26T20:56:51.721650+0000","last_became_peered":"2024-04-26T20:56:51.721650+0000","last_unstale":"2024-04-26T20:56:53.226190+0000","last_undegraded":"2024-04-26T20:56:53.226190+0000","last_fullsized":"2024-04-26T20:56:53.226190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:01.617412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716559+0000","last_change":"2024-04-26T20:56:50.680262+0000","last_active":"2024-04-26T20:56:52.716559+0000","last_peered":"2024-04-26T20:56:52.716559+0000","last_clean":"2024-04-26T20:56:52.716559+0000","last_became_active":"2024-04-26T20:56:50.675501+0000","last_became_peered":"2024-04-26T20:56:50.675501+0000","last_unstale":"2024-04-26T20:56:52.716559+0000","last_undegraded":"2024-04-26T20:56:52.716559+0000","last_fullsized":"2024-04-26T20:56:52.716559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:31.999414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713009+0000","last_change":"2024-04-26T20:56:51.714659+0000","last_active":"2024-04-26T20:56:52.713009+0000","last_peered":"2024-04-26T20:56:52.713009+0000","last_clean":"2024-04-26T20:56:52.713009+0000","last_became_active":"2024-04-26T20:56:51.714340+0000","last_became_peered":"2024-04-26T20:56:51.714340+0000","last_unstale":"2024-04-26T20:56:52.713009+0000","last_undegraded":"2024-04-26T20:56:52.713009+0000","last_fullsized":"2024-04-26T20:56:52.713009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:53.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730521+0000","last_change":"2024-04-26T20:56:50.689246+0000","last_active":"2024-04-26T20:56:52.730521+0000","last_peered":"2024-04-26T20:56:52.730521+0000","last_clean":"2024-04-26T20:56:52.730521+0000","last_became_active":"2024-04-26T20:56:50.688882+0000","last_became_peered":"2024-04-26T20:56:50.688882+0000","last_unstale":"2024-04-26T20:56:52.730521+0000","last_undegraded":"2024-04-26T20:56:52.730521+0000","last_fullsized":"2024-04-26T20:56:52.730521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:39.328210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.718004+0000","last_change":"2024-04-26T20:56:51.737626+0000","last_active":"2024-04-26T20:56:52.718004+0000","last_peered":"2024-04-26T20:56:52.718004+0000","last_clean":"2024-04-26T20:56:52.718004+0000","last_became_active":"2024-04-26T20:56:51.737273+0000","last_became_peered":"2024-04-26T20:56:51.737273+0000","last_unstale":"2024-04-26T20:56:52.718004+0000","last_undegraded":"2024-04-26T20:56:52.718004+0000","last_fullsized":"2024-04-26T20:56:52.718004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:33.950624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.718371+0000","last_change":"2024-04-26T20:56:50.690573+0000","last_active":"2024-04-26T20:56:52.718371+0000","last_peered":"2024-04-26T20:56:52.718371+0000","last_clean":"2024-04-26T20:56:52.718371+0000","last_became_active":"2024-04-26T20:56:50.690198+0000","last_became_peered":"2024-04-26T20:56:50.690198+0000","last_unstale":"2024-04-26T20:56:52.718371+0000","last_undegraded":"2024-04-26T20:56:52.718371+0000","last_fullsized":"2024-04-26T20:56:52.718371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:25.776704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144843+0000","last_change":"2024-04-26T20:56:51.733136+0000","last_active":"2024-04-26T20:56:53.144843+0000","last_peered":"2024-04-26T20:56:53.144843+0000","last_clean":"2024-04-26T20:56:53.144843+0000","last_became_active":"2024-04-26T20:56:51.732782+0000","last_became_peered":"2024-04-26T20:56:51.732782+0000","last_unstale":"2024-04-26T20:56:53.144843+0000","last_undegraded":"2024-04-26T20:56:53.144843+0000","last_fullsized":"2024-04-26T20:56:53.144843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:16.275518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713411+0000","last_change":"2024-04-26T20:56:50.689210+0000","last_active":"2024-04-26T20:56:52.713411+0000","last_peered":"2024-04-26T20:56:52.713411+0000","last_clean":"2024-04-26T20:56:52.713411+0000","last_became_active":"2024-04-26T20:56:50.688655+0000","last_became_peered":"2024-04-26T20:56:50.688655+0000","last_unstale":"2024-04-26T20:56:52.713411+0000","last_undegraded":"2024-04-26T20:56:52.713411+0000","last_fullsized":"2024-04-26T20:56:52.713411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:25.776704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.715628+0000","last_change":"2024-04-26T20:56:51.737157+0000","last_active":"2024-04-26T20:56:52.715628+0000","last_peered":"2024-04-26T20:56:52.715628+0000","last_clean":"2024-04-26T20:56:52.715628+0000","last_became_active":"2024-04-26T20:56:51.736082+0000","last_became_peered":"2024-04-26T20:56:51.736082+0000","last_unstale":"2024-04-26T20:56:52.715628+0000","last_undegraded":"2024-04-26T20:56:52.715628+0000","last_fullsized":"2024-04-26T20:56:52.715628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:40.342624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730295+0000","last_change":"2024-04-26T20:56:50.695225+0000","last_active":"2024-04-26T20:56:52.730295+0000","last_peered":"2024-04-26T20:56:52.730295+0000","last_clean":"2024-04-26T20:56:52.730295+0000","last_became_active":"2024-04-26T20:56:50.694892+0000","last_became_peered":"2024-04-26T20:56:50.694892+0000","last_unstale":"2024-04-26T20:56:52.730295+0000","last_undegraded":"2024-04-26T20:56:52.730295+0000","last_fullsized":"2024-04-26T20:56:52.730295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:41.662331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153191+0000","last_change":"2024-04-26T20:56:51.722030+0000","last_active":"2024-04-26T20:56:53.153191+0000","last_peered":"2024-04-26T20:56:53.153191+0000","last_clean":"2024-04-26T20:56:53.153191+0000","last_became_active":"2024-04-26T20:56:51.721724+0000","last_became_peered":"2024-04-26T20:56:51.721724+0000","last_unstale":"2024-04-26T20:56:53.153191+0000","last_undegraded":"2024-04-26T20:56:53.153191+0000","last_fullsized":"2024-04-26T20:56:53.153191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:52.330678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.145449+0000","last_change":"2024-04-26T20:56:50.682167+0000","last_active":"2024-04-26T20:56:53.145449+0000","last_peered":"2024-04-26T20:56:53.145449+0000","last_clean":"2024-04-26T20:56:53.145449+0000","last_became_active":"2024-04-26T20:56:50.675090+0000","last_became_peered":"2024-04-26T20:56:50.675090+0000","last_unstale":"2024-04-26T20:56:53.145449+0000","last_undegraded":"2024-04-26T20:56:53.145449+0000","last_fullsized":"2024-04-26T20:56:53.145449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:03.566375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.145134+0000","last_change":"2024-04-26T20:56:51.736362+0000","last_active":"2024-04-26T20:56:53.145134+0000","last_peered":"2024-04-26T20:56:53.145134+0000","last_clean":"2024-04-26T20:56:53.145134+0000","last_became_active":"2024-04-26T20:56:51.736050+0000","last_became_peered":"2024-04-26T20:56:51.736050+0000","last_unstale":"2024-04-26T20:56:53.145134+0000","last_undegraded":"2024-04-26T20:56:53.145134+0000","last_fullsized":"2024-04-26T20:56:53.145134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:24.679612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717056+0000","last_change":"2024-04-26T20:56:50.683775+0000","last_active":"2024-04-26T20:56:52.717056+0000","last_peered":"2024-04-26T20:56:52.717056+0000","last_clean":"2024-04-26T20:56:52.717056+0000","last_became_active":"2024-04-26T20:56:50.683470+0000","last_became_peered":"2024-04-26T20:56:50.683470+0000","last_unstale":"2024-04-26T20:56:52.717056+0000","last_undegraded":"2024-04-26T20:56:52.717056+0000","last_fullsized":"2024-04-26T20:56:52.717056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:50.526509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153154+0000","last_change":"2024-04-26T20:56:51.730365+0000","last_active":"2024-04-26T20:56:53.153154+0000","last_peered":"2024-04-26T20:56:53.153154+0000","last_clean":"2024-04-26T20:56:53.153154+0000","last_became_active":"2024-04-26T20:56:51.730015+0000","last_became_peered":"2024-04-26T20:56:51.730015+0000","last_unstale":"2024-04-26T20:56:53.153154+0000","last_undegraded":"2024-04-26T20:56:53.153154+0000","last_fullsized":"2024-04-26T20:56:53.153154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:22.643252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.729839+0000","last_change":"2024-04-26T20:56:50.686055+0000","last_active":"2024-04-26T20:56:52.729839+0000","last_peered":"2024-04-26T20:56:52.729839+0000","last_clean":"2024-04-26T20:56:52.729839+0000","last_became_active":"2024-04-26T20:56:50.685771+0000","last_became_peered":"2024-04-26T20:56:50.685771+0000","last_unstale":"2024-04-26T20:56:52.729839+0000","last_undegraded":"2024-04-26T20:56:52.729839+0000","last_fullsized":"2024-04-26T20:56:52.729839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:31.999414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.729780+0000","last_change":"2024-04-26T20:56:51.719273+0000","last_active":"2024-04-26T20:56:52.729780+0000","last_peered":"2024-04-26T20:56:52.729780+0000","last_clean":"2024-04-26T20:56:52.729780+0000","last_became_active":"2024-04-26T20:56:51.718659+0000","last_became_peered":"2024-04-26T20:56:51.718659+0000","last_unstale":"2024-04-26T20:56:52.729780+0000","last_undegraded":"2024-04-26T20:56:52.729780+0000","last_fullsized":"2024-04-26T20:56:52.729780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:16.275518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225397+0000","last_change":"2024-04-26T20:56:50.686714+0000","last_active":"2024-04-26T20:56:53.225397+0000","last_peered":"2024-04-26T20:56:53.225397+0000","last_clean":"2024-04-26T20:56:53.225397+0000","last_became_active":"2024-04-26T20:56:50.686348+0000","last_became_peered":"2024-04-26T20:56:50.686348+0000","last_unstale":"2024-04-26T20:56:53.225397+0000","last_undegraded":"2024-04-26T20:56:53.225397+0000","last_fullsized":"2024-04-26T20:56:53.225397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:14.594594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.152873+0000","last_change":"2024-04-26T20:56:51.730639+0000","last_active":"2024-04-26T20:56:53.152873+0000","last_peered":"2024-04-26T20:56:53.152873+0000","last_clean":"2024-04-26T20:56:53.152873+0000","last_became_active":"2024-04-26T20:56:51.730196+0000","last_became_peered":"2024-04-26T20:56:51.730196+0000","last_unstale":"2024-04-26T20:56:53.152873+0000","last_undegraded":"2024-04-26T20:56:53.152873+0000","last_fullsized":"2024-04-26T20:56:53.152873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:47.662176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.715683+0000","last_change":"2024-04-26T20:56:50.685016+0000","last_active":"2024-04-26T20:56:52.715683+0000","last_peered":"2024-04-26T20:56:52.715683+0000","last_clean":"2024-04-26T20:56:52.715683+0000","last_became_active":"2024-04-26T20:56:50.684679+0000","last_became_peered":"2024-04-26T20:56:50.684679+0000","last_unstale":"2024-04-26T20:56:52.715683+0000","last_undegraded":"2024-04-26T20:56:52.715683+0000","last_fullsized":"2024-04-26T20:56:52.715683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:34.811589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153972+0000","last_change":"2024-04-26T20:56:51.728717+0000","last_active":"2024-04-26T20:56:53.153972+0000","last_peered":"2024-04-26T20:56:53.153972+0000","last_clean":"2024-04-26T20:56:53.153972+0000","last_became_active":"2024-04-26T20:56:51.728261+0000","last_became_peered":"2024-04-26T20:56:51.728261+0000","last_unstale":"2024-04-26T20:56:53.153972+0000","last_undegraded":"2024-04-26T20:56:53.153972+0000","last_fullsized":"2024-04-26T20:56:53.153972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:02:31.238922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225598+0000","last_change":"2024-04-26T20:56:50.695251+0000","last_active":"2024-04-26T20:56:53.225598+0000","last_peered":"2024-04-26T20:56:53.225598+0000","last_clean":"2024-04-26T20:56:53.225598+0000","last_became_active":"2024-04-26T20:56:50.694956+0000","last_became_peered":"2024-04-26T20:56:50.694956+0000","last_unstale":"2024-04-26T20:56:53.225598+0000","last_undegraded":"2024-04-26T20:56:53.225598+0000","last_fullsized":"2024-04-26T20:56:53.225598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:22.304242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153932+0000","last_change":"2024-04-26T20:56:51.736963+0000","last_active":"2024-04-26T20:56:53.153932+0000","last_peered":"2024-04-26T20:56:53.153932+0000","last_clean":"2024-04-26T20:56:53.153932+0000","last_became_active":"2024-04-26T20:56:51.734437+0000","last_became_peered":"2024-04-26T20:56:51.734437+0000","last_unstale":"2024-04-26T20:56:53.153932+0000","last_undegraded":"2024-04-26T20:56:53.153932+0000","last_fullsized":"2024-04-26T20:56:53.153932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:08.679353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713517+0000","last_change":"2024-04-26T20:56:50.684034+0000","last_active":"2024-04-26T20:56:52.713517+0000","last_peered":"2024-04-26T20:56:52.713517+0000","last_clean":"2024-04-26T20:56:52.713517+0000","last_became_active":"2024-04-26T20:56:50.683720+0000","last_became_peered":"2024-04-26T20:56:50.683720+0000","last_unstale":"2024-04-26T20:56:52.713517+0000","last_undegraded":"2024-04-26T20:56:52.713517+0000","last_fullsized":"2024-04-26T20:56:52.713517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:46.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717068+0000","last_change":"2024-04-26T20:56:51.725602+0000","last_active":"2024-04-26T20:56:52.717068+0000","last_peered":"2024-04-26T20:56:52.717068+0000","last_clean":"2024-04-26T20:56:52.717068+0000","last_became_active":"2024-04-26T20:56:51.725308+0000","last_became_peered":"2024-04-26T20:56:51.725308+0000","last_unstale":"2024-04-26T20:56:52.717068+0000","last_undegraded":"2024-04-26T20:56:52.717068+0000","last_fullsized":"2024-04-26T20:56:52.717068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:52.330678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225216+0000","last_change":"2024-04-26T20:56:50.688464+0000","last_active":"2024-04-26T20:56:53.225216+0000","last_peered":"2024-04-26T20:56:53.225216+0000","last_clean":"2024-04-26T20:56:53.225216+0000","last_became_active":"2024-04-26T20:56:50.688101+0000","last_became_peered":"2024-04-26T20:56:50.688101+0000","last_unstale":"2024-04-26T20:56:53.225216+0000","last_undegraded":"2024-04-26T20:56:53.225216+0000","last_fullsized":"2024-04-26T20:56:53.225216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:50.526509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153386+0000","last_change":"2024-04-26T20:56:51.734773+0000","last_active":"2024-04-26T20:56:53.153386+0000","last_peered":"2024-04-26T20:56:53.153386+0000","last_clean":"2024-04-26T20:56:53.153386+0000","last_became_active":"2024-04-26T20:56:51.733903+0000","last_became_peered":"2024-04-26T20:56:51.733903+0000","last_unstale":"2024-04-26T20:56:53.153386+0000","last_undegraded":"2024-04-26T20:56:53.153386+0000","last_fullsized":"2024-04-26T20:56:53.153386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:23.450962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.152300+0000","last_change":"2024-04-26T20:56:46.848603+0000","last_active":"2024-04-26T20:56:53.152300+0000","last_peered":"2024-04-26T20:56:53.152300+0000","last_clean":"2024-04-26T20:56:53.152300+0000","last_became_active":"2024-04-26T20:56:46.846091+0000","last_became_peered":"2024-04-26T20:56:46.846091+0000","last_unstale":"2024-04-26T20:56:53.152300+0000","last_undegraded":"2024-04-26T20:56:53.152300+0000","last_fullsized":"2024-04-26T20:56:53.152300+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_clean_scrub_stamp":"2024-04-26T20:56:45.594041+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:00.232387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716618+0000","last_change":"2024-04-26T20:56:50.681942+0000","last_active":"2024-04-26T20:56:52.716618+0000","last_peered":"2024-04-26T20:56:52.716618+0000","last_clean":"2024-04-26T20:56:52.716618+0000","last_became_active":"2024-04-26T20:56:50.681645+0000","last_became_peered":"2024-04-26T20:56:50.681645+0000","last_unstale":"2024-04-26T20:56:52.716618+0000","last_undegraded":"2024-04-26T20:56:52.716618+0000","last_fullsized":"2024-04-26T20:56:52.716618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:39.328210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226904+0000","last_change":"2024-04-26T20:56:51.730048+0000","last_active":"2024-04-26T20:56:53.226904+0000","last_peered":"2024-04-26T20:56:53.226904+0000","last_clean":"2024-04-26T20:56:53.226904+0000","last_became_active":"2024-04-26T20:56:51.729733+0000","last_became_peered":"2024-04-26T20:56:51.729733+0000","last_unstale":"2024-04-26T20:56:53.226904+0000","last_undegraded":"2024-04-26T20:56:53.226904+0000","last_fullsized":"2024-04-26T20:56:53.226904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:24.679612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.152261+0000","last_change":"2024-04-26T20:56:50.699113+0000","last_active":"2024-04-26T20:56:53.152261+0000","last_peered":"2024-04-26T20:56:53.152261+0000","last_clean":"2024-04-26T20:56:53.152261+0000","last_became_active":"2024-04-26T20:56:50.698744+0000","last_became_peered":"2024-04-26T20:56:50.698744+0000","last_unstale":"2024-04-26T20:56:53.152261+0000","last_undegraded":"2024-04-26T20:56:53.152261+0000","last_fullsized":"2024-04-26T20:56:53.152261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:11.694549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144333+0000","last_change":"2024-04-26T20:56:51.732556+0000","last_active":"2024-04-26T20:56:53.144333+0000","last_peered":"2024-04-26T20:56:53.144333+0000","last_clean":"2024-04-26T20:56:53.144333+0000","last_became_active":"2024-04-26T20:56:51.732197+0000","last_became_peered":"2024-04-26T20:56:51.732197+0000","last_unstale":"2024-04-26T20:56:53.144333+0000","last_undegraded":"2024-04-26T20:56:53.144333+0000","last_fullsized":"2024-04-26T20:56:53.144333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:27:15.978359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225098+0000","last_change":"2024-04-26T20:56:50.694069+0000","last_active":"2024-04-26T20:56:53.225098+0000","last_peered":"2024-04-26T20:56:53.225098+0000","last_clean":"2024-04-26T20:56:53.225098+0000","last_became_active":"2024-04-26T20:56:50.693779+0000","last_became_peered":"2024-04-26T20:56:50.693779+0000","last_unstale":"2024-04-26T20:56:53.225098+0000","last_undegraded":"2024-04-26T20:56:53.225098+0000","last_fullsized":"2024-04-26T20:56:53.225098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:31.999414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.714833+0000","last_change":"2024-04-26T20:56:51.710866+0000","last_active":"2024-04-26T20:56:52.714833+0000","last_peered":"2024-04-26T20:56:52.714833+0000","last_clean":"2024-04-26T20:56:52.714833+0000","last_became_active":"2024-04-26T20:56:51.709325+0000","last_became_peered":"2024-04-26T20:56:51.709325+0000","last_unstale":"2024-04-26T20:56:52.714833+0000","last_undegraded":"2024-04-26T20:56:52.714833+0000","last_fullsized":"2024-04-26T20:56:52.714833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:23.268496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717176+0000","last_change":"2024-04-26T20:56:50.698231+0000","last_active":"2024-04-26T20:56:52.717176+0000","last_peered":"2024-04-26T20:56:52.717176+0000","last_clean":"2024-04-26T20:56:52.717176+0000","last_became_active":"2024-04-26T20:56:50.697872+0000","last_became_peered":"2024-04-26T20:56:50.697872+0000","last_unstale":"2024-04-26T20:56:52.717176+0000","last_undegraded":"2024-04-26T20:56:52.717176+0000","last_fullsized":"2024-04-26T20:56:52.717176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:41.662331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.715920+0000","last_change":"2024-04-26T20:56:51.720748+0000","last_active":"2024-04-26T20:56:52.715920+0000","last_peered":"2024-04-26T20:56:52.715920+0000","last_clean":"2024-04-26T20:56:52.715920+0000","last_became_active":"2024-04-26T20:56:51.720478+0000","last_became_peered":"2024-04-26T20:56:51.720478+0000","last_unstale":"2024-04-26T20:56:52.715920+0000","last_undegraded":"2024-04-26T20:56:52.715920+0000","last_fullsized":"2024-04-26T20:56:52.715920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:26.791119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713396+0000","last_change":"2024-04-26T20:56:50.685444+0000","last_active":"2024-04-26T20:56:52.713396+0000","last_peered":"2024-04-26T20:56:52.713396+0000","last_clean":"2024-04-26T20:56:52.713396+0000","last_became_active":"2024-04-26T20:56:50.684937+0000","last_became_peered":"2024-04-26T20:56:50.684937+0000","last_unstale":"2024-04-26T20:56:52.713396+0000","last_undegraded":"2024-04-26T20:56:52.713396+0000","last_fullsized":"2024-04-26T20:56:52.713396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:39.328210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717440+0000","last_change":"2024-04-26T20:56:51.727132+0000","last_active":"2024-04-26T20:56:52.717440+0000","last_peered":"2024-04-26T20:56:52.717440+0000","last_clean":"2024-04-26T20:56:52.717440+0000","last_became_active":"2024-04-26T20:56:51.720623+0000","last_became_peered":"2024-04-26T20:56:51.720623+0000","last_unstale":"2024-04-26T20:56:52.717440+0000","last_undegraded":"2024-04-26T20:56:52.717440+0000","last_fullsized":"2024-04-26T20:56:52.717440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:06.270681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153818+0000","last_change":"2024-04-26T20:56:50.693929+0000","last_active":"2024-04-26T20:56:53.153818+0000","last_peered":"2024-04-26T20:56:53.153818+0000","last_clean":"2024-04-26T20:56:53.153818+0000","last_became_active":"2024-04-26T20:56:50.693576+0000","last_became_peered":"2024-04-26T20:56:50.693576+0000","last_unstale":"2024-04-26T20:56:53.153818+0000","last_undegraded":"2024-04-26T20:56:53.153818+0000","last_fullsized":"2024-04-26T20:56:53.153818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:46.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713243+0000","last_change":"2024-04-26T20:56:51.714289+0000","last_active":"2024-04-26T20:56:52.713243+0000","last_peered":"2024-04-26T20:56:52.713243+0000","last_clean":"2024-04-26T20:56:52.713243+0000","last_became_active":"2024-04-26T20:56:51.713970+0000","last_became_peered":"2024-04-26T20:56:51.713970+0000","last_unstale":"2024-04-26T20:56:52.713243+0000","last_undegraded":"2024-04-26T20:56:52.713243+0000","last_fullsized":"2024-04-26T20:56:52.713243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:24.679612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.728669+0000","last_change":"2024-04-26T20:56:50.690572+0000","last_active":"2024-04-26T20:56:52.728669+0000","last_peered":"2024-04-26T20:56:52.728669+0000","last_clean":"2024-04-26T20:56:52.728669+0000","last_became_active":"2024-04-26T20:56:50.690221+0000","last_became_peered":"2024-04-26T20:56:50.690221+0000","last_unstale":"2024-04-26T20:56:52.728669+0000","last_undegraded":"2024-04-26T20:56:52.728669+0000","last_fullsized":"2024-04-26T20:56:52.728669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:11.694549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.227813+0000","last_change":"2024-04-26T20:56:51.740978+0000","last_active":"2024-04-26T20:56:53.227813+0000","last_peered":"2024-04-26T20:56:53.227813+0000","last_clean":"2024-04-26T20:56:53.227813+0000","last_became_active":"2024-04-26T20:56:51.739561+0000","last_became_peered":"2024-04-26T20:56:51.739561+0000","last_unstale":"2024-04-26T20:56:53.227813+0000","last_undegraded":"2024-04-26T20:56:53.227813+0000","last_fullsized":"2024-04-26T20:56:53.227813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:47.662176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730672+0000","last_change":"2024-04-26T20:56:50.694545+0000","last_active":"2024-04-26T20:56:52.730672+0000","last_peered":"2024-04-26T20:56:52.730672+0000","last_clean":"2024-04-26T20:56:52.730672+0000","last_became_active":"2024-04-26T20:56:50.694189+0000","last_became_peered":"2024-04-26T20:56:50.694189+0000","last_unstale":"2024-04-26T20:56:52.730672+0000","last_undegraded":"2024-04-26T20:56:52.730672+0000","last_fullsized":"2024-04-26T20:56:52.730672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:25.776704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144945+0000","last_change":"2024-04-26T20:56:51.724210+0000","last_active":"2024-04-26T20:56:53.144945+0000","last_peered":"2024-04-26T20:56:53.144945+0000","last_clean":"2024-04-26T20:56:53.144945+0000","last_became_active":"2024-04-26T20:56:51.723882+0000","last_became_peered":"2024-04-26T20:56:51.723882+0000","last_unstale":"2024-04-26T20:56:53.144945+0000","last_undegraded":"2024-04-26T20:56:53.144945+0000","last_fullsized":"2024-04-26T20:56:53.144945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:53.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.152108+0000","last_change":"2024-04-26T20:56:50.694039+0000","last_active":"2024-04-26T20:56:53.152108+0000","last_peered":"2024-04-26T20:56:53.152108+0000","last_clean":"2024-04-26T20:56:53.152108+0000","last_became_active":"2024-04-26T20:56:50.693569+0000","last_became_peered":"2024-04-26T20:56:50.693569+0000","last_unstale":"2024-04-26T20:56:53.152108+0000","last_undegraded":"2024-04-26T20:56:53.152108+0000","last_fullsized":"2024-04-26T20:56:53.152108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:41.662331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.714862+0000","last_change":"2024-04-26T20:56:51.719531+0000","last_active":"2024-04-26T20:56:52.714862+0000","last_peered":"2024-04-26T20:56:52.714862+0000","last_clean":"2024-04-26T20:56:52.714862+0000","last_became_active":"2024-04-26T20:56:51.714881+0000","last_became_peered":"2024-04-26T20:56:51.714881+0000","last_unstale":"2024-04-26T20:56:52.714862+0000","last_undegraded":"2024-04-26T20:56:52.714862+0000","last_fullsized":"2024-04-26T20:56:52.714862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:33.013829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713174+0000","last_change":"2024-04-26T20:56:50.689750+0000","last_active":"2024-04-26T20:56:52.713174+0000","last_peered":"2024-04-26T20:56:52.713174+0000","last_clean":"2024-04-26T20:56:52.713174+0000","last_became_active":"2024-04-26T20:56:50.689337+0000","last_became_peered":"2024-04-26T20:56:50.689337+0000","last_unstale":"2024-04-26T20:56:52.713174+0000","last_undegraded":"2024-04-26T20:56:52.713174+0000","last_fullsized":"2024-04-26T20:56:52.713174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:41.662331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226387+0000","last_change":"2024-04-26T20:56:51.712295+0000","last_active":"2024-04-26T20:56:53.226387+0000","last_peered":"2024-04-26T20:56:53.226387+0000","last_clean":"2024-04-26T20:56:53.226387+0000","last_became_active":"2024-04-26T20:56:51.711999+0000","last_became_peered":"2024-04-26T20:56:51.711999+0000","last_unstale":"2024-04-26T20:56:53.226387+0000","last_undegraded":"2024-04-26T20:56:53.226387+0000","last_fullsized":"2024-04-26T20:56:53.226387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:16.275518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713352+0000","last_change":"2024-04-26T20:56:50.697160+0000","last_active":"2024-04-26T20:56:52.713352+0000","last_peered":"2024-04-26T20:56:52.713352+0000","last_clean":"2024-04-26T20:56:52.713352+0000","last_became_active":"2024-04-26T20:56:50.695595+0000","last_became_peered":"2024-04-26T20:56:50.695595+0000","last_unstale":"2024-04-26T20:56:52.713352+0000","last_undegraded":"2024-04-26T20:56:52.713352+0000","last_fullsized":"2024-04-26T20:56:52.713352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:50.526509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226926+0000","last_change":"2024-04-26T20:56:51.717706+0000","last_active":"2024-04-26T20:56:53.226926+0000","last_peered":"2024-04-26T20:56:53.226926+0000","last_clean":"2024-04-26T20:56:53.226926+0000","last_became_active":"2024-04-26T20:56:51.717422+0000","last_became_peered":"2024-04-26T20:56:51.717422+0000","last_unstale":"2024-04-26T20:56:53.226926+0000","last_undegraded":"2024-04-26T20:56:53.226926+0000","last_fullsized":"2024-04-26T20:56:53.226926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:37.778325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153433+0000","last_change":"2024-04-26T20:56:50.695179+0000","last_active":"2024-04-26T20:56:53.153433+0000","last_peered":"2024-04-26T20:56:53.153433+0000","last_clean":"2024-04-26T20:56:53.153433+0000","last_became_active":"2024-04-26T20:56:50.694889+0000","last_became_peered":"2024-04-26T20:56:50.694889+0000","last_unstale":"2024-04-26T20:56:53.153433+0000","last_undegraded":"2024-04-26T20:56:53.153433+0000","last_fullsized":"2024-04-26T20:56:53.153433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:50.526509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730113+0000","last_change":"2024-04-26T20:56:51.713029+0000","last_active":"2024-04-26T20:56:52.730113+0000","last_peered":"2024-04-26T20:56:52.730113+0000","last_clean":"2024-04-26T20:56:52.730113+0000","last_became_active":"2024-04-26T20:56:51.712311+0000","last_became_peered":"2024-04-26T20:56:51.712311+0000","last_unstale":"2024-04-26T20:56:52.730113+0000","last_undegraded":"2024-04-26T20:56:52.730113+0000","last_fullsized":"2024-04-26T20:56:52.730113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:36.354765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717262+0000","last_change":"2024-04-26T20:56:51.730361+0000","last_active":"2024-04-26T20:56:52.717262+0000","last_peered":"2024-04-26T20:56:52.717262+0000","last_clean":"2024-04-26T20:56:52.717262+0000","last_became_active":"2024-04-26T20:56:51.730068+0000","last_became_peered":"2024-04-26T20:56:51.730068+0000","last_unstale":"2024-04-26T20:56:52.717262+0000","last_undegraded":"2024-04-26T20:56:52.717262+0000","last_fullsized":"2024-04-26T20:56:52.717262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:07.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153083+0000","last_change":"2024-04-26T20:56:50.696973+0000","last_active":"2024-04-26T20:56:53.153083+0000","last_peered":"2024-04-26T20:56:53.153083+0000","last_clean":"2024-04-26T20:56:53.153083+0000","last_became_active":"2024-04-26T20:56:50.696603+0000","last_became_peered":"2024-04-26T20:56:50.696603+0000","last_unstale":"2024-04-26T20:56:53.153083+0000","last_undegraded":"2024-04-26T20:56:53.153083+0000","last_fullsized":"2024-04-26T20:56:53.153083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:25.776704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717108+0000","last_change":"2024-04-26T20:56:51.728709+0000","last_active":"2024-04-26T20:56:52.717108+0000","last_peered":"2024-04-26T20:56:52.717108+0000","last_clean":"2024-04-26T20:56:52.717108+0000","last_became_active":"2024-04-26T20:56:51.727858+0000","last_became_peered":"2024-04-26T20:56:51.727858+0000","last_unstale":"2024-04-26T20:56:52.717108+0000","last_undegraded":"2024-04-26T20:56:52.717108+0000","last_fullsized":"2024-04-26T20:56:52.717108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:23.450962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153531+0000","last_change":"2024-04-26T20:56:50.696966+0000","last_active":"2024-04-26T20:56:53.153531+0000","last_peered":"2024-04-26T20:56:53.153531+0000","last_clean":"2024-04-26T20:56:53.153531+0000","last_became_active":"2024-04-26T20:56:50.696448+0000","last_became_peered":"2024-04-26T20:56:50.696448+0000","last_unstale":"2024-04-26T20:56:53.153531+0000","last_undegraded":"2024-04-26T20:56:53.153531+0000","last_fullsized":"2024-04-26T20:56:53.153531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:39.328210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144675+0000","last_change":"2024-04-26T20:56:51.731938+0000","last_active":"2024-04-26T20:56:53.144675+0000","last_peered":"2024-04-26T20:56:53.144675+0000","last_clean":"2024-04-26T20:56:53.144675+0000","last_became_active":"2024-04-26T20:56:51.731655+0000","last_became_peered":"2024-04-26T20:56:51.731655+0000","last_unstale":"2024-04-26T20:56:53.144675+0000","last_undegraded":"2024-04-26T20:56:53.144675+0000","last_fullsized":"2024-04-26T20:56:53.144675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:18.631055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.227678+0000","last_change":"2024-04-26T20:56:50.692059+0000","last_active":"2024-04-26T20:56:53.227678+0000","last_peered":"2024-04-26T20:56:53.227678+0000","last_clean":"2024-04-26T20:56:53.227678+0000","last_became_active":"2024-04-26T20:56:50.690628+0000","last_became_peered":"2024-04-26T20:56:50.690628+0000","last_unstale":"2024-04-26T20:56:53.227678+0000","last_undegraded":"2024-04-26T20:56:53.227678+0000","last_fullsized":"2024-04-26T20:56:53.227678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:00.125994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.227526+0000","last_change":"2024-04-26T20:56:51.719241+0000","last_active":"2024-04-26T20:56:53.227526+0000","last_peered":"2024-04-26T20:56:53.227526+0000","last_clean":"2024-04-26T20:56:53.227526+0000","last_became_active":"2024-04-26T20:56:51.718893+0000","last_became_peered":"2024-04-26T20:56:51.718893+0000","last_unstale":"2024-04-26T20:56:53.227526+0000","last_undegraded":"2024-04-26T20:56:53.227526+0000","last_fullsized":"2024-04-26T20:56:53.227526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:23.268496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144835+0000","last_change":"2024-04-26T20:56:50.691078+0000","last_active":"2024-04-26T20:56:53.144835+0000","last_peered":"2024-04-26T20:56:53.144835+0000","last_clean":"2024-04-26T20:56:53.144835+0000","last_became_active":"2024-04-26T20:56:50.689362+0000","last_became_peered":"2024-04-26T20:56:50.689362+0000","last_unstale":"2024-04-26T20:56:53.144835+0000","last_undegraded":"2024-04-26T20:56:53.144835+0000","last_fullsized":"2024-04-26T20:56:53.144835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:00.125994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":204800,"data_stored":69305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":811008,"data_stored":660311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:56:56.656 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T20:56:56.656 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T20:56:56.656 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-26T20:56:56.656 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-26T20:56:56.657 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-26T20:56:56.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.896+0000 7fee33200640 1 Processor -- start 2024-04-26T20:56:56.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.896+0000 7fee33200640 1 -- start start 2024-04-26T20:56:56.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.898+0000 7fee33200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fee2c15f1b0 0x7fee2c0d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:56.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.898+0000 7fee33200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee2c0d0710 con 0x7fee2c0cf700 2024-04-26T20:56:56.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.898+0000 7fee33200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee2c0d0150 con 0x7fee2c0d5540 2024-04-26T20:56:56.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.898+0000 7fee33200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fee2c0cff00 con 0x7fee2c15f1b0 2024-04-26T20:56:56.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.898+0000 7fee31400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fee2c0cf700 0x7fee2c0d4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57714/0 (socket says 172.21.15.28:57714) 2024-04-26T20:56:56.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.898+0000 7fee31400640 1 -- 172.21.15.28:0/1117778558 learned_addr learned my addr 172.21.15.28:0/1117778558 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:56.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.899+0000 7fee31e00640 1 --2- 172.21.15.28:0/1117778558 >> v2:172.21.15.103:3300/0 conn(0x7fee2c15f1b0 0x7fee2c0d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:56.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.899+0000 7fee30a00640 1 -- 172.21.15.28:0/1117778558 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1963132080 0 0) 0x7fee1c001110 con 0x7fee2c0cf700 2024-04-26T20:56:56.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.899+0000 7fee30a00640 1 -- 172.21.15.28:0/1117778558 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee14003520 con 0x7fee2c0cf700 2024-04-26T20:56:56.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.899+0000 7fee30a00640 1 -- 172.21.15.28:0/1117778558 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2782381866 0 0) 0x7fee14003520 con 0x7fee2c0cf700 2024-04-26T20:56:56.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.899+0000 7fee30a00640 1 -- 172.21.15.28:0/1117778558 >> v2:172.21.15.103:3300/0 conn(0x7fee2c15f1b0 msgr2=0x7fee2c0d4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:56.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.899+0000 7fee30a00640 1 --2- 172.21.15.28:0/1117778558 >> v2:172.21.15.103:3300/0 conn(0x7fee2c15f1b0 0x7fee2c0d4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:56.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.899+0000 7fee30a00640 1 -- 172.21.15.28:0/1117778558 >> v1:172.21.15.103:6789/0 conn(0x7fee2c0d5540 legacy=0x7fee2c0b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:56.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.899+0000 7fee30a00640 1 -- 172.21.15.28:0/1117778558 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2c0d0430 con 0x7fee2c0cf700 2024-04-26T20:56:56.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.899+0000 7fee30a00640 1 -- 172.21.15.28:0/1117778558 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fee1c001840 con 0x7fee2c0cf700 2024-04-26T20:56:56.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.900+0000 7fee30a00640 1 -- 172.21.15.28:0/1117778558 >> v1:172.21.15.28:6789/0 conn(0x7fee2c0cf700 legacy=0x7fee2c0d4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:56.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.900+0000 7fee30a00640 1 --2- 172.21.15.28:0/1117778558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee140039b0 0x7fee14003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:56.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.900+0000 7fee30a00640 1 --2- 172.21.15.28:0/1117778558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee14004800 0x7fee14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:56.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.900+0000 7fee30a00640 1 --2- 172.21.15.28:0/1117778558 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee140051c0 0x7fee1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:56.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.900+0000 7fee30a00640 1 -- 172.21.15.28:0/1117778558 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c0cff00 con 0x7fee14004800 2024-04-26T20:56:56.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.900+0000 7fee30a00640 1 -- 172.21.15.28:0/1117778558 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee1400c070 con 0x7fee140039b0 2024-04-26T20:56:56.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.900+0000 7fee30a00640 1 -- 172.21.15.28:0/1117778558 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fee1400c270 con 0x7fee140051c0 2024-04-26T20:56:56.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.900+0000 7fee31e00640 1 --2- 172.21.15.28:0/1117778558 >> v2:172.21.15.103:3300/0 conn(0x7fee2c15f1b0 0x7fee2c0d4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:56:56.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.900+0000 7fee31400640 1 --2- 172.21.15.28:0/1117778558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee14004800 0x7fee14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:56.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.900+0000 7fee32800640 1 --2- 172.21.15.28:0/1117778558 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee140051c0 0x7fee1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:56.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.900+0000 7fee31400640 1 -- 172.21.15.28:0/1117778558 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee140051c0 msgr2=0x7fee1400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:56.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.900+0000 7fee31400640 1 --2- 172.21.15.28:0/1117778558 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee140051c0 0x7fee1400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:56.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.900+0000 7fee31400640 1 -- 172.21.15.28:0/1117778558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee140039b0 msgr2=0x7fee14003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:56.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.900+0000 7fee31400640 1 --2- 172.21.15.28:0/1117778558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee140039b0 0x7fee14003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:56.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.900+0000 7fee31400640 1 -- 172.21.15.28:0/1117778558 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee14003520 con 0x7fee14004800 2024-04-26T20:56:56.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.901+0000 7fee32800640 1 --2- 172.21.15.28:0/1117778558 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee140051c0 0x7fee1400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:56:56.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.901+0000 7fee32800640 1 -- 172.21.15.28:0/1117778558 reap_dead start 2024-04-26T20:56:56.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.901+0000 7fee31400640 1 --2- 172.21.15.28:0/1117778558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee14004800 0x7fee14004c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00c4c0 tx=0x7fee14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3272b8eac9ca6bfa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:56.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.902+0000 7fee30a00640 1 -- 172.21.15.28:0/1117778558 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c00c0e0 con 0x7fee14004800 2024-04-26T20:56:56.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.902+0000 7fee30a00640 1 -- 172.21.15.28:0/1117778558 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee1c034070 con 0x7fee14004800 2024-04-26T20:56:56.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.902+0000 7fee30a00640 1 -- 172.21.15.28:0/1117778558 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c005d70 con 0x7fee14004800 2024-04-26T20:56:56.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.902+0000 7fee33200640 1 -- 172.21.15.28:0/1117778558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee14004800 msgr2=0x7fee14004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:56.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.902+0000 7fee33200640 1 --2- 172.21.15.28:0/1117778558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee14004800 0x7fee14004c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fee1c00c4c0 tx=0x7fee14005830 comp rx=0 tx=0).stop 2024-04-26T20:56:56.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.902+0000 7fee33200640 1 -- 172.21.15.28:0/1117778558 shutdown_connections 2024-04-26T20:56:56.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.902+0000 7fee33200640 1 --2- 172.21.15.28:0/1117778558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee14004800 0x7fee14004c80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:56.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.902+0000 7fee33200640 1 -- 172.21.15.28:0/1117778558 >> 172.21.15.28:0/1117778558 conn(0x7fee2c06c770 msgr2=0x7fee2c06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:56.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.902+0000 7fee33200640 1 -- 172.21.15.28:0/1117778558 shutdown_connections 2024-04-26T20:56:56.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.902+0000 7fee33200640 1 -- 172.21.15.28:0/1117778558 wait complete. 2024-04-26T20:56:56.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.904+0000 7fee33200640 1 Processor -- start 2024-04-26T20:56:56.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.904+0000 7fee33200640 1 -- start start 2024-04-26T20:56:56.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.905+0000 7fee33200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee14004800 0x7fee2c0cb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:56.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.905+0000 7fee33200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee2c161e70 0x7fee2c1622f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:56.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.905+0000 7fee33200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee2c0ce0c0 0x7fee2c0ce540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:56.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.905+0000 7fee33200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c15f590 con 0x7fee2c0ce0c0 2024-04-26T20:56:56.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.905+0000 7fee33200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c0d4270 con 0x7fee14004800 2024-04-26T20:56:56.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.905+0000 7fee33200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fee2c0b4410 con 0x7fee2c161e70 2024-04-26T20:56:56.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.906+0000 7fee32800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee2c0ce0c0 0x7fee2c0ce540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:56.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.906+0000 7fee32800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee2c0ce0c0 0x7fee2c0ce540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42080/0 (socket says 172.21.15.28:42080) 2024-04-26T20:56:56.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.906+0000 7fee32800640 1 -- 172.21.15.28:0/4137465920 learned_addr learned my addr 172.21.15.28:0/4137465920 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:56.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.906+0000 7fee31e00640 1 --2- 172.21.15.28:0/4137465920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee14004800 0x7fee2c0cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:56.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.906+0000 7fee31400640 1 --2- 172.21.15.28:0/4137465920 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee2c161e70 0x7fee2c1622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:56.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.906+0000 7fee32800640 1 -- 172.21.15.28:0/4137465920 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee2c161e70 msgr2=0x7fee2c1622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:56.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.906+0000 7fee32800640 1 --2- 172.21.15.28:0/4137465920 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee2c161e70 0x7fee2c1622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:56.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.906+0000 7fee32800640 1 -- 172.21.15.28:0/4137465920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee14004800 msgr2=0x7fee2c0cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:56.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.906+0000 7fee32800640 1 --2- 172.21.15.28:0/4137465920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee14004800 0x7fee2c0cb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:56.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.906+0000 7fee32800640 1 -- 172.21.15.28:0/4137465920 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2c0cd060 con 0x7fee2c0ce0c0 2024-04-26T20:56:56.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.906+0000 7fee32800640 1 --2- 172.21.15.28:0/4137465920 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee2c0ce0c0 0x7fee2c0ce540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fee2c162c70 tx=0x7fee2800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:56.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.907+0000 7fee26000640 1 -- 172.21.15.28:0/4137465920 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28018070 con 0x7fee2c0ce0c0 2024-04-26T20:56:56.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.907+0000 7fee26000640 1 -- 172.21.15.28:0/4137465920 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee28003b90 con 0x7fee2c0ce0c0 2024-04-26T20:56:56.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.907+0000 7fee26000640 1 -- 172.21.15.28:0/4137465920 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28005070 con 0x7fee2c0ce0c0 2024-04-26T20:56:56.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.907+0000 7fee33200640 1 -- 172.21.15.28:0/4137465920 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee2c0cd300 con 0x7fee2c0ce0c0 2024-04-26T20:56:56.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.907+0000 7fee33200640 1 -- 172.21.15.28:0/4137465920 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee2c0cd8c0 con 0x7fee2c0ce0c0 2024-04-26T20:56:56.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.908+0000 7fee33200640 1 -- 172.21.15.28:0/4137465920 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee2c06a360 con 0x7fee2c0ce0c0 2024-04-26T20:56:56.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.909+0000 7fee26000640 1 -- 172.21.15.28:0/4137465920 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee2801b070 con 0x7fee2c0ce0c0 2024-04-26T20:56:56.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.910+0000 7fee26000640 1 --2- 172.21.15.28:0/4137465920 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fee1006c8f0 0x7fee1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:56.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.910+0000 7fee26000640 1 -- 172.21.15.28:0/4137465920 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fee2808b340 con 0x7fee2c0ce0c0 2024-04-26T20:56:56.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.910+0000 7fee31e00640 1 --2- 172.21.15.28:0/4137465920 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fee1006c8f0 0x7fee1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:56.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.911+0000 7fee31e00640 1 --2- 172.21.15.28:0/4137465920 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fee1006c8f0 0x7fee1006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fee2c06af40 tx=0x7fee20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:56.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:56.914+0000 7fee26000640 1 -- 172.21.15.28:0/4137465920 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fee280596d0 con 0x7fee2c0ce0c0 2024-04-26T20:56:57.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.163+0000 7fee33200640 1 -- 172.21.15.28:0/4137465920 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fee2c0b67b0 con 0x7fee2c0ce0c0 2024-04-26T20:56:57.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.167+0000 7fee26000640 1 -- 172.21.15.28:0/4137465920 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174137 (secure 0 0 0) 0x7fee28058fd0 con 0x7fee2c0ce0c0 2024-04-26T20:56:57.169 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:56:57.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.172+0000 7fee33200640 1 -- 172.21.15.28:0/4137465920 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fee1006c8f0 msgr2=0x7fee1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:57.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.172+0000 7fee33200640 1 --2- 172.21.15.28:0/4137465920 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fee1006c8f0 0x7fee1006edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fee2c06af40 tx=0x7fee20008040 comp rx=0 tx=0).stop 2024-04-26T20:56:57.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.172+0000 7fee33200640 1 -- 172.21.15.28:0/4137465920 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee2c0ce0c0 msgr2=0x7fee2c0ce540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:57.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.172+0000 7fee33200640 1 --2- 172.21.15.28:0/4137465920 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee2c0ce0c0 0x7fee2c0ce540 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fee2c162c70 tx=0x7fee2800dc00 comp rx=0 tx=0).stop 2024-04-26T20:56:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.172+0000 7fee33200640 1 -- 172.21.15.28:0/4137465920 shutdown_connections 2024-04-26T20:56:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.172+0000 7fee33200640 1 --2- 172.21.15.28:0/4137465920 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fee1006c8f0 0x7fee1006edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.172+0000 7fee33200640 1 --2- 172.21.15.28:0/4137465920 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee2c0ce0c0 0x7fee2c0ce540 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.172+0000 7fee33200640 1 --2- 172.21.15.28:0/4137465920 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee2c161e70 0x7fee2c1622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.172+0000 7fee33200640 1 --2- 172.21.15.28:0/4137465920 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee14004800 0x7fee2c0cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.172+0000 7fee33200640 1 -- 172.21.15.28:0/4137465920 >> 172.21.15.28:0/4137465920 conn(0x7fee2c06c770 msgr2=0x7fee2c06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.172+0000 7fee33200640 1 -- 172.21.15.28:0/4137465920 shutdown_connections 2024-04-26T20:56:57.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.172+0000 7fee33200640 1 -- 172.21.15.28:0/4137465920 wait complete. 2024-04-26T20:56:57.191 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":2367459182},{"type":"v1","addr":"172.21.15.28:6833","nonce":2367459182}]},"active_addr":"172.21.15.28:6833/2367459182","active_change":"2024-04-26T20:56:36.678023+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4109,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":3500231754}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":2074672144}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":2408411264}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":4202999399}]}]} 2024-04-26T20:56:57.195 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-26T20:56:57.195 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-26T20:56:57.195 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:56:57.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.402+0000 7f8b0fc00640 1 Processor -- start 2024-04-26T20:56:57.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.402+0000 7f8b0fc00640 1 -- start start 2024-04-26T20:56:57.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.403+0000 7f8b0fc00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f8b0815f420 0x7f8b08162ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:57.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.403+0000 7f8b0fc00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b080bad60 0x7f8b080cfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:57.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.403+0000 7f8b0fc00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b0815f800 con 0x7f8b080d0420 2024-04-26T20:56:57.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.403+0000 7f8b0fc00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f8b08163740 con 0x7f8b0815f420 2024-04-26T20:56:57.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.403+0000 7f8b0fc00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f8b08163a20 con 0x7f8b080bad60 2024-04-26T20:56:57.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.404+0000 7f8b0dc00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8b080d0420 0x7f8b0806c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57728/0 (socket says 172.21.15.28:57728) 2024-04-26T20:56:57.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.404+0000 7f8b0dc00640 1 -- 172.21.15.28:0/575025004 learned_addr learned my addr 172.21.15.28:0/575025004 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:57.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.404+0000 7f8b0d200640 1 --2- 172.21.15.28:0/575025004 >> v2:172.21.15.103:3300/0 conn(0x7f8b0815f420 0x7f8b08162ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:57.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.404+0000 7f8b07e00640 1 --2- 172.21.15.28:0/575025004 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b080bad60 0x7f8b080cfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:57.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.404+0000 7f8b07400640 1 -- 172.21.15.28:0/575025004 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3372548877 0 0) 0x7f8b0815f800 con 0x7f8b080d0420 2024-04-26T20:56:57.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.404+0000 7f8b07400640 1 -- 172.21.15.28:0/575025004 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8aec003520 con 0x7f8b080d0420 2024-04-26T20:56:57.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.404+0000 7f8b0d200640 1 -- 172.21.15.28:0/575025004 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b080bad60 msgr2=0x7f8b080cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:57.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.404+0000 7f8b07400640 1 -- 172.21.15.28:0/575025004 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4127894592 0 0) 0x7f8aec003520 con 0x7f8b080d0420 2024-04-26T20:56:57.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.404+0000 7f8b0d200640 1 --2- 172.21.15.28:0/575025004 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b080bad60 0x7f8b080cfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:57.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.404+0000 7f8b0d200640 1 -- 172.21.15.28:0/575025004 >> v1:172.21.15.28:6789/0 conn(0x7f8b080d0420 legacy=0x7f8b0806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:57.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.405+0000 7f8b0d200640 1 -- 172.21.15.28:0/575025004 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b08163d00 con 0x7f8b0815f420 2024-04-26T20:56:57.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.406+0000 7f8b0d200640 1 --2- 172.21.15.28:0/575025004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0815f420 0x7f8b08162ef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8af8009d90 tx=0x7f8af802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab1ae61eb98add04 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:57.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.406+0000 7f8b07400640 1 -- 172.21.15.28:0/575025004 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b00001670 con 0x7f8b0815f420 2024-04-26T20:56:57.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.406+0000 7f8b07400640 1 -- 172.21.15.28:0/575025004 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af8007540 con 0x7f8b0815f420 2024-04-26T20:56:57.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.406+0000 7f8b07400640 1 -- 172.21.15.28:0/575025004 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af8007910 con 0x7f8b0815f420 2024-04-26T20:56:57.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.407+0000 7f8b0fc00640 1 -- 172.21.15.28:0/575025004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0815f420 msgr2=0x7f8b08162ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:57.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.407+0000 7f8b0fc00640 1 --2- 172.21.15.28:0/575025004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0815f420 0x7f8b08162ef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8af8009d90 tx=0x7f8af802a680 comp rx=0 tx=0).stop 2024-04-26T20:56:57.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.407+0000 7f8b0fc00640 1 -- 172.21.15.28:0/575025004 shutdown_connections 2024-04-26T20:56:57.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.407+0000 7f8b0fc00640 1 --2- 172.21.15.28:0/575025004 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b080bad60 0x7f8b080cfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:57.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.407+0000 7f8b0fc00640 1 --2- 172.21.15.28:0/575025004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0815f420 0x7f8b08162ef0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:57.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.407+0000 7f8b0fc00640 1 -- 172.21.15.28:0/575025004 >> 172.21.15.28:0/575025004 conn(0x7f8b080b6c00 msgr2=0x7f8b080b7010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:57.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.407+0000 7f8b0fc00640 1 -- 172.21.15.28:0/575025004 shutdown_connections 2024-04-26T20:56:57.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.407+0000 7f8b0fc00640 1 -- 172.21.15.28:0/575025004 wait complete. 2024-04-26T20:56:57.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.408+0000 7f8b0fc00640 1 Processor -- start 2024-04-26T20:56:57.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.409+0000 7f8b0fc00640 1 -- start start 2024-04-26T20:56:57.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.409+0000 7f8b0fc00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b080bad60 0x7f8b081565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:57.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.410+0000 7f8b0fc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b080d0420 0x7f8b081f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:57.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.410+0000 7f8b0fc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0815f420 0x7f8b081608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:57.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.410+0000 7f8b0fc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8b08163430 con 0x7f8b080d0420 2024-04-26T20:56:57.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.410+0000 7f8b0fc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8b080d37c0 con 0x7f8b0815f420 2024-04-26T20:56:57.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.410+0000 7f8b0fc00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f8b080b8240 con 0x7f8b080bad60 2024-04-26T20:56:57.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.410+0000 7f8b07e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b080d0420 0x7f8b081f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:57.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.410+0000 7f8b07e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b080d0420 0x7f8b081f5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42082/0 (socket says 172.21.15.28:42082) 2024-04-26T20:56:57.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.410+0000 7f8b07e00640 1 -- 172.21.15.28:0/1421666281 learned_addr learned my addr 172.21.15.28:0/1421666281 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:57.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.410+0000 7f8b0d200640 1 --2- 172.21.15.28:0/1421666281 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b080bad60 0x7f8b081565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:57.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.410+0000 7f8b07e00640 1 -- 172.21.15.28:0/1421666281 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b080bad60 msgr2=0x7f8b081565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:57.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.410+0000 7f8b07e00640 1 --2- 172.21.15.28:0/1421666281 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b080bad60 0x7f8b081565c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:57.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.410+0000 7f8b0dc00640 1 --2- 172.21.15.28:0/1421666281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0815f420 0x7f8b081608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:57.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.410+0000 7f8b07e00640 1 -- 172.21.15.28:0/1421666281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0815f420 msgr2=0x7f8b081608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:57.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.410+0000 7f8b07e00640 1 --2- 172.21.15.28:0/1421666281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0815f420 0x7f8b081608c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:57.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.410+0000 7f8b07e00640 1 -- 172.21.15.28:0/1421666281 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b08161360 con 0x7f8b080d0420 2024-04-26T20:56:57.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.411+0000 7f8b07e00640 1 --2- 172.21.15.28:0/1421666281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b080d0420 0x7f8b081f5170 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8af000e9b0 tx=0x7f8af000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:57.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.411+0000 7f8b04c00640 1 -- 172.21.15.28:0/1421666281 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af0017070 con 0x7f8b080d0420 2024-04-26T20:56:57.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.411+0000 7f8b04c00640 1 -- 172.21.15.28:0/1421666281 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8af000c9a0 con 0x7f8b080d0420 2024-04-26T20:56:57.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.411+0000 7f8b04c00640 1 -- 172.21.15.28:0/1421666281 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8af000cd90 con 0x7f8b080d0420 2024-04-26T20:56:57.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.411+0000 7f8b0fc00640 1 -- 172.21.15.28:0/1421666281 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b080ca2a0 con 0x7f8b080d0420 2024-04-26T20:56:57.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.411+0000 7f8b0fc00640 1 -- 172.21.15.28:0/1421666281 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b080ca860 con 0x7f8b080d0420 2024-04-26T20:56:57.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.412+0000 7f8b0fc00640 1 -- 172.21.15.28:0/1421666281 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b0806a400 con 0x7f8b080d0420 2024-04-26T20:56:57.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.413+0000 7f8b04c00640 1 -- 172.21.15.28:0/1421666281 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8af001a070 con 0x7f8b080d0420 2024-04-26T20:56:57.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.414+0000 7f8b04c00640 1 --2- 172.21.15.28:0/1421666281 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8ae406c920 0x7f8ae406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:57.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.414+0000 7f8b04c00640 1 -- 172.21.15.28:0/1421666281 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8af008bef0 con 0x7f8b080d0420 2024-04-26T20:56:57.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.414+0000 7f8b0d200640 1 --2- 172.21.15.28:0/1421666281 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8ae406c920 0x7f8ae406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:57.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.415+0000 7f8b0d200640 1 --2- 172.21.15.28:0/1421666281 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8ae406c920 0x7f8ae406ede0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8af8034000 tx=0x7f8af802f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:57.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.419+0000 7f8b04c00640 1 -- 172.21.15.28:0/1421666281 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8af005a310 con 0x7f8b080d0420 2024-04-26T20:56:57.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.625+0000 7f8b0fc00640 1 -- 172.21.15.28:0/1421666281 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b080d0ba0 con 0x7f8b080d0420 2024-04-26T20:56:57.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.628+0000 7f8b04c00640 1 -- 172.21.15.28:0/1421666281 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7f8af0059c10 con 0x7f8b080d0420 2024-04-26T20:56:57.630 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:56:57.630 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":21,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:56:52.699663+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:56:57.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.630+0000 7f8b0fc00640 1 -- 172.21.15.28:0/1421666281 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8ae406c920 msgr2=0x7f8ae406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:57.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.630+0000 7f8b0fc00640 1 --2- 172.21.15.28:0/1421666281 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8ae406c920 0x7f8ae406ede0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8af8034000 tx=0x7f8af802f000 comp rx=0 tx=0).stop 2024-04-26T20:56:57.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.630+0000 7f8b0fc00640 1 -- 172.21.15.28:0/1421666281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b080d0420 msgr2=0x7f8b081f5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:57.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.630+0000 7f8b0fc00640 1 --2- 172.21.15.28:0/1421666281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b080d0420 0x7f8b081f5170 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8af000e9b0 tx=0x7f8af000ee70 comp rx=0 tx=0).stop 2024-04-26T20:56:57.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.631+0000 7f8b0fc00640 1 -- 172.21.15.28:0/1421666281 shutdown_connections 2024-04-26T20:56:57.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.631+0000 7f8b0fc00640 1 --2- 172.21.15.28:0/1421666281 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8ae406c920 0x7f8ae406ede0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:57.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.631+0000 7f8b0fc00640 1 --2- 172.21.15.28:0/1421666281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8b0815f420 0x7f8b081608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:57.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.631+0000 7f8b0fc00640 1 --2- 172.21.15.28:0/1421666281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b080d0420 0x7f8b081f5170 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:57.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.631+0000 7f8b0fc00640 1 --2- 172.21.15.28:0/1421666281 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8b080bad60 0x7f8b081565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:57.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.631+0000 7f8b0fc00640 1 -- 172.21.15.28:0/1421666281 >> 172.21.15.28:0/1421666281 conn(0x7f8b080b6c00 msgr2=0x7f8b080d1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:57.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.631+0000 7f8b0fc00640 1 -- 172.21.15.28:0/1421666281 shutdown_connections 2024-04-26T20:56:57.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.631+0000 7f8b0fc00640 1 -- 172.21.15.28:0/1421666281 wait complete. 2024-04-26T20:56:57.651 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-26T20:56:57.651 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:56:57.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.864+0000 7fc327e00640 1 Processor -- start 2024-04-26T20:56:57.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.864+0000 7fc327e00640 1 -- start start 2024-04-26T20:56:57.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.866+0000 7fc327e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc328146060 0x7fc328071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:57.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.866+0000 7fc327e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fc328146670 0x7fc3281609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:57.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.866+0000 7fc327e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc328160ef0 0x7fc32806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:57.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.866+0000 7fc327e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fc32806f510 con 0x7fc328160ef0 2024-04-26T20:56:57.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.866+0000 7fc327e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fc328146440 con 0x7fc328146670 2024-04-26T20:56:57.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.866+0000 7fc327e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc32806f010 con 0x7fc328146060 2024-04-26T20:56:57.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.866+0000 7fc327400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc328160ef0 0x7fc32806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:57.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.866+0000 7fc327400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc328160ef0 0x7fc32806de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42086/0 (socket says 172.21.15.28:42086) 2024-04-26T20:56:57.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.866+0000 7fc327400640 1 -- 172.21.15.28:0/3816165971 learned_addr learned my addr 172.21.15.28:0/3816165971 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:57.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.867+0000 7fc326000640 1 --2- 172.21.15.28:0/3816165971 >> v2:172.21.15.103:3300/0 conn(0x7fc328146670 0x7fc3281609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:57.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.867+0000 7fc326a00640 1 --2- 172.21.15.28:0/3816165971 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc328146060 0x7fc328071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:57.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.867+0000 7fc327400640 1 -- 172.21.15.28:0/3816165971 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc328146060 msgr2=0x7fc328071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:57.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.867+0000 7fc327400640 1 --2- 172.21.15.28:0/3816165971 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc328146060 0x7fc328071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:57.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.867+0000 7fc327400640 1 -- 172.21.15.28:0/3816165971 >> v2:172.21.15.103:3300/0 conn(0x7fc328146670 msgr2=0x7fc3281609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:57.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.867+0000 7fc327400640 1 --2- 172.21.15.28:0/3816165971 >> v2:172.21.15.103:3300/0 conn(0x7fc328146670 0x7fc3281609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:57.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.867+0000 7fc327400640 1 -- 172.21.15.28:0/3816165971 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3281617a0 con 0x7fc328160ef0 2024-04-26T20:56:57.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.868+0000 7fc327400640 1 --2- 172.21.15.28:0/3816165971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc328160ef0 0x7fc32806de80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc32806d720 tx=0x7fc32002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be4385718ced71f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:57.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.869+0000 7fc325600640 1 -- 172.21.15.28:0/3816165971 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc320036070 con 0x7fc328160ef0 2024-04-26T20:56:57.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.869+0000 7fc325600640 1 -- 172.21.15.28:0/3816165971 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc320031600 con 0x7fc328160ef0 2024-04-26T20:56:57.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.869+0000 7fc325600640 1 -- 172.21.15.28:0/3816165971 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3200319d0 con 0x7fc328160ef0 2024-04-26T20:56:57.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.869+0000 7fc327e00640 1 -- 172.21.15.28:0/3816165971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc328160ef0 msgr2=0x7fc32806de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:57.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.869+0000 7fc327e00640 1 --2- 172.21.15.28:0/3816165971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc328160ef0 0x7fc32806de80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc32806d720 tx=0x7fc32002a610 comp rx=0 tx=0).stop 2024-04-26T20:56:57.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.869+0000 7fc327e00640 1 -- 172.21.15.28:0/3816165971 shutdown_connections 2024-04-26T20:56:57.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.869+0000 7fc327e00640 1 --2- 172.21.15.28:0/3816165971 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc328160ef0 0x7fc32806de80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:57.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.869+0000 7fc327e00640 1 --2- 172.21.15.28:0/3816165971 >> v2:172.21.15.103:3300/0 conn(0x7fc328146670 0x7fc3281609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:57.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.869+0000 7fc327e00640 1 --2- 172.21.15.28:0/3816165971 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc328146060 0x7fc328071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:57.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.869+0000 7fc327e00640 1 -- 172.21.15.28:0/3816165971 >> 172.21.15.28:0/3816165971 conn(0x7fc3280d0a50 msgr2=0x7fc3280d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:57.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.870+0000 7fc327e00640 1 -- 172.21.15.28:0/3816165971 shutdown_connections 2024-04-26T20:56:57.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.870+0000 7fc327e00640 1 -- 172.21.15.28:0/3816165971 wait complete. 2024-04-26T20:56:57.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.871+0000 7fc327e00640 1 Processor -- start 2024-04-26T20:56:57.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.872+0000 7fc327e00640 1 -- start start 2024-04-26T20:56:57.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.872+0000 7fc327e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc328146060 0x7fc3281ea600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:57.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.872+0000 7fc327e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc328146670 0x7fc3281335c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:57.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.872+0000 7fc327e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc328160ef0 0x7fc3280b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:57.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.872+0000 7fc327e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc328161360 con 0x7fc328160ef0 2024-04-26T20:56:57.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.872+0000 7fc327e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc32806ff60 con 0x7fc328146060 2024-04-26T20:56:57.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.872+0000 7fc327e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc3281614e0 con 0x7fc328146670 2024-04-26T20:56:57.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.873+0000 7fc327400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc328160ef0 0x7fc3280b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:57.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.873+0000 7fc327400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc328160ef0 0x7fc3280b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42088/0 (socket says 172.21.15.28:42088) 2024-04-26T20:56:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.873+0000 7fc327400640 1 -- 172.21.15.28:0/3818096167 learned_addr learned my addr 172.21.15.28:0/3818096167 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.873+0000 7fc326000640 1 --2- 172.21.15.28:0/3818096167 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc328146670 0x7fc3281335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.873+0000 7fc327400640 1 -- 172.21.15.28:0/3818096167 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc328146670 msgr2=0x7fc3281335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.873+0000 7fc327400640 1 --2- 172.21.15.28:0/3818096167 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc328146670 0x7fc3281335c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.873+0000 7fc327400640 1 -- 172.21.15.28:0/3818096167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc328146060 msgr2=0x7fc3281ea600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.873+0000 7fc326a00640 1 --2- 172.21.15.28:0/3818096167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc328146060 0x7fc3281ea600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.873+0000 7fc327400640 1 --2- 172.21.15.28:0/3818096167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc328146060 0x7fc3281ea600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.873+0000 7fc327400640 1 -- 172.21.15.28:0/3818096167 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc328133fa0 con 0x7fc328160ef0 2024-04-26T20:56:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.874+0000 7fc327400640 1 --2- 172.21.15.28:0/3818096167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc328160ef0 0x7fc3280b5970 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc320034040 tx=0x7fc320004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.874+0000 7fc31ea00640 1 -- 172.21.15.28:0/3818096167 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc320036050 con 0x7fc328160ef0 2024-04-26T20:56:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.874+0000 7fc31ea00640 1 -- 172.21.15.28:0/3818096167 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc320042070 con 0x7fc328160ef0 2024-04-26T20:56:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.874+0000 7fc31ea00640 1 -- 172.21.15.28:0/3818096167 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc320009510 con 0x7fc328160ef0 2024-04-26T20:56:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.874+0000 7fc327e00640 1 -- 172.21.15.28:0/3818096167 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc328134280 con 0x7fc328160ef0 2024-04-26T20:56:57.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.874+0000 7fc327e00640 1 -- 172.21.15.28:0/3818096167 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc328134840 con 0x7fc328160ef0 2024-04-26T20:56:57.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.875+0000 7fc327e00640 1 -- 172.21.15.28:0/3818096167 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc328072d10 con 0x7fc328160ef0 2024-04-26T20:56:57.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.876+0000 7fc31ea00640 1 -- 172.21.15.28:0/3818096167 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc320007bf0 con 0x7fc328160ef0 2024-04-26T20:56:57.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.877+0000 7fc31ea00640 1 --2- 172.21.15.28:0/3818096167 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc30406c920 0x7fc30406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:57.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.877+0000 7fc31ea00640 1 -- 172.21.15.28:0/3818096167 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc3200ac920 con 0x7fc328160ef0 2024-04-26T20:56:57.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.877+0000 7fc326a00640 1 --2- 172.21.15.28:0/3818096167 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc30406c920 0x7fc30406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:57.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.878+0000 7fc326a00640 1 --2- 172.21.15.28:0/3818096167 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc30406c920 0x7fc30406ede0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc3281f8200 tx=0x7fc318008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:57.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:57.882+0000 7fc31ea00640 1 -- 172.21.15.28:0/3818096167 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc328072d10 con 0x7fc328160ef0 2024-04-26T20:56:58.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.085+0000 7fc327e00640 1 -- 172.21.15.28:0/3818096167 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc32806a360 con 0x7fc328160ef0 2024-04-26T20:56:58.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.088+0000 7fc31ea00640 1 -- 172.21.15.28:0/3818096167 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7fc32007acb0 con 0x7fc328160ef0 2024-04-26T20:56:58.090 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:56:58.090 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":21,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:56:52.699663+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:56:58.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.091+0000 7fc327e00640 1 -- 172.21.15.28:0/3818096167 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc30406c920 msgr2=0x7fc30406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.091+0000 7fc327e00640 1 --2- 172.21.15.28:0/3818096167 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc30406c920 0x7fc30406ede0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc3281f8200 tx=0x7fc318008040 comp rx=0 tx=0).stop 2024-04-26T20:56:58.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.091+0000 7fc327e00640 1 -- 172.21.15.28:0/3818096167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc328160ef0 msgr2=0x7fc3280b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.091+0000 7fc327e00640 1 --2- 172.21.15.28:0/3818096167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc328160ef0 0x7fc3280b5970 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc320034040 tx=0x7fc320004a40 comp rx=0 tx=0).stop 2024-04-26T20:56:58.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.091+0000 7fc327e00640 1 -- 172.21.15.28:0/3818096167 shutdown_connections 2024-04-26T20:56:58.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.091+0000 7fc327e00640 1 --2- 172.21.15.28:0/3818096167 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc30406c920 0x7fc30406ede0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.091+0000 7fc327e00640 1 --2- 172.21.15.28:0/3818096167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc328160ef0 0x7fc3280b5970 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.091+0000 7fc327e00640 1 --2- 172.21.15.28:0/3818096167 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc328146670 0x7fc3281335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.091+0000 7fc327e00640 1 --2- 172.21.15.28:0/3818096167 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc328146060 0x7fc3281ea600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.091+0000 7fc327e00640 1 -- 172.21.15.28:0/3818096167 >> 172.21.15.28:0/3818096167 conn(0x7fc3280d0a50 msgr2=0x7fc3280c9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.092+0000 7fc327e00640 1 -- 172.21.15.28:0/3818096167 shutdown_connections 2024-04-26T20:56:58.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.092+0000 7fc327e00640 1 -- 172.21.15.28:0/3818096167 wait complete. 2024-04-26T20:56:58.112 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T20:56:58.112 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T20:56:58.113 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T20:56:58.113 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-26T20:56:58.113 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-26T20:56:58.114 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-26T20:56:58.114 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-26T20:56:58.115 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-26T20:56:58.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.407+0000 7fc7b7e00640 1 Processor -- start 2024-04-26T20:56:58.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.407+0000 7fc7b7e00640 1 -- start start 2024-04-26T20:56:58.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.407+0000 7f9016600640 1 Processor -- start 2024-04-26T20:56:58.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.408+0000 7f9016600640 1 -- start start 2024-04-26T20:56:58.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.409+0000 7fc7b7e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b80d13d0 0x7fc7b80b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.409+0000 7fc7b7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc7b80ae8c0 0x7fc7b806e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.409+0000 7fc7b7e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7b80aeda0 con 0x7fc7b815edc0 2024-04-26T20:56:58.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.409+0000 7fc7b7e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fc7b80af080 con 0x7fc7b80ae8c0 2024-04-26T20:56:58.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.409+0000 7fc7b7e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b80af360 con 0x7fc7b80d13d0 2024-04-26T20:56:58.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.410+0000 7f9016600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90080069e0 0x7f9008006e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.410+0000 7fc7b6000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b80d13d0 0x7fc7b80b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.410+0000 7f9015200640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f90080ead00 0x7f900804fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57740/0 (socket says 172.21.15.28:57740) 2024-04-26T20:56:58.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.410+0000 7f9015200640 1 -- 172.21.15.28:0/45372648 learned_addr learned my addr 172.21.15.28:0/45372648 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:58.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.410+0000 7f9016600640 1 -- 172.21.15.28:0/45372648 --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9008053720 con 0x7f90080ead00 2024-04-26T20:56:58.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.411+0000 7f9016600640 1 -- 172.21.15.28:0/45372648 --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9008053a00 con 0x7f9008005e90 2024-04-26T20:56:58.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.411+0000 7f9016600640 1 -- 172.21.15.28:0/45372648 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9008054330 con 0x7f90080069e0 2024-04-26T20:56:58.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.411+0000 7f9015c00640 1 --2- 172.21.15.28:0/45372648 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90080069e0 0x7f9008006e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.412+0000 7f9015c00640 1 -- 172.21.15.28:0/45372648 >> v1:172.21.15.103:6789/0 conn(0x7f9008005e90 legacy=0x7f90080062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.412+0000 7f900f400640 1 -- 172.21.15.28:0/45372648 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3892381029 0 0) 0x7f9008053720 con 0x7f90080ead00 2024-04-26T20:56:58.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.412+0000 7f9015c00640 1 -- 172.21.15.28:0/45372648 >> v1:172.21.15.28:6789/0 conn(0x7f90080ead00 legacy=0x7f900804fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.412+0000 7f9015c00640 1 -- 172.21.15.28:0/45372648 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9008053ce0 con 0x7f90080069e0 2024-04-26T20:56:58.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.411+0000 7fc7b6000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b80d13d0 0x7fc7b80b4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.28:59140/0 (socket says 172.21.15.28:59140) 2024-04-26T20:56:58.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.411+0000 7fc7b7400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc7b80ae8c0 0x7fc7b806e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.412+0000 7fc7b7400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc7b80ae8c0 0x7fc7b806e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42094/0 (socket says 172.21.15.28:42094) 2024-04-26T20:56:58.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.412+0000 7fc7b7400640 1 -- 172.21.15.28:0/785753843 learned_addr learned my addr 172.21.15.28:0/785753843 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.413+0000 7fc7b5600640 1 -- 172.21.15.28:0/785753843 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3748930667 0 0) 0x7fc7b80aeda0 con 0x7fc7b815edc0 2024-04-26T20:56:58.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.413+0000 7fc7b5600640 1 -- 172.21.15.28:0/785753843 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc79c00bd30 con 0x7fc7b815edc0 2024-04-26T20:56:58.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.413+0000 7fc7b5600640 1 -- 172.21.15.28:0/785753843 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2177084057 0 0) 0x7fc79c00bd30 con 0x7fc7b815edc0 2024-04-26T20:56:58.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.413+0000 7fc7b5600640 1 -- 172.21.15.28:0/785753843 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b80d13d0 msgr2=0x7fc7b80b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.413+0000 7fc7b5600640 1 --2- 172.21.15.28:0/785753843 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b80d13d0 0x7fc7b80b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.413+0000 7fc7b5600640 1 -- 172.21.15.28:0/785753843 >> v2:172.21.15.28:3300/0 conn(0x7fc7b80ae8c0 msgr2=0x7fc7b806e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.413+0000 7fc7b5600640 1 --2- 172.21.15.28:0/785753843 >> v2:172.21.15.28:3300/0 conn(0x7fc7b80ae8c0 0x7fc7b806e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.413+0000 7fc7b5600640 1 -- 172.21.15.28:0/785753843 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b80af640 con 0x7fc7b815edc0 2024-04-26T20:56:58.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.413+0000 7fc7b7400640 1 --2- 172.21.15.28:0/785753843 >> v2:172.21.15.28:3300/0 conn(0x7fc7b80ae8c0 0x7fc7b806e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:56:58.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.413+0000 7fc7b6000640 1 --2- 172.21.15.28:0/785753843 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b80d13d0 0x7fc7b80b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:56:58.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.413+0000 7fc7b5600640 1 -- 172.21.15.28:0/785753843 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fc7b00019f0 con 0x7fc7b815edc0 2024-04-26T20:56:58.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.414+0000 7fc7b5600640 1 -- 172.21.15.28:0/785753843 >> v1:172.21.15.28:6789/0 conn(0x7fc7b815edc0 legacy=0x7fc7b80d0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.414+0000 7fc7b5600640 1 --2- 172.21.15.28:0/785753843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc79c00c1c0 0x7fc79c00c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.414+0000 7fc7b5600640 1 --2- 172.21.15.28:0/785753843 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b80d13d0 0x7fc79c00d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.414+0000 7fc7b5600640 1 --2- 172.21.15.28:0/785753843 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc79c00d550 0x7fc79c00d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.414+0000 7fc7b5600640 1 -- 172.21.15.28:0/785753843 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b80af080 con 0x7fc79c00d550 2024-04-26T20:56:58.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.414+0000 7fc7b5600640 1 -- 172.21.15.28:0/785753843 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b80af360 con 0x7fc79c00c1c0 2024-04-26T20:56:58.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.414+0000 7fc7b5600640 1 -- 172.21.15.28:0/785753843 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc79c009470 con 0x7fc7b80d13d0 2024-04-26T20:56:58.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.414+0000 7f9015c00640 1 --2- 172.21.15.28:0/45372648 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90080069e0 0x7f9008006e70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9010065e50 tx=0x7f90100661d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=952f6dc5e4e471ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.414+0000 7fc7b6000640 1 --2- 172.21.15.28:0/785753843 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b80d13d0 0x7fc79c00d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.415+0000 7fc7b7400640 1 --2- 172.21.15.28:0/785753843 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc79c00d550 0x7fc79c00d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.415+0000 7fc7b6a00640 1 --2- 172.21.15.28:0/785753843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc79c00c1c0 0x7fc79c00c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.416+0000 7fc7b7400640 1 -- 172.21.15.28:0/785753843 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b80d13d0 msgr2=0x7fc79c00d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.416+0000 7fc7b7400640 1 --2- 172.21.15.28:0/785753843 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b80d13d0 0x7fc79c00d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.416+0000 7fc7b7400640 1 -- 172.21.15.28:0/785753843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc79c00c1c0 msgr2=0x7fc79c00c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.416+0000 7fc7b7400640 1 --2- 172.21.15.28:0/785753843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc79c00c1c0 0x7fc79c00c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.416+0000 7fc7b7400640 1 -- 172.21.15.28:0/785753843 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79c00bd30 con 0x7fc79c00d550 2024-04-26T20:56:58.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.416+0000 7f900f400640 1 -- 172.21.15.28:0/45372648 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90100b2070 con 0x7f90080069e0 2024-04-26T20:56:58.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.416+0000 7f900f400640 1 -- 172.21.15.28:0/45372648 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f901006e8a0 con 0x7f90080069e0 2024-04-26T20:56:58.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.416+0000 7f900f400640 1 -- 172.21.15.28:0/45372648 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f901006ed10 con 0x7f90080069e0 2024-04-26T20:56:58.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.417+0000 7f9016600640 1 -- 172.21.15.28:0/45372648 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90080069e0 msgr2=0x7f9008006e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.417+0000 7f9016600640 1 --2- 172.21.15.28:0/45372648 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90080069e0 0x7f9008006e70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9010065e50 tx=0x7f90100661d0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.417+0000 7fc7b7400640 1 --2- 172.21.15.28:0/785753843 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc79c00d550 0x7fc79c00d9d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009bc0 tx=0x7fc79c00e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe724d638aa8620b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.417+0000 7f9016600640 1 -- 172.21.15.28:0/45372648 shutdown_connections 2024-04-26T20:56:58.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.417+0000 7f9016600640 1 --2- 172.21.15.28:0/45372648 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90080069e0 0x7f9008006e70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.418+0000 7f9016600640 1 -- 172.21.15.28:0/45372648 >> 172.21.15.28:0/45372648 conn(0x7f9008009fb0 msgr2=0x7f900800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.418+0000 7fc7b5600640 1 -- 172.21.15.28:0/785753843 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a400f040 con 0x7fc79c00d550 2024-04-26T20:56:58.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.418+0000 7fc7b5600640 1 -- 172.21.15.28:0/785753843 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7a4031600 con 0x7fc79c00d550 2024-04-26T20:56:58.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.418+0000 7fc7b5600640 1 -- 172.21.15.28:0/785753843 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40319b0 con 0x7fc79c00d550 2024-04-26T20:56:58.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.419+0000 7fc7b7e00640 1 -- 172.21.15.28:0/785753843 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc79c00d550 msgr2=0x7fc79c00d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.419+0000 7fc7b7e00640 1 --2- 172.21.15.28:0/785753843 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc79c00d550 0x7fc79c00d9d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009bc0 tx=0x7fc79c00e040 comp rx=0 tx=0).stop 2024-04-26T20:56:58.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.419+0000 7fc7b7400640 1 -- 172.21.15.28:0/785753843 reap_dead start 2024-04-26T20:56:58.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.419+0000 7fc7b7e00640 1 -- 172.21.15.28:0/785753843 shutdown_connections 2024-04-26T20:56:58.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.419+0000 7fc7b7e00640 1 -- 172.21.15.28:0/785753843 >> 172.21.15.28:0/785753843 conn(0x7fc7b806c350 msgr2=0x7fc7b806c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.419+0000 7f9016600640 1 -- 172.21.15.28:0/45372648 shutdown_connections 2024-04-26T20:56:58.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.419+0000 7fc7b7e00640 1 -- 172.21.15.28:0/785753843 shutdown_connections 2024-04-26T20:56:58.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.419+0000 7f9016600640 1 -- 172.21.15.28:0/45372648 wait complete. 2024-04-26T20:56:58.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.420+0000 7fc7b7e00640 1 -- 172.21.15.28:0/785753843 wait complete. 2024-04-26T20:56:58.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.420+0000 7fd605400640 1 Processor -- start 2024-04-26T20:56:58.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.422+0000 7f9016600640 1 Processor -- start 2024-04-26T20:56:58.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.422+0000 7fc7b7e00640 1 Processor -- start 2024-04-26T20:56:58.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.422+0000 7f9016600640 1 -- start start 2024-04-26T20:56:58.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.423+0000 7fc7b7e00640 1 -- start start 2024-04-26T20:56:58.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.421+0000 7fd605400640 1 -- start start 2024-04-26T20:56:58.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.423+0000 7fd605400640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fd60015efd0 0x7fd6000ba770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.423+0000 7fd605400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd60015f3b0 0x7fd60006c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.423+0000 7fd605400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6000d3bd0 con 0x7fd60006c890 2024-04-26T20:56:58.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.423+0000 7fd605400640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fd6000d3e70 con 0x7fd60015efd0 2024-04-26T20:56:58.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.423+0000 7fd605400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fd6000d4150 con 0x7fd60015f3b0 2024-04-26T20:56:58.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.423+0000 7fd5ff200640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd60006c890 0x7fd6000b38c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57758/0 (socket says 172.21.15.28:57758) 2024-04-26T20:56:58.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.423+0000 7fd5ff200640 1 -- 172.21.15.28:0/3433092400 learned_addr learned my addr 172.21.15.28:0/3433092400 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:58.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.423+0000 7fd5fde00640 1 --2- 172.21.15.28:0/3433092400 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd60015f3b0 0x7fd60006c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.424+0000 7f9016600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9008005e90 0x7f90080d5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.424+0000 7fd5fd400640 1 -- 172.21.15.28:0/3433092400 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1145952702 0 0) 0x7fd6000d3bd0 con 0x7fd60006c890 2024-04-26T20:56:58.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.424+0000 7fd5fd400640 1 -- 172.21.15.28:0/3433092400 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5e4003520 con 0x7fd60006c890 2024-04-26T20:56:58.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.424+0000 7f9015200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9008005e90 0x7f90080d5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.426+0000 7fc7b7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc79c00d550 0x7fc7b8146580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.426+0000 7fc7b7e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b8140aa0 0x7fc7b8140f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.426+0000 7fc7b7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b8141c10 0x7fc7b8142090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.425+0000 7f13ffa00640 1 Processor -- start 2024-04-26T20:56:58.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.425+0000 7f13ffa00640 1 -- start start 2024-04-26T20:56:58.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.425+0000 7fd5fd400640 1 -- 172.21.15.28:0/3433092400 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 582332182 0 0) 0x7fd5e4003520 con 0x7fd60006c890 2024-04-26T20:56:58.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.425+0000 7fd5fd400640 1 -- 172.21.15.28:0/3433092400 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd60015f3b0 msgr2=0x7fd60006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.425+0000 7fd5fd400640 1 --2- 172.21.15.28:0/3433092400 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd60015f3b0 0x7fd60006c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.425+0000 7fd5fe800640 1 --2- 172.21.15.28:0/3433092400 >> v2:172.21.15.103:3300/0 conn(0x7fd60015efd0 0x7fd6000ba770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.425+0000 7fd5fd400640 1 -- 172.21.15.28:0/3433092400 >> v2:172.21.15.103:3300/0 conn(0x7fd60015efd0 msgr2=0x7fd6000ba770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.425+0000 7fd5fd400640 1 --2- 172.21.15.28:0/3433092400 >> v2:172.21.15.103:3300/0 conn(0x7fd60015efd0 0x7fd6000ba770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.425+0000 7fd5fd400640 1 -- 172.21.15.28:0/3433092400 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6000d4430 con 0x7fd60006c890 2024-04-26T20:56:58.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.425+0000 7fd5fd400640 1 -- 172.21.15.28:0/3433092400 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fd5f80019f0 con 0x7fd60006c890 2024-04-26T20:56:58.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.425+0000 7fd5fe800640 1 --2- 172.21.15.28:0/3433092400 >> v2:172.21.15.103:3300/0 conn(0x7fd60015efd0 0x7fd6000ba770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:56:58.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.426+0000 7f9016600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90080069e0 0x7f90080d6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.426+0000 7f9016600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90080ead00 0x7f90080d8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.426+0000 7f9016600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f90080547c0 con 0x7f9008005e90 2024-04-26T20:56:58.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.426+0000 7f9016600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90080501a0 con 0x7f90080ead00 2024-04-26T20:56:58.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.426+0000 7f9015200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9008005e90 0x7f90080d5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42112/0 (socket says 172.21.15.28:42112) 2024-04-26T20:56:58.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7f9015200640 1 -- 172.21.15.28:0/1096148521 learned_addr learned my addr 172.21.15.28:0/1096148521 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7f9016600640 1 -- 172.21.15.28:0/1096148521 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9008054940 con 0x7f90080069e0 2024-04-26T20:56:58.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7f900fe00640 1 --2- 172.21.15.28:0/1096148521 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90080069e0 0x7f90080d6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7f9015c00640 1 --2- 172.21.15.28:0/1096148521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90080ead00 0x7f90080d8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7f13ffa00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13f815efd0 0x7f13f80ba770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7fc7b7400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b8141c10 0x7fc7b8142090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7fc7b7400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b8141c10 0x7fc7b8142090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.28:35576/0 (socket says 172.21.15.28:35576) 2024-04-26T20:56:58.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7fc7b7400640 1 -- 172.21.15.28:0/370158799 learned_addr learned my addr 172.21.15.28:0/370158799 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7f9015200640 1 -- 172.21.15.28:0/1096148521 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90080069e0 msgr2=0x7f90080d6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7f9015200640 1 --2- 172.21.15.28:0/1096148521 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f90080069e0 0x7f90080d6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7f9015200640 1 -- 172.21.15.28:0/1096148521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90080ead00 msgr2=0x7f90080d8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7f9015200640 1 --2- 172.21.15.28:0/1096148521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90080ead00 0x7f90080d8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7f9015200640 1 -- 172.21.15.28:0/1096148521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90080d7210 con 0x7f9008005e90 2024-04-26T20:56:58.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7fc7b6a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc79c00d550 0x7fc7b8146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.428+0000 7fc7b7e00640 1 -- 172.21.15.28:0/370158799 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b815f1a0 con 0x7fc79c00d550 2024-04-26T20:56:58.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7f13ffa00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f13f806ca60 0x7f13f80b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.428+0000 7fc7b7e00640 1 -- 172.21.15.28:0/370158799 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b8073c20 con 0x7fc7b8141c10 2024-04-26T20:56:58.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7f13ffa00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13f80d3bd0 con 0x7f13f815f3b0 2024-04-26T20:56:58.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.428+0000 7f9015200640 1 --2- 172.21.15.28:0/1096148521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9008005e90 0x7f90080d5f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00c3f0 tx=0x7f8ffc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.428+0000 7fc7b7e00640 1 -- 172.21.15.28:0/370158799 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b815f320 con 0x7fc7b8140aa0 2024-04-26T20:56:58.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7f13ffa00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f13f80d3e70 con 0x7f13f806ca60 2024-04-26T20:56:58.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7f13ffa00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f13f80d4150 con 0x7f13f815efd0 2024-04-26T20:56:58.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7f13efe00640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f13f815f3b0 0x7f13f806c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.28:36034/0 (socket says 172.21.15.28:36034) 2024-04-26T20:56:58.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7f13efe00640 1 -- 172.21.15.28:0/382665288 learned_addr learned my addr 172.21.15.28:0/382665288 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:58.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.425+0000 7fd5fd400640 1 -- 172.21.15.28:0/3433092400 >> v1:172.21.15.28:6789/0 conn(0x7fd60006c890 legacy=0x7fd6000b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.426+0000 7fd5fd400640 1 --2- 172.21.15.28:0/3433092400 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd60015f3b0 0x7fd5e40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.426+0000 7fd5fd400640 1 --2- 172.21.15.28:0/3433092400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5e4003f80 0x7fd5e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.426+0000 7fd5fd400640 1 --2- 172.21.15.28:0/3433092400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5e4004d40 0x7fd5e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.426+0000 7fd5fd400640 1 -- 172.21.15.28:0/3433092400 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd6000d3e70 con 0x7fd5e4004d40 2024-04-26T20:56:58.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.426+0000 7fd5fd400640 1 -- 172.21.15.28:0/3433092400 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd6000d4150 con 0x7fd5e4003f80 2024-04-26T20:56:58.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.426+0000 7fd5fd400640 1 -- 172.21.15.28:0/3433092400 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e400bb50 con 0x7fd60015f3b0 2024-04-26T20:56:58.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.426+0000 7fd5ff200640 1 --2- 172.21.15.28:0/3433092400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5e4004d40 0x7fd5e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7fd5fe800640 1 --2- 172.21.15.28:0/3433092400 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd60015f3b0 0x7fd5e40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7fd5ff200640 1 -- 172.21.15.28:0/3433092400 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd60015f3b0 msgr2=0x7fd5e40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7fd5ff200640 1 --2- 172.21.15.28:0/3433092400 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd60015f3b0 0x7fd5e40039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7fd5ff200640 1 -- 172.21.15.28:0/3433092400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5e4003f80 msgr2=0x7fd5e4004800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T20:56:58.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7fd5ff200640 1 --2- 172.21.15.28:0/3433092400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd5e4003f80 0x7fd5e4004800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.427+0000 7fd5ff200640 1 -- 172.21.15.28:0/3433092400 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e4003520 con 0x7fd5e4004d40 2024-04-26T20:56:58.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.429+0000 7fc7b6000640 1 --2- 172.21.15.28:0/370158799 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b8140aa0 0x7fc7b8140f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.429+0000 7f13f7e00640 1 -- 172.21.15.28:0/382665288 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 441093212 0 0) 0x7f13f80d3bd0 con 0x7f13f815f3b0 2024-04-26T20:56:58.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.429+0000 7f13f7e00640 1 -- 172.21.15.28:0/382665288 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13dc003520 con 0x7f13f815f3b0 2024-04-26T20:56:58.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.429+0000 7f900cc00640 1 -- 172.21.15.28:0/1096148521 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc0074e0 con 0x7f9008005e90 2024-04-26T20:56:58.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.429+0000 7f900cc00640 1 -- 172.21.15.28:0/1096148521 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ffc0047e0 con 0x7f9008005e90 2024-04-26T20:56:58.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.429+0000 7f900cc00640 1 -- 172.21.15.28:0/1096148521 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc004c10 con 0x7f9008005e90 2024-04-26T20:56:58.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.429+0000 7f9016600640 1 -- 172.21.15.28:0/1096148521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90080d74f0 con 0x7f9008005e90 2024-04-26T20:56:58.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.429+0000 7f9016600640 1 -- 172.21.15.28:0/1096148521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90080d7ab0 con 0x7f9008005e90 2024-04-26T20:56:58.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.429+0000 7fc523000640 1 Processor -- start 2024-04-26T20:56:58.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.429+0000 7fc523000640 1 -- start start 2024-04-26T20:56:58.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.429+0000 7fc7b6a00640 1 -- 172.21.15.28:0/370158799 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b8140aa0 msgr2=0x7fc7b8140f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.429+0000 7fc7b6a00640 1 --2- 172.21.15.28:0/370158799 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc7b8140aa0 0x7fc7b8140f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.429+0000 7fc7b6a00640 1 -- 172.21.15.28:0/370158799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b8141c10 msgr2=0x7fc7b8142090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.429+0000 7fc7b6a00640 1 --2- 172.21.15.28:0/370158799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc7b8141c10 0x7fc7b8142090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.429+0000 7fc7b6a00640 1 -- 172.21.15.28:0/370158799 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b813e9a0 con 0x7fc79c00d550 2024-04-26T20:56:58.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.428+0000 7fa46e600640 1 Processor -- start 2024-04-26T20:56:58.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.428+0000 7fa46e600640 1 -- start start 2024-04-26T20:56:58.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.430+0000 7f13fda00640 1 --2- 172.21.15.28:0/382665288 >> v2:172.21.15.103:3300/0 conn(0x7f13f806ca60 0x7f13f80b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.430+0000 7fc7b6a00640 1 --2- 172.21.15.28:0/370158799 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc79c00d550 0x7fc7b8146580 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000ead0 tx=0x7fc7b000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.430+0000 7f13fd000640 1 --2- 172.21.15.28:0/382665288 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13f815efd0 0x7f13f80ba770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.430+0000 7f13fda00640 1 -- 172.21.15.28:0/382665288 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13f815efd0 msgr2=0x7f13f80ba770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.430+0000 7f13f7e00640 1 -- 172.21.15.28:0/382665288 <== mon.1 v1:172.21.15.103:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3637752508 0 0) 0x7f13dc003520 con 0x7f13f815f3b0 2024-04-26T20:56:58.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.430+0000 7f13fda00640 1 --2- 172.21.15.28:0/382665288 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13f815efd0 0x7f13f80ba770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.431+0000 7f9016600640 1 -- 172.21.15.28:0/1096148521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9008005390 con 0x7f9008005e90 2024-04-26T20:56:58.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.430+0000 7f13fda00640 1 -- 172.21.15.28:0/382665288 >> v1:172.21.15.103:6789/0 conn(0x7f13f815f3b0 legacy=0x7f13f806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.430+0000 7f13fda00640 1 -- 172.21.15.28:0/382665288 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f80d4430 con 0x7f13f806ca60 2024-04-26T20:56:58.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.431+0000 7fa46e600640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa4680cfbc0 0x7fa468163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.431+0000 7fa46e600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa468164ac0 con 0x7fa46815dd10 2024-04-26T20:56:58.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.431+0000 7fa46e600640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4680d4c10 con 0x7fa4680d59d0 2024-04-26T20:56:58.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.431+0000 7fa46e600640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fa4680d4ef0 con 0x7fa4680cfbc0 2024-04-26T20:56:58.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.431+0000 7fd5ff200640 1 --2- 172.21.15.28:0/3433092400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5e4004d40 0x7fd5e40051c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800a3b0 tx=0x7fd5e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54fba95f58a1ca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.431+0000 7fc7aea00640 1 -- 172.21.15.28:0/370158799 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0003b80 con 0x7fc79c00d550 2024-04-26T20:56:58.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.431+0000 7fc7aea00640 1 -- 172.21.15.28:0/370158799 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b0003d20 con 0x7fc79c00d550 2024-04-26T20:56:58.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.431+0000 7ff1fb600640 1 Processor -- start 2024-04-26T20:56:58.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.431+0000 7fc7aea00640 1 -- 172.21.15.28:0/370158799 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0006b50 con 0x7fc79c00d550 2024-04-26T20:56:58.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.431+0000 7fd5fd400640 1 -- 172.21.15.28:0/3433092400 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f800a3f0 con 0x7fd5e4004d40 2024-04-26T20:56:58.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.431+0000 7fc7b7e00640 1 -- 172.21.15.28:0/370158799 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b813ec80 con 0x7fc79c00d550 2024-04-26T20:56:58.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.432+0000 7fa467e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa4680cfbc0 0x7fa468163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.432+0000 7fa467e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa4680cfbc0 0x7fa468163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.28:35592/0 (socket says 172.21.15.28:35592) 2024-04-26T20:56:58.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.432+0000 7fc7b7e00640 1 -- 172.21.15.28:0/370158799 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b813f240 con 0x7fc79c00d550 2024-04-26T20:56:58.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.432+0000 7fa467e00640 1 -- 172.21.15.28:0/1410878323 learned_addr learned my addr 172.21.15.28:0/1410878323 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.432+0000 7f13fda00640 1 --2- 172.21.15.28:0/382665288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13f806ca60 0x7f13f80b38c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f13f00097a0 tx=0x7f13f002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc6b37715461321 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.431+0000 7fd5fd400640 1 -- 172.21.15.28:0/3433092400 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5f803b630 con 0x7fd5e4004d40 2024-04-26T20:56:58.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.432+0000 7fa467400640 1 -- 172.21.15.28:0/1410878323 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 746773754 0 0) 0x7fa4680d4c10 con 0x7fa4680d59d0 2024-04-26T20:56:58.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.433+0000 7fa467400640 1 -- 172.21.15.28:0/1410878323 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa450003520 con 0x7fa4680d59d0 2024-04-26T20:56:58.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.432+0000 7fd605400640 1 -- 172.21.15.28:0/3433092400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5e4004d40 msgr2=0x7fd5e40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.433+0000 7fd5fd400640 1 -- 172.21.15.28:0/3433092400 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f800a3f0 con 0x7fd5e4004d40 2024-04-26T20:56:58.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.433+0000 7fc51be00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc51c14c460 0x7fc51c072020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57774/0 (socket says 172.21.15.28:57774) 2024-04-26T20:56:58.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.433+0000 7fa467400640 1 -- 172.21.15.28:0/1410878323 <== mon.1 v1:172.21.15.103:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1410117541 0 0) 0x7fa450003520 con 0x7fa4680d59d0 2024-04-26T20:56:58.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.433+0000 7fa467400640 1 -- 172.21.15.28:0/1410878323 >> v2:172.21.15.103:3300/0 conn(0x7fa4680cfbc0 msgr2=0x7fa468163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.433+0000 7fa467400640 1 --2- 172.21.15.28:0/1410878323 >> v2:172.21.15.103:3300/0 conn(0x7fa4680cfbc0 0x7fa468163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.433+0000 7fa467400640 1 -- 172.21.15.28:0/1410878323 >> v1:172.21.15.28:6789/0 conn(0x7fa46815dd10 legacy=0x7fa4680cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.433+0000 7fa467400640 1 -- 172.21.15.28:0/1410878323 --> v1:172.21.15.103:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4680d51d0 con 0x7fa4680d59d0 2024-04-26T20:56:58.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.433+0000 7fa467e00640 1 --2- 172.21.15.28:0/1410878323 >> v2:172.21.15.103:3300/0 conn(0x7fa4680cfbc0 0x7fa468163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:56:58.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.433+0000 7fa467400640 1 -- 172.21.15.28:0/1410878323 <== mon.1 v1:172.21.15.103:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fa460001650 con 0x7fa4680d59d0 2024-04-26T20:56:58.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.433+0000 7fa467400640 1 -- 172.21.15.28:0/1410878323 >> v1:172.21.15.103:6789/0 conn(0x7fa4680d59d0 legacy=0x7fa4681642d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.433+0000 7fa467400640 1 --2- 172.21.15.28:0/1410878323 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4500039d0 0x7fa450003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.433+0000 7fa467400640 1 --2- 172.21.15.28:0/1410878323 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa450004800 0x7fa450004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.433+0000 7fa467400640 1 --2- 172.21.15.28:0/1410878323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa450005210 0x7fa45000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.433+0000 7fa467400640 1 -- 172.21.15.28:0/1410878323 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa4680d4ef0 con 0x7fa4500039d0 2024-04-26T20:56:58.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.433+0000 7fa467400640 1 -- 172.21.15.28:0/1410878323 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa45000c140 con 0x7fa450005210 2024-04-26T20:56:58.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.433+0000 7fa467400640 1 -- 172.21.15.28:0/1410878323 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa45000c340 con 0x7fa450004800 2024-04-26T20:56:58.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.433+0000 7f900cc00640 1 -- 172.21.15.28:0/1096148521 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ffc011070 con 0x7f9008005e90 2024-04-26T20:56:58.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.433+0000 7f13f7e00640 1 -- 172.21.15.28:0/382665288 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0037070 con 0x7f13f806ca60 2024-04-26T20:56:58.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.434+0000 7f13f7e00640 1 -- 172.21.15.28:0/382665288 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f002aba0 con 0x7f13f806ca60 2024-04-26T20:56:58.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.434+0000 7f13f7e00640 1 -- 172.21.15.28:0/382665288 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0001670 con 0x7f13f806ca60 2024-04-26T20:56:58.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.434+0000 7fd605400640 1 --2- 172.21.15.28:0/3433092400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd5e4004d40 0x7fd5e40051c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800a3b0 tx=0x7fd5e4005830 comp rx=0 tx=0).stop 2024-04-26T20:56:58.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.434+0000 7fd5ff200640 1 -- 172.21.15.28:0/3433092400 reap_dead start 2024-04-26T20:56:58.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.434+0000 7fa46dc00640 1 --2- 172.21.15.28:0/1410878323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa450005210 0x7fa45000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.434+0000 7fa467e00640 1 --2- 172.21.15.28:0/1410878323 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa450004800 0x7fa450004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.434+0000 7fa46d200640 1 --2- 172.21.15.28:0/1410878323 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4500039d0 0x7fa450003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.434+0000 7fc7b7e00640 1 -- 172.21.15.28:0/370158799 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc7b806a360 con 0x7fc79c00d550 2024-04-26T20:56:58.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.434+0000 7fd605400640 1 -- 172.21.15.28:0/3433092400 shutdown_connections 2024-04-26T20:56:58.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.434+0000 7fd605400640 1 -- 172.21.15.28:0/3433092400 >> 172.21.15.28:0/3433092400 conn(0x7fd6000d4e50 msgr2=0x7fd6000d5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.435+0000 7fd605400640 1 -- 172.21.15.28:0/3433092400 shutdown_connections 2024-04-26T20:56:58.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.434+0000 7fc523000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc51c06fcf0 0x7fc51c16e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.435+0000 7fc51be00640 1 -- 172.21.15.28:0/1975667636 learned_addr learned my addr 172.21.15.28:0/1975667636 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:58.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.435+0000 7fc523000640 1 -- 172.21.15.28:0/1975667636 --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc51c072990 con 0x7fc51c14c460 2024-04-26T20:56:58.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.435+0000 7fc523000640 1 -- 172.21.15.28:0/1975667636 --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc51c072c70 con 0x7fc51c14cae0 2024-04-26T20:56:58.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.435+0000 7fc523000640 1 -- 172.21.15.28:0/1975667636 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc51c073a30 con 0x7fc51c06fcf0 2024-04-26T20:56:58.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.435+0000 7fc521000640 1 --2- 172.21.15.28:0/1975667636 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc51c06fcf0 0x7fc51c16e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.435+0000 7fa467e00640 1 -- 172.21.15.28:0/1410878323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa450005210 msgr2=0x7fa45000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.435+0000 7fa467e00640 1 --2- 172.21.15.28:0/1410878323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa450005210 0x7fa45000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.435+0000 7fa467e00640 1 -- 172.21.15.28:0/1410878323 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4500039d0 msgr2=0x7fa450003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.432+0000 7ff1fb600640 1 -- start start 2024-04-26T20:56:58.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.434+0000 7ff1fb600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff1f415afc0 0x7ff1f415c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.434+0000 7ff1fb600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff1f40d0180 0x7ff1f40d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.434+0000 7ff1fb600640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff1f40cf2b0 0x7ff1f40cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.434+0000 7ff1fb600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f415c6a0 con 0x7ff1f40d0180 2024-04-26T20:56:58.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.434+0000 7ff1fb600640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f40d0b20 con 0x7ff1f40cf2b0 2024-04-26T20:56:58.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.434+0000 7ff1fb600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f40ce810 con 0x7ff1f415afc0 2024-04-26T20:56:58.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.436+0000 7ff1f3e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff1f40d0180 0x7ff1f40d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.436+0000 7ff1f3e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff1f40d0180 0x7ff1f40d05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42148/0 (socket says 172.21.15.28:42148) 2024-04-26T20:56:58.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.436+0000 7ff1f3e00640 1 -- 172.21.15.28:0/3720906340 learned_addr learned my addr 172.21.15.28:0/3720906340 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.436+0000 7ff1f8c00640 1 --2- 172.21.15.28:0/3720906340 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff1f415afc0 0x7ff1f415c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.435+0000 7fd605400640 1 -- 172.21.15.28:0/3433092400 wait complete. 2024-04-26T20:56:58.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.435+0000 7f900cc00640 1 --2- 172.21.15.28:0/1096148521 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8ff006c9c0 0x7f8ff006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.435+0000 7f900cc00640 1 -- 172.21.15.28:0/1096148521 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8ffc08b5d0 con 0x7f9008005e90 2024-04-26T20:56:58.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.436+0000 7f900cc00640 1 --2- 172.21.15.28:0/1096148521 >> [v2:172.21.15.28:6808/993399689,v1:172.21.15.28:6809/993399689] conn(0x7f8ff0074be0 0x7f8ff0077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.436+0000 7f900cc00640 1 -- 172.21.15.28:0/1096148521 --> [v2:172.21.15.28:6808/993399689,v1:172.21.15.28:6809/993399689] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ff00776b0 con 0x7f8ff0074be0 2024-04-26T20:56:58.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.436+0000 7f900cc00640 1 -- 172.21.15.28:0/1096148521 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f8ffc092050 con 0x7f9008005e90 2024-04-26T20:56:58.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.436+0000 7fc51aa00640 1 -- 172.21.15.28:0/1975667636 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3249092476 0 0) 0x7fc51c072990 con 0x7fc51c14c460 2024-04-26T20:56:58.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.436+0000 7fc51aa00640 1 -- 172.21.15.28:0/1975667636 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc500003520 con 0x7fc51c14c460 2024-04-26T20:56:58.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.436+0000 7fc51aa00640 1 -- 172.21.15.28:0/1975667636 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 371196466 0 0) 0x7fc51c072c70 con 0x7fc51c14cae0 2024-04-26T20:56:58.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.436+0000 7fc51aa00640 1 -- 172.21.15.28:0/1975667636 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc51c072990 con 0x7fc51c14cae0 2024-04-26T20:56:58.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.436+0000 7fa467e00640 1 --2- 172.21.15.28:0/1410878323 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4500039d0 0x7fa450003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.437+0000 7ff1f9600640 1 --2- 172.21.15.28:0/3720906340 >> v2:172.21.15.103:3300/0 conn(0x7ff1f40cf2b0 0x7ff1f40cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.436+0000 7fa467e00640 1 -- 172.21.15.28:0/1410878323 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa468164ac0 con 0x7fa450004800 2024-04-26T20:56:58.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.436+0000 7fa46dc00640 1 -- 172.21.15.28:0/1410878323 reap_dead start 2024-04-26T20:56:58.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.436+0000 7fa46d200640 1 --2- 172.21.15.28:0/1410878323 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa4500039d0 0x7fa450003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:56:58.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.437+0000 7f9015c00640 1 --2- 172.21.15.28:0/1096148521 >> [v2:172.21.15.28:6808/993399689,v1:172.21.15.28:6809/993399689] conn(0x7f8ff0074be0 0x7f8ff0077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.437+0000 7f13ffa00640 1 -- 172.21.15.28:0/382665288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13f806ca60 msgr2=0x7f13f80b38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.437+0000 7f13ffa00640 1 --2- 172.21.15.28:0/382665288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13f806ca60 0x7f13f80b38c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f13f00097a0 tx=0x7f13f002a110 comp rx=0 tx=0).stop 2024-04-26T20:56:58.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.436+0000 7fc521000640 1 -- 172.21.15.28:0/1975667636 >> v1:172.21.15.103:6789/0 conn(0x7fc51c14cae0 legacy=0x7fc51c06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.436+0000 7fc521000640 1 -- 172.21.15.28:0/1975667636 >> v1:172.21.15.28:6789/0 conn(0x7fc51c14c460 legacy=0x7fc51c072020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.436+0000 7fc521000640 1 -- 172.21.15.28:0/1975667636 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51c072f50 con 0x7fc51c06fcf0 2024-04-26T20:56:58.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.436+0000 7fc7aea00640 1 -- 172.21.15.28:0/370158799 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7b001a020 con 0x7fc79c00d550 2024-04-26T20:56:58.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.438+0000 7f9015c00640 1 --2- 172.21.15.28:0/1096148521 >> [v2:172.21.15.28:6808/993399689,v1:172.21.15.28:6809/993399689] conn(0x7f8ff0074be0 0x7f8ff0077000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.438+0000 7fa467e00640 1 --2- 172.21.15.28:0/1410878323 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa450004800 0x7fa450004c80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa454009db0 tx=0x7fa450005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43ee39380a257e77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.438+0000 7f900fe00640 1 --2- 172.21.15.28:0/1096148521 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8ff006c9c0 0x7f8ff006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.438+0000 7fd605400640 1 Processor -- start 2024-04-26T20:56:58.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.437+0000 7ff1f8c00640 1 -- 172.21.15.28:0/3720906340 >> v2:172.21.15.103:3300/0 conn(0x7ff1f40cf2b0 msgr2=0x7ff1f40cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.437+0000 7ff1f8c00640 1 --2- 172.21.15.28:0/3720906340 >> v2:172.21.15.103:3300/0 conn(0x7ff1f40cf2b0 0x7ff1f40cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.437+0000 7ff1f8c00640 1 -- 172.21.15.28:0/3720906340 >> v2:172.21.15.28:3300/0 conn(0x7ff1f40d0180 msgr2=0x7ff1f40d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.437+0000 7ff1f8c00640 1 --2- 172.21.15.28:0/3720906340 >> v2:172.21.15.28:3300/0 conn(0x7ff1f40d0180 0x7ff1f40d05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.438+0000 7fc7aea00640 1 --2- 172.21.15.28:0/370158799 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc79806c940 0x7fc79806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.438+0000 7fc7aea00640 1 -- 172.21.15.28:0/370158799 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc7b008b4a0 con 0x7fc79c00d550 2024-04-26T20:56:58.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.439+0000 7fc7aea00640 1 --2- 172.21.15.28:0/370158799 >> [v2:172.21.15.28:6800/3928805442,v1:172.21.15.28:6801/3928805442] conn(0x7fc798074bc0 0x7fc798076fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.439+0000 7fc7aea00640 1 -- 172.21.15.28:0/370158799 --> [v2:172.21.15.28:6800/3928805442,v1:172.21.15.28:6801/3928805442] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc7b0011e50 con 0x7fc798074bc0 2024-04-26T20:56:58.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.439+0000 7fc7aea00640 1 -- 172.21.15.28:0/370158799 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc7b0092050 con 0x7fc79c00d550 2024-04-26T20:56:58.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.439+0000 7fc7b6000640 1 --2- 172.21.15.28:0/370158799 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc79806c940 0x7fc79806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.439+0000 7f13ffa00640 1 -- 172.21.15.28:0/382665288 shutdown_connections 2024-04-26T20:56:58.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.439+0000 7f13ffa00640 1 --2- 172.21.15.28:0/382665288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13f806ca60 0x7f13f80b38c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.439+0000 7f13ffa00640 1 --2- 172.21.15.28:0/382665288 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13f815efd0 0x7f13f80ba770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.439+0000 7f13ffa00640 1 -- 172.21.15.28:0/382665288 >> 172.21.15.28:0/382665288 conn(0x7f13f80d4e50 msgr2=0x7f13f80d5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.437+0000 7ff1f8c00640 1 -- 172.21.15.28:0/3720906340 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f415c0e0 con 0x7ff1f415afc0 2024-04-26T20:56:58.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.439+0000 7fa467400640 1 -- 172.21.15.28:0/1410878323 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa454036070 con 0x7fa450004800 2024-04-26T20:56:58.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.439+0000 7fa467400640 1 -- 172.21.15.28:0/1410878323 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa454031400 con 0x7fa450004800 2024-04-26T20:56:58.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.439+0000 7fa467400640 1 -- 172.21.15.28:0/1410878323 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4540099b0 con 0x7fa450004800 2024-04-26T20:56:58.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.439+0000 7f900cc00640 1 -- 172.21.15.28:0/1096148521 <== osd.0 v2:172.21.15.28:6808/993399689 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ff00776b0 con 0x7f8ff0074be0 2024-04-26T20:56:58.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.439+0000 7fc521000640 1 --2- 172.21.15.28:0/1975667636 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc51c06fcf0 0x7fc51c16e090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009dc0 tx=0x7fc50c02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97b957d5b3884d09 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.438+0000 7fd605400640 1 -- start start 2024-04-26T20:56:58.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.439+0000 7f13ffa00640 1 -- 172.21.15.28:0/382665288 shutdown_connections 2024-04-26T20:56:58.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.439+0000 7f13ffa00640 1 -- 172.21.15.28:0/382665288 wait complete. 2024-04-26T20:56:58.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.439+0000 7fc7b7400640 1 --2- 172.21.15.28:0/370158799 >> [v2:172.21.15.28:6800/3928805442,v1:172.21.15.28:6801/3928805442] conn(0x7fc798074bc0 0x7fc798076fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.440+0000 7fc51aa00640 1 -- 172.21.15.28:0/1975667636 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c02ae60 con 0x7fc51c06fcf0 2024-04-26T20:56:58.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.438+0000 7ff1f8c00640 1 --2- 172.21.15.28:0/3720906340 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff1f415afc0 0x7ff1f415c850 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff1f415ebb0 tx=0x7ff1e002e420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bdbc24461f089b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.440+0000 7fc51aa00640 1 -- 172.21.15.28:0/1975667636 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc50c007740 con 0x7fc51c06fcf0 2024-04-26T20:56:58.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.440+0000 7fc51aa00640 1 -- 172.21.15.28:0/1975667636 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c007b10 con 0x7fc51c06fcf0 2024-04-26T20:56:58.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.440+0000 7ff1f3400640 1 -- 172.21.15.28:0/3720906340 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e003a070 con 0x7ff1f415afc0 2024-04-26T20:56:58.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.440+0000 7ff1f3400640 1 -- 172.21.15.28:0/3720906340 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1e0008df0 con 0x7ff1f415afc0 2024-04-26T20:56:58.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.440+0000 7f900fe00640 1 --2- 172.21.15.28:0/1096148521 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8ff006c9c0 0x7f8ff006ee80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f900804f050 tx=0x7f9004009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.440+0000 7ff1f3400640 1 -- 172.21.15.28:0/3720906340 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e000a2d0 con 0x7ff1f415afc0 2024-04-26T20:56:58.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.440+0000 7fa46e600640 1 -- 172.21.15.28:0/1410878323 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa450004800 msgr2=0x7fa450004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.440+0000 7fa46e600640 1 --2- 172.21.15.28:0/1410878323 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa450004800 0x7fa450004c80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa454009db0 tx=0x7fa450005830 comp rx=0 tx=0).stop 2024-04-26T20:56:58.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.440+0000 7fd605400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd6000cb270 0x7fd6001f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.440+0000 7fd605400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6001f4790 0x7fd6001f4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7fd605400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd600070fb0 0x7fd600071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7fd605400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd6000bacb0 con 0x7fd6001f4790 2024-04-26T20:56:58.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.440+0000 7fa46e600640 1 -- 172.21.15.28:0/1410878323 shutdown_connections 2024-04-26T20:56:58.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.440+0000 7fa46e600640 1 --2- 172.21.15.28:0/1410878323 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa450004800 0x7fa450004c80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.440+0000 7fa46e600640 1 -- 172.21.15.28:0/1410878323 >> 172.21.15.28:0/1410878323 conn(0x7fa46806c770 msgr2=0x7fa46806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7fd605400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd6000d5ba0 con 0x7fd600070fb0 2024-04-26T20:56:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7fd605400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fd6000bae30 con 0x7fd6000cb270 2024-04-26T20:56:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7ff1fb600640 1 -- 172.21.15.28:0/3720906340 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff1f415afc0 msgr2=0x7ff1f415c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7ff1fb600640 1 --2- 172.21.15.28:0/3720906340 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff1f415afc0 0x7ff1f415c850 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff1f415ebb0 tx=0x7ff1e002e420 comp rx=0 tx=0).stop 2024-04-26T20:56:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7ff1fb600640 1 -- 172.21.15.28:0/3720906340 shutdown_connections 2024-04-26T20:56:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7ff1fb600640 1 --2- 172.21.15.28:0/3720906340 >> v2:172.21.15.103:3300/0 conn(0x7ff1f40cf2b0 0x7ff1f40cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7ff1fb600640 1 --2- 172.21.15.28:0/3720906340 >> v2:172.21.15.28:3300/0 conn(0x7ff1f40d0180 0x7ff1f40d05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7ff1fb600640 1 --2- 172.21.15.28:0/3720906340 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff1f415afc0 0x7ff1f415c850 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7ff1fb600640 1 -- 172.21.15.28:0/3720906340 >> 172.21.15.28:0/3720906340 conn(0x7ff1f40b2a80 msgr2=0x7ff1f40b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7fd5fe800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd6000cb270 0x7fd6001f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7fd5fe800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd6000cb270 0x7fd6001f4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.28:59240/0 (socket says 172.21.15.28:59240) 2024-04-26T20:56:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7fd5fe800640 1 -- 172.21.15.28:0/3780653389 learned_addr learned my addr 172.21.15.28:0/3780653389 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7fd5fde00640 1 --2- 172.21.15.28:0/3780653389 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6001f4790 0x7fd6001f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7fc523000640 1 -- 172.21.15.28:0/1975667636 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc51c06fcf0 msgr2=0x7fc51c16e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7fc523000640 1 --2- 172.21.15.28:0/1975667636 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc51c06fcf0 0x7fc51c16e090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009dc0 tx=0x7fc50c02a680 comp rx=0 tx=0).stop 2024-04-26T20:56:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7ff1fb600640 1 -- 172.21.15.28:0/3720906340 shutdown_connections 2024-04-26T20:56:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7fc523000640 1 -- 172.21.15.28:0/1975667636 shutdown_connections 2024-04-26T20:56:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7fc523000640 1 --2- 172.21.15.28:0/1975667636 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc51c06fcf0 0x7fc51c16e090 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7fc523000640 1 -- 172.21.15.28:0/1975667636 >> 172.21.15.28:0/1975667636 conn(0x7fc51c0b2a80 msgr2=0x7fc51c0b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.440+0000 7fc7b6000640 1 --2- 172.21.15.28:0/370158799 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc79806c940 0x7fc79806ee00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8147190 tx=0x7fc7a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.441+0000 7fc7b7400640 1 --2- 172.21.15.28:0/370158799 >> [v2:172.21.15.28:6800/3928805442,v1:172.21.15.28:6801/3928805442] conn(0x7fc798074bc0 0x7fc798076fe0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.442+0000 7fa46e600640 1 -- 172.21.15.28:0/1410878323 shutdown_connections 2024-04-26T20:56:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.442+0000 7f13ffa00640 1 Processor -- start 2024-04-26T20:56:58.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.442+0000 7fd5fde00640 1 -- 172.21.15.28:0/3780653389 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd6000cb270 msgr2=0x7fd6001f4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.442+0000 7fd5fde00640 1 --2- 172.21.15.28:0/3780653389 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd6000cb270 0x7fd6001f4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.442+0000 7fd5fde00640 1 -- 172.21.15.28:0/3780653389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd600070fb0 msgr2=0x7fd600071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.442+0000 7fd5fde00640 1 --2- 172.21.15.28:0/3780653389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd600070fb0 0x7fd600071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.442+0000 7fd5fde00640 1 -- 172.21.15.28:0/3780653389 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6000ccb60 con 0x7fd6001f4790 2024-04-26T20:56:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.442+0000 7fc523000640 1 -- 172.21.15.28:0/1975667636 shutdown_connections 2024-04-26T20:56:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.442+0000 7ff1fb600640 1 -- 172.21.15.28:0/3720906340 wait complete. 2024-04-26T20:56:58.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.442+0000 7f13ffa00640 1 -- start start 2024-04-26T20:56:58.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.443+0000 7f0fc9800640 1 Processor -- start 2024-04-26T20:56:58.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.443+0000 7fd5fde00640 1 --2- 172.21.15.28:0/3780653389 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6001f4790 0x7fd6001f4c10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00e820 tx=0x7fd5ec00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.443+0000 7fc7aea00640 1 -- 172.21.15.28:0/370158799 <== osd.1 v2:172.21.15.28:6800/3928805442 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc7b0011e50 con 0x7fc798074bc0 2024-04-26T20:56:58.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.443+0000 7fd5f6a00640 1 -- 172.21.15.28:0/3780653389 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec017070 con 0x7fd6001f4790 2024-04-26T20:56:58.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.443+0000 7f0fc9800640 1 -- start start 2024-04-26T20:56:58.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.443+0000 7fd5f6a00640 1 -- 172.21.15.28:0/3780653389 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5ec00ee70 con 0x7fd6001f4790 2024-04-26T20:56:58.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.443+0000 7fd605400640 1 -- 172.21.15.28:0/3780653389 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd60015a3e0 con 0x7fd6001f4790 2024-04-26T20:56:58.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.443+0000 7fd605400640 1 -- 172.21.15.28:0/3780653389 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6000707b0 con 0x7fd6001f4790 2024-04-26T20:56:58.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.443+0000 7fd5f6a00640 1 -- 172.21.15.28:0/3780653389 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec00cb40 con 0x7fd6001f4790 2024-04-26T20:56:58.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.444+0000 7fa46e600640 1 -- 172.21.15.28:0/1410878323 wait complete. 2024-04-26T20:56:58.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.444+0000 7f13ffa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13f806ca60 0x7f13f80cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.444+0000 7f13ffa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13f815efd0 0x7f13f8070e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.444+0000 7f13ffa00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13f815f3b0 0x7f13f80713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.444+0000 7ff1fb600640 1 Processor -- start 2024-04-26T20:56:58.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.444+0000 7f13ffa00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f13f80bacb0 con 0x7f13f806ca60 2024-04-26T20:56:58.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.444+0000 7f13ffa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f13f80d5ba0 con 0x7f13f815efd0 2024-04-26T20:56:58.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.444+0000 7f13ffa00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f13f80bae30 con 0x7f13f815f3b0 2024-04-26T20:56:58.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.444+0000 7ff1fb600640 1 -- start start 2024-04-26T20:56:58.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.442+0000 7fc523000640 1 -- 172.21.15.28:0/1975667636 wait complete. 2024-04-26T20:56:58.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.445+0000 7fc523000640 1 Processor -- start 2024-04-26T20:56:58.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.445+0000 7fd605400640 1 -- 172.21.15.28:0/3780653389 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd5d8001030 con 0x7fd6001f4790 2024-04-26T20:56:58.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.445+0000 7fc523000640 1 -- start start 2024-04-26T20:56:58.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.446+0000 7ff1fb600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff1f40cf2b0 0x7ff1f41535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.446+0000 7ff1fb600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1f40d0180 0x7ff1f41527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.446+0000 7ff1fb600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1f415afc0 0x7ff1f40aa590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.446+0000 7ff1fb600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f40ced90 con 0x7ff1f415afc0 2024-04-26T20:56:58.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.446+0000 7ff1fb600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f40ceab0 con 0x7ff1f40d0180 2024-04-26T20:56:58.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.446+0000 7ff1fb600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f40cfc80 con 0x7ff1f40cf2b0 2024-04-26T20:56:58.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.446+0000 7ff1f3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1f40d0180 0x7ff1f41527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.447+0000 7ff1f9600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1f415afc0 0x7ff1f40aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.447+0000 7ff1f9600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1f415afc0 0x7ff1f40aa590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42176/0 (socket says 172.21.15.28:42176) 2024-04-26T20:56:58.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.447+0000 7ff1f9600640 1 -- 172.21.15.28:0/3873108524 learned_addr learned my addr 172.21.15.28:0/3873108524 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.446+0000 7ff1f8c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff1f40cf2b0 0x7ff1f41535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.445+0000 7f13fda00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13f815f3b0 0x7f13f80713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.445+0000 7f13fda00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13f815f3b0 0x7f13f80713c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.28:59242/0 (socket says 172.21.15.28:59242) 2024-04-26T20:56:58.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.445+0000 7f13fda00640 1 -- 172.21.15.28:0/1451789063 learned_addr learned my addr 172.21.15.28:0/1451789063 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.445+0000 7f13efe00640 1 --2- 172.21.15.28:0/1451789063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13f815efd0 0x7f13f8070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.446+0000 7f13fda00640 1 -- 172.21.15.28:0/1451789063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13f815efd0 msgr2=0x7f13f8070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.446+0000 7f13fda00640 1 --2- 172.21.15.28:0/1451789063 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13f815efd0 0x7f13f8070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.446+0000 7f13fda00640 1 -- 172.21.15.28:0/1451789063 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13f806ca60 msgr2=0x7f13f80cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.446+0000 7f13fda00640 1 --2- 172.21.15.28:0/1451789063 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13f806ca60 0x7f13f80cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.446+0000 7f13fda00640 1 -- 172.21.15.28:0/1451789063 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f80cdf60 con 0x7f13f815f3b0 2024-04-26T20:56:58.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.447+0000 7f13fda00640 1 --2- 172.21.15.28:0/1451789063 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13f815f3b0 0x7f13f80713c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f13f002a700 tx=0x7f13f00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.447+0000 7f13f5600640 1 -- 172.21.15.28:0/1451789063 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0037040 con 0x7f13f815f3b0 2024-04-26T20:56:58.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.447+0000 7f13f5600640 1 -- 172.21.15.28:0/1451789063 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f0004230 con 0x7f13f815f3b0 2024-04-26T20:56:58.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.447+0000 7f13f5600640 1 -- 172.21.15.28:0/1451789063 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f0003610 con 0x7f13f815f3b0 2024-04-26T20:56:58.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.448+0000 7f13ffa00640 1 -- 172.21.15.28:0/1451789063 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f80ce240 con 0x7f13f815f3b0 2024-04-26T20:56:58.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.448+0000 7f13ffa00640 1 -- 172.21.15.28:0/1451789063 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f80ce800 con 0x7f13f815f3b0 2024-04-26T20:56:58.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.447+0000 7fc523000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc51c06fcf0 0x7fc51c13fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.447+0000 7fc523000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc51c14c460 0x7fc51c163a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.448+0000 7fc523000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc51c163f50 0x7fc51c166100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.448+0000 7f0fc9800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc406ee50 0x7f0fc4073400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.448+0000 7f0fc9800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fc40b3f20 con 0x7f0fc4073940 2024-04-26T20:56:58.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.448+0000 7f0fc9800640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fc40b4200 con 0x7f0fc415afc0 2024-04-26T20:56:58.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.448+0000 7f0fc9800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc40b4850 con 0x7f0fc406ee50 2024-04-26T20:56:58.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.448+0000 7fc51be00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc51c06fcf0 0x7fc51c13fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.448+0000 7fc51be00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc51c06fcf0 0x7fc51c13fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.28:59258/0 (socket says 172.21.15.28:59258) 2024-04-26T20:56:58.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.447+0000 7ff1f9600640 1 -- 172.21.15.28:0/3873108524 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff1f40cf2b0 msgr2=0x7ff1f41535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.447+0000 7ff1f9600640 1 --2- 172.21.15.28:0/3873108524 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff1f40cf2b0 0x7ff1f41535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.448+0000 7ff1f9600640 1 -- 172.21.15.28:0/3873108524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1f40d0180 msgr2=0x7ff1f41527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.448+0000 7ff1f9600640 1 --2- 172.21.15.28:0/3873108524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1f40d0180 0x7ff1f41527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.448+0000 7ff1f9600640 1 -- 172.21.15.28:0/3873108524 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f4153cd0 con 0x7ff1f415afc0 2024-04-26T20:56:58.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.448+0000 7ff1f8c00640 1 --2- 172.21.15.28:0/3873108524 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff1f40cf2b0 0x7ff1f41535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:56:58.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.448+0000 7ff1f3e00640 1 --2- 172.21.15.28:0/3873108524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1f40d0180 0x7ff1f41527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:56:58.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.448+0000 7fc521000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc51c163f50 0x7fc51c166100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.448+0000 7fc521000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc51c163f50 0x7fc51c166100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42184/0 (socket says 172.21.15.28:42184) 2024-04-26T20:56:58.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.448+0000 7fc521000640 1 -- 172.21.15.28:0/250069524 learned_addr learned my addr 172.21.15.28:0/250069524 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.448+0000 7ff1f9600640 1 --2- 172.21.15.28:0/3873108524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1f415afc0 0x7ff1f40aa590 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00efc0 tx=0x7ff1ec00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.448+0000 7fc523000640 1 -- 172.21.15.28:0/250069524 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c072730 con 0x7fc51c163f50 2024-04-26T20:56:58.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.448+0000 7fc523000640 1 -- 172.21.15.28:0/250069524 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc51c0735a0 con 0x7fc51c14c460 2024-04-26T20:56:58.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.449+0000 7f0fc2a00640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f0fc415afc0 0x7f0fc406e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.28:36062/0 (socket says 172.21.15.28:36062) 2024-04-26T20:56:58.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.449+0000 7f0fc2a00640 1 -- 172.21.15.28:0/3508438198 learned_addr learned my addr 172.21.15.28:0/3508438198 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:58.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.449+0000 7f0fc2000640 1 --2- 172.21.15.28:0/3508438198 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc406ee50 0x7f0fc4073400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.449+0000 7fa46e600640 1 Processor -- start 2024-04-26T20:56:58.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.449+0000 7fc523000640 1 -- 172.21.15.28:0/250069524 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc51c0cf2f0 con 0x7fc51c06fcf0 2024-04-26T20:56:58.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.449+0000 7fc51b400640 1 --2- 172.21.15.28:0/250069524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc51c14c460 0x7fc51c163a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.449+0000 7fa46e600640 1 -- start start 2024-04-26T20:56:58.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.450+0000 7f13ffa00640 1 -- 172.21.15.28:0/1451789063 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f13f8076520 con 0x7f13f815f3b0 2024-04-26T20:56:58.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.450+0000 7f0fc1600640 1 -- 172.21.15.28:0/3508438198 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4166737483 0 0) 0x7f0fc40b3f20 con 0x7f0fc4073940 2024-04-26T20:56:58.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.450+0000 7f0fc2000640 1 -- 172.21.15.28:0/3508438198 >> v1:172.21.15.103:6789/0 conn(0x7f0fc415afc0 legacy=0x7f0fc406e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.449+0000 7ff1f0c00640 1 -- 172.21.15.28:0/3873108524 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec018020 con 0x7ff1f415afc0 2024-04-26T20:56:58.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.450+0000 7ff1f0c00640 1 -- 172.21.15.28:0/3873108524 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ec014480 con 0x7ff1f415afc0 2024-04-26T20:56:58.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.450+0000 7ff1f0c00640 1 -- 172.21.15.28:0/3873108524 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec014870 con 0x7ff1f415afc0 2024-04-26T20:56:58.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.450+0000 7f0fc2000640 1 -- 172.21.15.28:0/3508438198 >> v1:172.21.15.28:6789/0 conn(0x7f0fc4073940 legacy=0x7f0fc40ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.450+0000 7f0fc2000640 1 -- 172.21.15.28:0/3508438198 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc40b44e0 con 0x7f0fc406ee50 2024-04-26T20:56:58.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.450+0000 7ff1fb600640 1 -- 172.21.15.28:0/3873108524 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f4153fb0 con 0x7ff1f415afc0 2024-04-26T20:56:58.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.450+0000 7ff1fb600640 1 -- 172.21.15.28:0/3873108524 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f4154570 con 0x7ff1f415afc0 2024-04-26T20:56:58.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.449+0000 7fd5f6a00640 1 -- 172.21.15.28:0/3780653389 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd5ec01a070 con 0x7fd6001f4790 2024-04-26T20:56:58.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.451+0000 7ff1fb600640 1 -- 172.21.15.28:0/3873108524 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff1f406a400 con 0x7ff1f415afc0 2024-04-26T20:56:58.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.452+0000 7fc51be00640 1 -- 172.21.15.28:0/250069524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc51c14c460 msgr2=0x7fc51c163a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.452+0000 7fc51be00640 1 --2- 172.21.15.28:0/250069524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc51c14c460 0x7fc51c163a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.452+0000 7fc51be00640 1 -- 172.21.15.28:0/250069524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc51c163f50 msgr2=0x7fc51c166100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.452+0000 7fc51be00640 1 --2- 172.21.15.28:0/250069524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc51c163f50 0x7fc51c166100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.452+0000 7fa46e600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa450004800 0x7fa4681ff790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.452+0000 7fc51be00640 1 -- 172.21.15.28:0/250069524 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51c143ea0 con 0x7fc51c06fcf0 2024-04-26T20:56:58.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.452+0000 7fa46e600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa468205180 0x7fa4680d2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.452+0000 7fa46e600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4680d31f0 0x7fa468203390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.452+0000 7fa46e600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa4680cf630 con 0x7fa468205180 2024-04-26T20:56:58.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.452+0000 7fa46e600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa4680cf4b0 con 0x7fa4680d31f0 2024-04-26T20:56:58.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.452+0000 7fc51b400640 1 --2- 172.21.15.28:0/250069524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc51c14c460 0x7fc51c163a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:56:58.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.452+0000 7fa46e600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa4680cf7b0 con 0x7fa450004800 2024-04-26T20:56:58.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.453+0000 7f13f5600640 1 -- 172.21.15.28:0/1451789063 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13f00037b0 con 0x7f13f815f3b0 2024-04-26T20:56:58.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.453+0000 7fa467e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa468205180 0x7fa4680d2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.453+0000 7fc51be00640 1 --2- 172.21.15.28:0/250069524 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc51c06fcf0 0x7fc51c13fe30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc50800c3a0 tx=0x7fc50800c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.453+0000 7fa46d200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa450004800 0x7fa4681ff790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.453+0000 7fd5f6a00640 1 --2- 172.21.15.28:0/3780653389 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd5e006c920 0x7fd5e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.453+0000 7fa467e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa468205180 0x7fa4680d2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42196/0 (socket says 172.21.15.28:42196) 2024-04-26T20:56:58.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.453+0000 7fa46d200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa450004800 0x7fa4681ff790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.28:59274/0 (socket says 172.21.15.28:59274) 2024-04-26T20:56:58.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.453+0000 7fa46d200640 1 -- 172.21.15.28:0/1109700614 learned_addr learned my addr 172.21.15.28:0/1109700614 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.453+0000 7fc513e00640 1 -- 172.21.15.28:0/250069524 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc508017070 con 0x7fc51c06fcf0 2024-04-26T20:56:58.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.453+0000 7fd5f6a00640 1 -- 172.21.15.28:0/3780653389 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd5ec08b7e0 con 0x7fd6001f4790 2024-04-26T20:56:58.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.453+0000 7fc513e00640 1 -- 172.21.15.28:0/250069524 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc508005ce0 con 0x7fc51c06fcf0 2024-04-26T20:56:58.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.453+0000 7fc513e00640 1 -- 172.21.15.28:0/250069524 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5080072d0 con 0x7fc51c06fcf0 2024-04-26T20:56:58.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.453+0000 7fa46dc00640 1 --2- 172.21.15.28:0/1109700614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4680d31f0 0x7fa468203390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.453+0000 7f0fc2000640 1 --2- 172.21.15.28:0/3508438198 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc406ee50 0x7f0fc4073400 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00097d0 tx=0x7f0fb002c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f8d0c4d6c12f1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.453+0000 7f0fc1600640 1 -- 172.21.15.28:0/3508438198 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0038070 con 0x7f0fc406ee50 2024-04-26T20:56:58.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.454+0000 7f0fc1600640 1 -- 172.21.15.28:0/3508438198 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb0031710 con 0x7f0fc406ee50 2024-04-26T20:56:58.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.454+0000 7f0fc1600640 1 -- 172.21.15.28:0/3508438198 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0038070 con 0x7f0fc406ee50 2024-04-26T20:56:58.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.453+0000 7fd5f6a00640 1 --2- 172.21.15.28:0/3780653389 >> [v2:172.21.15.28:6818/2569592417,v1:172.21.15.28:6820/2569592417] conn(0x7fd5e0074ac0 0x7fd5e0076ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.454+0000 7fd5f6a00640 1 -- 172.21.15.28:0/3780653389 --> [v2:172.21.15.28:6818/2569592417,v1:172.21.15.28:6820/2569592417] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd5e0077590 con 0x7fd5e0074ac0 2024-04-26T20:56:58.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.454+0000 7fc523000640 1 -- 172.21.15.28:0/250069524 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51c164580 con 0x7fc51c06fcf0 2024-04-26T20:56:58.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.454+0000 7fd5f6a00640 1 -- 172.21.15.28:0/3780653389 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fd5ec0902c0 con 0x7fd6001f4790 2024-04-26T20:56:58.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.454+0000 7fa46d200640 1 -- 172.21.15.28:0/1109700614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4680d31f0 msgr2=0x7fa468203390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.454+0000 7fc523000640 1 -- 172.21.15.28:0/250069524 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc51c164b40 con 0x7fc51c06fcf0 2024-04-26T20:56:58.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.454+0000 7fa46d200640 1 --2- 172.21.15.28:0/1109700614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa4680d31f0 0x7fa468203390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.454+0000 7fa46d200640 1 -- 172.21.15.28:0/1109700614 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa468205180 msgr2=0x7fa4680d2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.454+0000 7fa46d200640 1 --2- 172.21.15.28:0/1109700614 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa468205180 0x7fa4680d2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.454+0000 7fa46d200640 1 -- 172.21.15.28:0/1109700614 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4682040e0 con 0x7fa450004800 2024-04-26T20:56:58.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.455+0000 7f0fc9800640 1 -- 172.21.15.28:0/3508438198 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc406ee50 msgr2=0x7f0fc4073400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.455+0000 7f0fc9800640 1 --2- 172.21.15.28:0/3508438198 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc406ee50 0x7f0fc4073400 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0fb00097d0 tx=0x7f0fb002c440 comp rx=0 tx=0).stop 2024-04-26T20:56:58.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.454+0000 7f13f5600640 1 --2- 172.21.15.28:0/1451789063 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f13d406c940 0x7f13d406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.454+0000 7f13f5600640 1 -- 172.21.15.28:0/1451789063 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f13f00ad360 con 0x7f13f815f3b0 2024-04-26T20:56:58.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.454+0000 7f13f5600640 1 --2- 172.21.15.28:0/1451789063 >> [v2:172.21.15.28:6816/2056655204,v1:172.21.15.28:6817/2056655204] conn(0x7f13d4074b60 0x7f13d4076f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.455+0000 7fd5fe800640 1 --2- 172.21.15.28:0/3780653389 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd5e006c920 0x7fd5e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.455+0000 7fd5ff200640 1 --2- 172.21.15.28:0/3780653389 >> [v2:172.21.15.28:6818/2569592417,v1:172.21.15.28:6820/2569592417] conn(0x7fd5e0074ac0 0x7fd5e0076ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.455+0000 7f13f5600640 1 -- 172.21.15.28:0/1451789063 --> [v2:172.21.15.28:6816/2056655204,v1:172.21.15.28:6817/2056655204] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f13f0008eb0 con 0x7f13d4074b60 2024-04-26T20:56:58.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.455+0000 7f13f5600640 1 -- 172.21.15.28:0/1451789063 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f13f00ad710 con 0x7f13f815f3b0 2024-04-26T20:56:58.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.455+0000 7f13fd000640 1 --2- 172.21.15.28:0/1451789063 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f13d406c940 0x7f13d406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.455+0000 7fc523000640 1 -- 172.21.15.28:0/250069524 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc51c06a400 con 0x7fc51c06fcf0 2024-04-26T20:56:58.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.455+0000 7fa46d200640 1 --2- 172.21.15.28:0/1109700614 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa450004800 0x7fa4681ff790 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa45c00fae0 tx=0x7fa45c005940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.455+0000 7fa464c00640 1 -- 172.21.15.28:0/1109700614 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45c018030 con 0x7fa450004800 2024-04-26T20:56:58.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.455+0000 7fa46e600640 1 -- 172.21.15.28:0/1109700614 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4682043c0 con 0x7fa450004800 2024-04-26T20:56:58.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.456+0000 7fa46e600640 1 -- 172.21.15.28:0/1109700614 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa468204980 con 0x7fa450004800 2024-04-26T20:56:58.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.456+0000 7fa464c00640 1 -- 172.21.15.28:0/1109700614 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa45c006b50 con 0x7fa450004800 2024-04-26T20:56:58.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.456+0000 7fa464c00640 1 -- 172.21.15.28:0/1109700614 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45c0040a0 con 0x7fa450004800 2024-04-26T20:56:58.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.456+0000 7f0fc9800640 1 -- 172.21.15.28:0/3508438198 shutdown_connections 2024-04-26T20:56:58.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.456+0000 7f0fc9800640 1 --2- 172.21.15.28:0/3508438198 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc406ee50 0x7f0fc4073400 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.456+0000 7f0fc9800640 1 -- 172.21.15.28:0/3508438198 >> 172.21.15.28:0/3508438198 conn(0x7f0fc406c770 msgr2=0x7f0fc406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.456+0000 7f0fc9800640 1 -- 172.21.15.28:0/3508438198 shutdown_connections 2024-04-26T20:56:58.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.457+0000 7f0fc9800640 1 -- 172.21.15.28:0/3508438198 wait complete. 2024-04-26T20:56:58.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.457+0000 7f13efe00640 1 --2- 172.21.15.28:0/1451789063 >> [v2:172.21.15.28:6816/2056655204,v1:172.21.15.28:6817/2056655204] conn(0x7f13d4074b60 0x7f13d4076f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.457+0000 7fa46e600640 1 -- 172.21.15.28:0/1109700614 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa444001030 con 0x7fa450004800 2024-04-26T20:56:58.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.458+0000 7f13efe00640 1 --2- 172.21.15.28:0/1451789063 >> [v2:172.21.15.28:6816/2056655204,v1:172.21.15.28:6817/2056655204] conn(0x7f13d4074b60 0x7f13d4076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.457+0000 7fd5fe800640 1 --2- 172.21.15.28:0/3780653389 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd5e006c920 0x7fd5e006ede0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000bb40 tx=0x7fd5f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.457+0000 7fd5ff200640 1 --2- 172.21.15.28:0/3780653389 >> [v2:172.21.15.28:6818/2569592417,v1:172.21.15.28:6820/2569592417] conn(0x7fd5e0074ac0 0x7fd5e0076ee0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.457+0000 7ff1f0c00640 1 -- 172.21.15.28:0/3873108524 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff1ec005d10 con 0x7ff1f415afc0 2024-04-26T20:56:58.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.459+0000 7fd5f6a00640 1 -- 172.21.15.28:0/3780653389 <== osd.3 v2:172.21.15.28:6818/2569592417 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd5e0077590 con 0x7fd5e0074ac0 2024-04-26T20:56:58.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.459+0000 7ff1f0c00640 1 --2- 172.21.15.28:0/3873108524 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff1d406c9c0 0x7ff1d406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.459+0000 7ff1f0c00640 1 -- 172.21.15.28:0/3873108524 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff1ec08cd70 con 0x7ff1f415afc0 2024-04-26T20:56:58.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.459+0000 7ff1f0c00640 1 --2- 172.21.15.28:0/3873108524 >> [v2:172.21.15.103:6800/4239183489,v1:172.21.15.103:6801/4239183489] conn(0x7ff1d4074be0 0x7ff1d4077000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.459+0000 7ff1f0c00640 1 -- 172.21.15.28:0/3873108524 --> [v2:172.21.15.103:6800/4239183489,v1:172.21.15.103:6801/4239183489] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff1ec012e50 con 0x7ff1d4074be0 2024-04-26T20:56:58.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.459+0000 7ff1f0c00640 1 -- 172.21.15.28:0/3873108524 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff1ec08d160 con 0x7ff1f415afc0 2024-04-26T20:56:58.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.460+0000 7f13fd000640 1 --2- 172.21.15.28:0/1451789063 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f13d406c940 0x7f13d406ee00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f13e80070b0 tx=0x7f13e8007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.460+0000 7fc513e00640 1 -- 172.21.15.28:0/250069524 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc50801a070 con 0x7fc51c06fcf0 2024-04-26T20:56:58.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.460+0000 7f13f5600640 1 -- 172.21.15.28:0/1451789063 <== osd.2 v2:172.21.15.28:6816/2056655204 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f13f0008eb0 con 0x7f13d4074b60 2024-04-26T20:56:58.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.460+0000 7ff1f8c00640 1 --2- 172.21.15.28:0/3873108524 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff1d406c9c0 0x7ff1d406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.459+0000 7f0fc9800640 1 Processor -- start 2024-04-26T20:56:58.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.460+0000 7f0fc9800640 1 -- start start 2024-04-26T20:56:58.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.460+0000 7fa464c00640 1 -- 172.21.15.28:0/1109700614 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa45c01b070 con 0x7fa450004800 2024-04-26T20:56:58.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.461+0000 7ff1f3e00640 1 --2- 172.21.15.28:0/3873108524 >> [v2:172.21.15.103:6800/4239183489,v1:172.21.15.103:6801/4239183489] conn(0x7ff1d4074be0 0x7ff1d4077000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.462+0000 7fc513e00640 1 --2- 172.21.15.28:0/250069524 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc4fc06c920 0x7fc4fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.462+0000 7fc513e00640 1 -- 172.21.15.28:0/250069524 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc50808af90 con 0x7fc51c06fcf0 2024-04-26T20:56:58.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.462+0000 7fc513e00640 1 --2- 172.21.15.28:0/250069524 >> [v2:172.21.15.103:6817/472173438,v1:172.21.15.103:6819/472173438] conn(0x7fc4fc074ac0 0x7fc4fc076ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.462+0000 7fc513e00640 1 -- 172.21.15.28:0/250069524 --> [v2:172.21.15.103:6817/472173438,v1:172.21.15.103:6819/472173438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc50801eec0 con 0x7fc4fc074ac0 2024-04-26T20:56:58.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.462+0000 7fc513e00640 1 -- 172.21.15.28:0/250069524 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc50808b340 con 0x7fc51c06fcf0 2024-04-26T20:56:58.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.462+0000 7f0fc9800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fc406ee50 0x7f0fc41f0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.462+0000 7f0fc9800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc4073940 0x7f0fc415d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.462+0000 7f0fc9800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0fc415afc0 0x7f0fc415dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.462+0000 7f0fc9800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc415b3a0 con 0x7f0fc415afc0 2024-04-26T20:56:58.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.462+0000 7f0fc9800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc4073dd0 con 0x7f0fc406ee50 2024-04-26T20:56:58.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.462+0000 7f0fc9800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc415b520 con 0x7f0fc4073940 2024-04-26T20:56:58.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.463+0000 7f0fc2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fc406ee50 0x7f0fc41f0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.463+0000 7f0fc2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fc406ee50 0x7f0fc41f0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.28:35670/0 (socket says 172.21.15.28:35670) 2024-04-26T20:56:58.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.463+0000 7f0fc2a00640 1 -- 172.21.15.28:0/2101807727 learned_addr learned my addr 172.21.15.28:0/2101807727 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.463+0000 7fc51b400640 1 --2- 172.21.15.28:0/250069524 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc4fc06c920 0x7fc4fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.462+0000 7ff1f8c00640 1 --2- 172.21.15.28:0/3873108524 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff1d406c9c0 0x7ff1d406ee80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0037000 tx=0x7ff1e000a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.462+0000 7fa464c00640 1 --2- 172.21.15.28:0/1109700614 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa44806c8f0 0x7fa44806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.463+0000 7fa464c00640 1 -- 172.21.15.28:0/1109700614 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa45c08c720 con 0x7fa450004800 2024-04-26T20:56:58.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.463+0000 7fa464c00640 1 --2- 172.21.15.28:0/1109700614 >> [v2:172.21.15.103:6816/1745016180,v1:172.21.15.103:6818/1745016180] conn(0x7fa448074af0 0x7fa448076f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.463+0000 7fa464c00640 1 -- 172.21.15.28:0/1109700614 --> [v2:172.21.15.103:6816/1745016180,v1:172.21.15.103:6818/1745016180] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa45c0020b0 con 0x7fa448074af0 2024-04-26T20:56:58.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.463+0000 7fa464c00640 1 -- 172.21.15.28:0/1109700614 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa45c08cad0 con 0x7fa450004800 2024-04-26T20:56:58.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.463+0000 7fc521000640 1 --2- 172.21.15.28:0/250069524 >> [v2:172.21.15.103:6817/472173438,v1:172.21.15.103:6819/472173438] conn(0x7fc4fc074ac0 0x7fc4fc076ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.463+0000 7f0fc2000640 1 --2- 172.21.15.28:0/2101807727 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc4073940 0x7f0fc415d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.464+0000 7ff1f3e00640 1 --2- 172.21.15.28:0/3873108524 >> [v2:172.21.15.103:6800/4239183489,v1:172.21.15.103:6801/4239183489] conn(0x7ff1d4074be0 0x7ff1d4077000 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.464+0000 7fa467e00640 1 --2- 172.21.15.28:0/1109700614 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa44806c8f0 0x7fa44806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.464+0000 7f0fc3400640 1 --2- 172.21.15.28:0/2101807727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0fc415afc0 0x7f0fc415dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.464+0000 7f0fc2a00640 1 -- 172.21.15.28:0/2101807727 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc4073940 msgr2=0x7f0fc415d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.464+0000 7f0fc2a00640 1 --2- 172.21.15.28:0/2101807727 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0fc4073940 0x7f0fc415d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.464+0000 7f0fc2a00640 1 -- 172.21.15.28:0/2101807727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0fc415afc0 msgr2=0x7f0fc415dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.464+0000 7fa46dc00640 1 --2- 172.21.15.28:0/1109700614 >> [v2:172.21.15.103:6816/1745016180,v1:172.21.15.103:6818/1745016180] conn(0x7fa448074af0 0x7fa448076f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.464+0000 7f0fc2a00640 1 --2- 172.21.15.28:0/2101807727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0fc415afc0 0x7f0fc415dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.464+0000 7f0fc2a00640 1 -- 172.21.15.28:0/2101807727 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc41f0570 con 0x7f0fc406ee50 2024-04-26T20:56:58.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.464+0000 7f0fc3400640 1 --2- 172.21.15.28:0/2101807727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0fc415afc0 0x7f0fc415dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:56:58.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.464+0000 7fc51b400640 1 --2- 172.21.15.28:0/250069524 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc4fc06c920 0x7fc4fc06ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc51c148ad0 tx=0x7fc514009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.465+0000 7fa467e00640 1 --2- 172.21.15.28:0/1109700614 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa44806c8f0 0x7fa44806edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa454033000 tx=0x7fa454008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.465+0000 7fc521000640 1 --2- 172.21.15.28:0/250069524 >> [v2:172.21.15.103:6817/472173438,v1:172.21.15.103:6819/472173438] conn(0x7fc4fc074ac0 0x7fc4fc076ee0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.465+0000 7f0fc2a00640 1 --2- 172.21.15.28:0/2101807727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fc406ee50 0x7f0fc41f0030 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400c480 tx=0x7f0fb400c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.465+0000 7fa46dc00640 1 --2- 172.21.15.28:0/1109700614 >> [v2:172.21.15.103:6816/1745016180,v1:172.21.15.103:6818/1745016180] conn(0x7fa448074af0 0x7fa448076f10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.465+0000 7f0fbaa00640 1 -- 172.21.15.28:0/2101807727 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4017070 con 0x7f0fc406ee50 2024-04-26T20:56:58.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.465+0000 7f0fbaa00640 1 -- 172.21.15.28:0/2101807727 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fb4005ce0 con 0x7f0fc406ee50 2024-04-26T20:56:58.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.465+0000 7f0fbaa00640 1 -- 172.21.15.28:0/2101807727 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb40072f0 con 0x7f0fc406ee50 2024-04-26T20:56:58.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.466+0000 7f0fc9800640 1 -- 172.21.15.28:0/2101807727 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc4150370 con 0x7f0fc406ee50 2024-04-26T20:56:58.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.466+0000 7f0fc9800640 1 -- 172.21.15.28:0/2101807727 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc4150930 con 0x7f0fc406ee50 2024-04-26T20:56:58.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.466+0000 7ff1f0c00640 1 -- 172.21.15.28:0/3873108524 <== osd.5 v2:172.21.15.103:6800/4239183489 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1ec012e50 con 0x7ff1d4074be0 2024-04-26T20:56:58.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.467+0000 7fc513e00640 1 -- 172.21.15.28:0/250069524 <== osd.4 v2:172.21.15.103:6817/472173438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc50801eec0 con 0x7fc4fc074ac0 2024-04-26T20:56:58.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.467+0000 7f0fc9800640 1 -- 172.21.15.28:0/2101807727 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0fc406a400 con 0x7f0fc406ee50 2024-04-26T20:56:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.468+0000 7fa464c00640 1 -- 172.21.15.28:0/1109700614 <== osd.6 v2:172.21.15.103:6816/1745016180 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa45c0020b0 con 0x7fa448074af0 2024-04-26T20:56:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.471+0000 7f0fbaa00640 1 -- 172.21.15.28:0/2101807727 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0fb401a070 con 0x7f0fc406ee50 2024-04-26T20:56:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.472+0000 7f0fbaa00640 1 --2- 172.21.15.28:0/2101807727 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0fa406c970 0x7f0fa406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.472+0000 7f0fbaa00640 1 -- 172.21.15.28:0/2101807727 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0fb408b570 con 0x7f0fc406ee50 2024-04-26T20:56:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.473+0000 7f0fbaa00640 1 --2- 172.21.15.28:0/2101807727 >> [v2:172.21.15.103:6808/583391321,v1:172.21.15.103:6809/583391321] conn(0x7f0fa4074b10 0x7f0fa4076f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.473+0000 7f0fbaa00640 1 -- 172.21.15.28:0/2101807727 --> [v2:172.21.15.103:6808/583391321,v1:172.21.15.103:6809/583391321] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0fa40775e0 con 0x7f0fa4074b10 2024-04-26T20:56:58.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.473+0000 7f0fbaa00640 1 -- 172.21.15.28:0/2101807727 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f0fb408b920 con 0x7f0fc406ee50 2024-04-26T20:56:58.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.473+0000 7f0fc2000640 1 --2- 172.21.15.28:0/2101807727 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0fa406c970 0x7f0fa406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.475+0000 7f0fc3400640 1 --2- 172.21.15.28:0/2101807727 >> [v2:172.21.15.103:6808/583391321,v1:172.21.15.103:6809/583391321] conn(0x7f0fa4074b10 0x7f0fa4076f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.475+0000 7f0fc2000640 1 --2- 172.21.15.28:0/2101807727 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0fa406c970 0x7f0fa406ee30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0035000 tx=0x7f0fb0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.475+0000 7fc7b7e00640 1 -- 172.21.15.28:0/370158799 --> [v2:172.21.15.28:6800/3928805442,v1:172.21.15.28:6801/3928805442] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc7b80b0200 con 0x7fc798074bc0 2024-04-26T20:56:58.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.476+0000 7f0fc3400640 1 --2- 172.21.15.28:0/2101807727 >> [v2:172.21.15.103:6808/583391321,v1:172.21.15.103:6809/583391321] conn(0x7f0fa4074b10 0x7f0fa4076f30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.476+0000 7fc7aea00640 1 -- 172.21.15.28:0/370158799 <== osd.1 v2:172.21.15.28:6800/3928805442 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc7b80b0200 con 0x7fc798074bc0 2024-04-26T20:56:58.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.477+0000 7fc7b7e00640 1 -- 172.21.15.28:0/370158799 >> [v2:172.21.15.28:6800/3928805442,v1:172.21.15.28:6801/3928805442] conn(0x7fc798074bc0 msgr2=0x7fc798076fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.477+0000 7fc7b7e00640 1 --2- 172.21.15.28:0/370158799 >> [v2:172.21.15.28:6800/3928805442,v1:172.21.15.28:6801/3928805442] conn(0x7fc798074bc0 0x7fc798076fe0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.477+0000 7fc7b7e00640 1 -- 172.21.15.28:0/370158799 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc79806c940 msgr2=0x7fc79806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.477+0000 7fc7b7e00640 1 --2- 172.21.15.28:0/370158799 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc79806c940 0x7fc79806ee00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8147190 tx=0x7fc7a8008040 comp rx=0 tx=0).stop 2024-04-26T20:56:58.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.477+0000 7fc7b7e00640 1 -- 172.21.15.28:0/370158799 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc79c00d550 msgr2=0x7fc7b8146580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.477+0000 7fc7b7e00640 1 --2- 172.21.15.28:0/370158799 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc79c00d550 0x7fc7b8146580 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000ead0 tx=0x7fc7b000ef90 comp rx=0 tx=0).stop 2024-04-26T20:56:58.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.477+0000 7fc7b7400640 1 -- 172.21.15.28:0/370158799 reap_dead start 2024-04-26T20:56:58.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.477+0000 7fc7b7e00640 1 -- 172.21.15.28:0/370158799 shutdown_connections 2024-04-26T20:56:58.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.477+0000 7fc7b7e00640 1 -- 172.21.15.28:0/370158799 >> 172.21.15.28:0/370158799 conn(0x7fc7b806c350 msgr2=0x7fc7b80b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.479+0000 7fc7b7e00640 1 -- 172.21.15.28:0/370158799 shutdown_connections 2024-04-26T20:56:58.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.479+0000 7fc7b7e00640 1 -- 172.21.15.28:0/370158799 wait complete. 2024-04-26T20:56:58.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.479+0000 7f0fbaa00640 1 -- 172.21.15.28:0/2101807727 <== osd.7 v2:172.21.15.103:6808/583391321 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0fa40775e0 con 0x7f0fa4074b10 2024-04-26T20:56:58.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.484+0000 7f9016600640 1 -- 172.21.15.28:0/1096148521 --> [v2:172.21.15.28:6808/993399689,v1:172.21.15.28:6809/993399689] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f900804ff80 con 0x7f8ff0074be0 2024-04-26T20:56:58.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.486+0000 7f900cc00640 1 -- 172.21.15.28:0/1096148521 <== osd.0 v2:172.21.15.28:6808/993399689 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f900804ff80 con 0x7f8ff0074be0 2024-04-26T20:56:58.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.487+0000 7f9002000640 1 -- 172.21.15.28:0/1096148521 >> [v2:172.21.15.28:6808/993399689,v1:172.21.15.28:6809/993399689] conn(0x7f8ff0074be0 msgr2=0x7f8ff0077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.487+0000 7f9002000640 1 --2- 172.21.15.28:0/1096148521 >> [v2:172.21.15.28:6808/993399689,v1:172.21.15.28:6809/993399689] conn(0x7f8ff0074be0 0x7f8ff0077000 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.487+0000 7f9002000640 1 -- 172.21.15.28:0/1096148521 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8ff006c9c0 msgr2=0x7f8ff006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.487+0000 7f9002000640 1 --2- 172.21.15.28:0/1096148521 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8ff006c9c0 0x7f8ff006ee80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f900804f050 tx=0x7f9004009040 comp rx=0 tx=0).stop 2024-04-26T20:56:58.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.487+0000 7f9002000640 1 -- 172.21.15.28:0/1096148521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9008005e90 msgr2=0x7f90080d5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.487+0000 7f9002000640 1 --2- 172.21.15.28:0/1096148521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9008005e90 0x7f90080d5f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00c3f0 tx=0x7f8ffc00c8b0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.487+0000 7f9015c00640 1 -- 172.21.15.28:0/1096148521 reap_dead start 2024-04-26T20:56:58.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.488+0000 7f9002000640 1 -- 172.21.15.28:0/1096148521 shutdown_connections 2024-04-26T20:56:58.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.488+0000 7f9002000640 1 -- 172.21.15.28:0/1096148521 >> 172.21.15.28:0/1096148521 conn(0x7f9008009fb0 msgr2=0x7f9008047b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.488+0000 7f9002000640 1 -- 172.21.15.28:0/1096148521 shutdown_connections 2024-04-26T20:56:58.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.488+0000 7f9002000640 1 -- 172.21.15.28:0/1096148521 wait complete. 2024-04-26T20:56:58.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.492+0000 7f13ffa00640 1 -- 172.21.15.28:0/1451789063 --> [v2:172.21.15.28:6816/2056655204,v1:172.21.15.28:6817/2056655204] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f13f8069840 con 0x7f13d4074b60 2024-04-26T20:56:58.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.492+0000 7f13f5600640 1 -- 172.21.15.28:0/1451789063 <== osd.2 v2:172.21.15.28:6816/2056655204 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f13f8069840 con 0x7f13d4074b60 2024-04-26T20:56:58.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.493+0000 7f13ffa00640 1 -- 172.21.15.28:0/1451789063 >> [v2:172.21.15.28:6816/2056655204,v1:172.21.15.28:6817/2056655204] conn(0x7f13d4074b60 msgr2=0x7f13d4076f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.493+0000 7f13ffa00640 1 --2- 172.21.15.28:0/1451789063 >> [v2:172.21.15.28:6816/2056655204,v1:172.21.15.28:6817/2056655204] conn(0x7f13d4074b60 0x7f13d4076f80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.497+0000 7f13ffa00640 1 -- 172.21.15.28:0/1451789063 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f13d406c940 msgr2=0x7f13d406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.497+0000 7f13ffa00640 1 --2- 172.21.15.28:0/1451789063 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f13d406c940 0x7f13d406ee00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f13e80070b0 tx=0x7f13e8007040 comp rx=0 tx=0).stop 2024-04-26T20:56:58.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.497+0000 7f13ffa00640 1 -- 172.21.15.28:0/1451789063 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13f815f3b0 msgr2=0x7f13f80713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.497+0000 7f13ffa00640 1 --2- 172.21.15.28:0/1451789063 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f13f815f3b0 0x7f13f80713c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f13f002a700 tx=0x7f13f00046c0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.497+0000 7f13fda00640 1 -- 172.21.15.28:0/1451789063 reap_dead start 2024-04-26T20:56:58.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.497+0000 7f13ffa00640 1 -- 172.21.15.28:0/1451789063 shutdown_connections 2024-04-26T20:56:58.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.497+0000 7f13ffa00640 1 -- 172.21.15.28:0/1451789063 >> 172.21.15.28:0/1451789063 conn(0x7f13f80d4e50 msgr2=0x7f13f80b4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.497+0000 7fd605400640 1 -- 172.21.15.28:0/3780653389 --> [v2:172.21.15.28:6818/2569592417,v1:172.21.15.28:6820/2569592417] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd5d8002dd0 con 0x7fd5e0074ac0 2024-04-26T20:56:58.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.497+0000 7f13ffa00640 1 -- 172.21.15.28:0/1451789063 shutdown_connections 2024-04-26T20:56:58.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.497+0000 7f13ffa00640 1 -- 172.21.15.28:0/1451789063 wait complete. 2024-04-26T20:56:58.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.498+0000 7ff1fb600640 1 -- 172.21.15.28:0/3873108524 --> [v2:172.21.15.103:6800/4239183489,v1:172.21.15.103:6801/4239183489] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff1f40b5340 con 0x7ff1d4074be0 2024-04-26T20:56:58.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.499+0000 7ff1f0c00640 1 -- 172.21.15.28:0/3873108524 <== osd.5 v2:172.21.15.103:6800/4239183489 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff1f40b5340 con 0x7ff1d4074be0 2024-04-26T20:56:58.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.500+0000 7fd5f6a00640 1 -- 172.21.15.28:0/3780653389 <== osd.3 v2:172.21.15.28:6818/2569592417 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd5d8002dd0 con 0x7fd5e0074ac0 2024-04-26T20:56:58.521 INFO:teuthology.orchestra.run.smithi028.stdout:55834574853 2024-04-26T20:56:58.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.501+0000 7fa46e600640 1 -- 172.21.15.28:0/1109700614 --> [v2:172.21.15.103:6816/1745016180,v1:172.21.15.103:6818/1745016180] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa444002dd0 con 0x7fa448074af0 2024-04-26T20:56:58.522 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T20:56:58.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.502+0000 7fd605400640 1 -- 172.21.15.28:0/3780653389 >> [v2:172.21.15.28:6818/2569592417,v1:172.21.15.28:6820/2569592417] conn(0x7fd5e0074ac0 msgr2=0x7fd5e0076ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.502+0000 7fd605400640 1 --2- 172.21.15.28:0/3780653389 >> [v2:172.21.15.28:6818/2569592417,v1:172.21.15.28:6820/2569592417] conn(0x7fd5e0074ac0 0x7fd5e0076ee0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.502+0000 7ff1fb600640 1 -- 172.21.15.28:0/3873108524 >> [v2:172.21.15.103:6800/4239183489,v1:172.21.15.103:6801/4239183489] conn(0x7ff1d4074be0 msgr2=0x7ff1d4077000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.502+0000 7ff1fb600640 1 --2- 172.21.15.28:0/3873108524 >> [v2:172.21.15.103:6800/4239183489,v1:172.21.15.103:6801/4239183489] conn(0x7ff1d4074be0 0x7ff1d4077000 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.502+0000 7fa464c00640 1 -- 172.21.15.28:0/1109700614 <== osd.6 v2:172.21.15.103:6816/1745016180 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa444002dd0 con 0x7fa448074af0 2024-04-26T20:56:58.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.502+0000 7ff1fb600640 1 -- 172.21.15.28:0/3873108524 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff1d406c9c0 msgr2=0x7ff1d406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.502+0000 7fd605400640 1 -- 172.21.15.28:0/3780653389 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd5e006c920 msgr2=0x7fd5e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.502+0000 7fd605400640 1 --2- 172.21.15.28:0/3780653389 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd5e006c920 0x7fd5e006ede0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000bb40 tx=0x7fd5f0008040 comp rx=0 tx=0).stop 2024-04-26T20:56:58.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.502+0000 7fd605400640 1 -- 172.21.15.28:0/3780653389 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6001f4790 msgr2=0x7fd6001f4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.502+0000 7fd605400640 1 --2- 172.21.15.28:0/3780653389 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6001f4790 0x7fd6001f4c10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00e820 tx=0x7fd5ec00ec20 comp rx=0 tx=0).stop 2024-04-26T20:56:58.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.502+0000 7ff1fb600640 1 --2- 172.21.15.28:0/3873108524 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff1d406c9c0 0x7ff1d406ee80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0037000 tx=0x7ff1e000a000 comp rx=0 tx=0).stop 2024-04-26T20:56:58.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.502+0000 7ff1fb600640 1 -- 172.21.15.28:0/3873108524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1f415afc0 msgr2=0x7ff1f40aa590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.502+0000 7ff1fb600640 1 --2- 172.21.15.28:0/3873108524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1f415afc0 0x7ff1f40aa590 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00efc0 tx=0x7ff1ec00c490 comp rx=0 tx=0).stop 2024-04-26T20:56:58.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.503+0000 7fd5ff200640 1 -- 172.21.15.28:0/3780653389 reap_dead start 2024-04-26T20:56:58.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.503+0000 7fd605400640 1 -- 172.21.15.28:0/3780653389 shutdown_connections 2024-04-26T20:56:58.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.503+0000 7fd605400640 1 -- 172.21.15.28:0/3780653389 >> 172.21.15.28:0/3780653389 conn(0x7fd6000d4e50 msgr2=0x7fd60015b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.503+0000 7ff1f9600640 1 -- 172.21.15.28:0/3873108524 reap_dead start 2024-04-26T20:56:58.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.503+0000 7ff1fb600640 1 -- 172.21.15.28:0/3873108524 shutdown_connections 2024-04-26T20:56:58.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.503+0000 7ff1fb600640 1 -- 172.21.15.28:0/3873108524 >> 172.21.15.28:0/3873108524 conn(0x7ff1f40b2a80 msgr2=0x7ff1f415a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.503+0000 7fa46e600640 1 -- 172.21.15.28:0/1109700614 >> [v2:172.21.15.103:6816/1745016180,v1:172.21.15.103:6818/1745016180] conn(0x7fa448074af0 msgr2=0x7fa448076f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.503+0000 7fa46e600640 1 --2- 172.21.15.28:0/1109700614 >> [v2:172.21.15.103:6816/1745016180,v1:172.21.15.103:6818/1745016180] conn(0x7fa448074af0 0x7fa448076f10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.503+0000 7fd605400640 1 -- 172.21.15.28:0/3780653389 shutdown_connections 2024-04-26T20:56:58.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.503+0000 7fd605400640 1 -- 172.21.15.28:0/3780653389 wait complete. 2024-04-26T20:56:58.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.503+0000 7fa46e600640 1 -- 172.21.15.28:0/1109700614 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa44806c8f0 msgr2=0x7fa44806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.503+0000 7fa46e600640 1 --2- 172.21.15.28:0/1109700614 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa44806c8f0 0x7fa44806edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa454033000 tx=0x7fa454008030 comp rx=0 tx=0).stop 2024-04-26T20:56:58.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.504+0000 7fa46e600640 1 -- 172.21.15.28:0/1109700614 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa450004800 msgr2=0x7fa4681ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.504+0000 7fa46e600640 1 --2- 172.21.15.28:0/1109700614 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa450004800 0x7fa4681ff790 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa45c00fae0 tx=0x7fa45c005940 comp rx=0 tx=0).stop 2024-04-26T20:56:58.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.504+0000 7fa46dc00640 1 -- 172.21.15.28:0/1109700614 reap_dead start 2024-04-26T20:56:58.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.504+0000 7fa46e600640 1 -- 172.21.15.28:0/1109700614 shutdown_connections 2024-04-26T20:56:58.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.504+0000 7fa46e600640 1 -- 172.21.15.28:0/1109700614 >> 172.21.15.28:0/1109700614 conn(0x7fa46806c770 msgr2=0x7fa4680b61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.504+0000 7ff1fb600640 1 -- 172.21.15.28:0/3873108524 shutdown_connections 2024-04-26T20:56:58.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.504+0000 7ff1fb600640 1 -- 172.21.15.28:0/3873108524 wait complete. 2024-04-26T20:56:58.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.504+0000 7fc523000640 1 -- 172.21.15.28:0/250069524 --> [v2:172.21.15.103:6817/472173438,v1:172.21.15.103:6819/472173438] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc51c0d0380 con 0x7fc4fc074ac0 2024-04-26T20:56:58.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.504+0000 7fa46e600640 1 -- 172.21.15.28:0/1109700614 shutdown_connections 2024-04-26T20:56:58.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.504+0000 7fa46e600640 1 -- 172.21.15.28:0/1109700614 wait complete. 2024-04-26T20:56:58.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.505+0000 7fc513e00640 1 -- 172.21.15.28:0/250069524 <== osd.4 v2:172.21.15.103:6817/472173438 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc51c0d0380 con 0x7fc4fc074ac0 2024-04-26T20:56:58.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.507+0000 7fc523000640 1 -- 172.21.15.28:0/250069524 >> [v2:172.21.15.103:6817/472173438,v1:172.21.15.103:6819/472173438] conn(0x7fc4fc074ac0 msgr2=0x7fc4fc076ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.507+0000 7fc523000640 1 --2- 172.21.15.28:0/250069524 >> [v2:172.21.15.103:6817/472173438,v1:172.21.15.103:6819/472173438] conn(0x7fc4fc074ac0 0x7fc4fc076ee0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.507+0000 7fc523000640 1 -- 172.21.15.28:0/250069524 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc4fc06c920 msgr2=0x7fc4fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.507+0000 7fc523000640 1 --2- 172.21.15.28:0/250069524 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc4fc06c920 0x7fc4fc06ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc51c148ad0 tx=0x7fc514009040 comp rx=0 tx=0).stop 2024-04-26T20:56:58.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.507+0000 7fc523000640 1 -- 172.21.15.28:0/250069524 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc51c06fcf0 msgr2=0x7fc51c13fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.507+0000 7fc523000640 1 --2- 172.21.15.28:0/250069524 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc51c06fcf0 0x7fc51c13fe30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc50800c3a0 tx=0x7fc50800c860 comp rx=0 tx=0).stop 2024-04-26T20:56:58.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.507+0000 7fc521000640 1 -- 172.21.15.28:0/250069524 reap_dead start 2024-04-26T20:56:58.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.508+0000 7fc523000640 1 -- 172.21.15.28:0/250069524 shutdown_connections 2024-04-26T20:56:58.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.508+0000 7fc523000640 1 -- 172.21.15.28:0/250069524 >> 172.21.15.28:0/250069524 conn(0x7fc51c0b2a80 msgr2=0x7fc51c0cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.508+0000 7fc523000640 1 -- 172.21.15.28:0/250069524 shutdown_connections 2024-04-26T20:56:58.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.508+0000 7fc523000640 1 -- 172.21.15.28:0/250069524 wait complete. 2024-04-26T20:56:58.531 INFO:teuthology.orchestra.run.smithi028.stdout:55834574852 2024-04-26T20:56:58.532 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T20:56:58.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.514+0000 7f0fc9800640 1 -- 172.21.15.28:0/2101807727 --> [v2:172.21.15.103:6808/583391321,v1:172.21.15.103:6809/583391321] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0fc41588b0 con 0x7f0fa4074b10 2024-04-26T20:56:58.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.515+0000 7f0fbaa00640 1 -- 172.21.15.28:0/2101807727 <== osd.7 v2:172.21.15.103:6808/583391321 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0fc41588b0 con 0x7f0fa4074b10 2024-04-26T20:56:58.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.515+0000 7f0fc9800640 1 -- 172.21.15.28:0/2101807727 >> [v2:172.21.15.103:6808/583391321,v1:172.21.15.103:6809/583391321] conn(0x7f0fa4074b10 msgr2=0x7f0fa4076f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.515+0000 7f0fc9800640 1 --2- 172.21.15.28:0/2101807727 >> [v2:172.21.15.103:6808/583391321,v1:172.21.15.103:6809/583391321] conn(0x7f0fa4074b10 0x7f0fa4076f30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.516+0000 7f0fc9800640 1 -- 172.21.15.28:0/2101807727 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0fa406c970 msgr2=0x7f0fa406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.516+0000 7f0fc9800640 1 --2- 172.21.15.28:0/2101807727 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0fa406c970 0x7f0fa406ee30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0035000 tx=0x7f0fb0008030 comp rx=0 tx=0).stop 2024-04-26T20:56:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.516+0000 7f0fc9800640 1 -- 172.21.15.28:0/2101807727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fc406ee50 msgr2=0x7f0fc41f0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.516+0000 7f0fc9800640 1 --2- 172.21.15.28:0/2101807727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0fc406ee50 0x7f0fc41f0030 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400c480 tx=0x7f0fb400c940 comp rx=0 tx=0).stop 2024-04-26T20:56:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.516+0000 7f0fc3400640 1 -- 172.21.15.28:0/2101807727 reap_dead start 2024-04-26T20:56:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.516+0000 7f0fc9800640 1 -- 172.21.15.28:0/2101807727 shutdown_connections 2024-04-26T20:56:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.516+0000 7f0fc9800640 1 -- 172.21.15.28:0/2101807727 >> 172.21.15.28:0/2101807727 conn(0x7f0fc406c770 msgr2=0x7f0fc4156bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.516+0000 7f0fc9800640 1 -- 172.21.15.28:0/2101807727 shutdown_connections 2024-04-26T20:56:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.516+0000 7f0fc9800640 1 -- 172.21.15.28:0/2101807727 wait complete. 2024-04-26T20:56:58.535 INFO:teuthology.orchestra.run.smithi028.stdout:55834574853 2024-04-26T20:56:58.535 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T20:56:58.536 INFO:teuthology.orchestra.run.smithi028.stdout:55834574852 2024-04-26T20:56:58.536 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T20:56:58.537 INFO:teuthology.orchestra.run.smithi028.stdout:55834574853 2024-04-26T20:56:58.537 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T20:56:58.538 INFO:teuthology.orchestra.run.smithi028.stdout:55834574852 2024-04-26T20:56:58.538 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T20:56:58.539 INFO:teuthology.orchestra.run.smithi028.stdout:55834574852 2024-04-26T20:56:58.539 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T20:56:58.540 INFO:teuthology.orchestra.run.smithi028.stdout:55834574853 2024-04-26T20:56:58.540 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T20:56:58.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.822+0000 7fb640000640 1 Processor -- start 2024-04-26T20:56:58.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.822+0000 7fb640000640 1 -- start start 2024-04-26T20:56:58.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.825+0000 7fb640000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb6380b43c0 0x7fb6380d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.825+0000 7fb640000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6380b6b10 0x7fb6380b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.826+0000 7fb640000640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6380b47a0 con 0x7fb6381607e0 2024-04-26T20:56:58.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.826+0000 7fb640000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb6380b74a0 con 0x7fb6380b43c0 2024-04-26T20:56:58.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.826+0000 7fb640000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb638160c90 con 0x7fb6380b6b10 2024-04-26T20:56:58.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.826+0000 7f1f0b600640 1 Processor -- start 2024-04-26T20:56:58.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.827+0000 7f1f0b600640 1 -- start start 2024-04-26T20:56:58.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.826+0000 7fb63d600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb6380b43c0 0x7fb6380d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.827+0000 7fb63d600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb6380b43c0 0x7fb6380d00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42208/0 (socket says 172.21.15.28:42208) 2024-04-26T20:56:58.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.827+0000 7fb63d600640 1 -- 172.21.15.28:0/2505698047 learned_addr learned my addr 172.21.15.28:0/2505698047 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.827+0000 7fb63e000640 1 --2- 172.21.15.28:0/2505698047 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6380b6b10 0x7fb6380b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.827+0000 7fb63d600640 1 -- 172.21.15.28:0/2505698047 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6380b6b10 msgr2=0x7fb6380b6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.827+0000 7fb63d600640 1 --2- 172.21.15.28:0/2505698047 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6380b6b10 0x7fb6380b6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.827+0000 7fb63d600640 1 -- 172.21.15.28:0/2505698047 >> v1:172.21.15.103:6789/0 conn(0x7fb6381607e0 legacy=0x7fb638163cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.827+0000 7fb63d600640 1 -- 172.21.15.28:0/2505698047 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6380d0740 con 0x7fb6380b43c0 2024-04-26T20:56:58.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.828+0000 7f1f0b600640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f1f040b43c0 0x7f1f040d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.828+0000 7f1f0b600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f041607e0 0x7f1f04163cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.829+0000 7f1f0b600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f040b47a0 con 0x7f1f040b6b10 2024-04-26T20:56:58.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.829+0000 7f1f0b600640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f1f040d0740 con 0x7f1f040b43c0 2024-04-26T20:56:58.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.829+0000 7f1f0b600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f1f040d0a20 con 0x7f1f041607e0 2024-04-26T20:56:58.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.829+0000 7fb63d600640 1 --2- 172.21.15.28:0/2505698047 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6380b43c0 0x7fb6380d00f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb624009870 tx=0x7fb62402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d917bda9e70554b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.829+0000 7f1f08c00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f1f040b43c0 0x7f1f040d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.829+0000 7f1f09600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1f040b6b10 0x7f1f040d3880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57796/0 (socket says 172.21.15.28:57796) 2024-04-26T20:56:58.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.829+0000 7f1f09600640 1 -- 172.21.15.28:0/2155817807 learned_addr learned my addr 172.21.15.28:0/2155817807 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:58.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.830+0000 7fb637e00640 1 -- 172.21.15.28:0/2505698047 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624037070 con 0x7fb6380b43c0 2024-04-26T20:56:58.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.830+0000 7f1f03e00640 1 --2- 172.21.15.28:0/2155817807 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f041607e0 0x7f1f04163cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.830+0000 7fb637e00640 1 -- 172.21.15.28:0/2505698047 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb624032440 con 0x7fb6380b43c0 2024-04-26T20:56:58.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.830+0000 7fb637e00640 1 -- 172.21.15.28:0/2505698047 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb624032810 con 0x7fb6380b43c0 2024-04-26T20:56:58.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.830+0000 7f1f03400640 1 -- 172.21.15.28:0/2155817807 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 348621388 0 0) 0x7f1f040b47a0 con 0x7f1f040b6b10 2024-04-26T20:56:58.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.830+0000 7f1f03400640 1 -- 172.21.15.28:0/2155817807 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ee8003520 con 0x7f1f040b6b10 2024-04-26T20:56:58.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.830+0000 7f1f08c00640 1 -- 172.21.15.28:0/2155817807 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f041607e0 msgr2=0x7f1f04163cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.830+0000 7f1f08c00640 1 --2- 172.21.15.28:0/2155817807 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f041607e0 0x7f1f04163cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.830+0000 7f1f08c00640 1 -- 172.21.15.28:0/2155817807 >> v1:172.21.15.28:6789/0 conn(0x7f1f040b6b10 legacy=0x7f1f040d3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.830+0000 7f1f08c00640 1 -- 172.21.15.28:0/2155817807 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f041634b0 con 0x7f1f040b43c0 2024-04-26T20:56:58.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.831+0000 7fc5ab200640 1 Processor -- start 2024-04-26T20:56:58.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.832+0000 7fd923c00640 1 Processor -- start 2024-04-26T20:56:58.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.832+0000 7fc5ab200640 1 -- start start 2024-04-26T20:56:58.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.832+0000 7f1f08c00640 1 --2- 172.21.15.28:0/2155817807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f040b43c0 0x7f1f040d00f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009d90 tx=0x7f1ef402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a05db0d9cc971d0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.832+0000 7fb640000640 1 -- 172.21.15.28:0/2505698047 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6380b43c0 msgr2=0x7fb6380d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.832+0000 7fb640000640 1 --2- 172.21.15.28:0/2505698047 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6380b43c0 0x7fb6380d00f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb624009870 tx=0x7fb62402a180 comp rx=0 tx=0).stop 2024-04-26T20:56:58.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.832+0000 7fb640000640 1 -- 172.21.15.28:0/2505698047 shutdown_connections 2024-04-26T20:56:58.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.832+0000 7fb640000640 1 --2- 172.21.15.28:0/2505698047 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6380b6b10 0x7fb6380b6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.832+0000 7fb640000640 1 --2- 172.21.15.28:0/2505698047 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6380b43c0 0x7fb6380d00f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.832+0000 7fb640000640 1 -- 172.21.15.28:0/2505698047 >> 172.21.15.28:0/2505698047 conn(0x7fb6380d1b10 msgr2=0x7fb6380d1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.833+0000 7f1f03400640 1 -- 172.21.15.28:0/2155817807 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4031600 con 0x7f1f040b43c0 2024-04-26T20:56:58.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.833+0000 7f1f03400640 1 -- 172.21.15.28:0/2155817807 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ef40317a0 con 0x7f1f040b43c0 2024-04-26T20:56:58.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.833+0000 7f1f03400640 1 -- 172.21.15.28:0/2155817807 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4031b50 con 0x7f1f040b43c0 2024-04-26T20:56:58.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.833+0000 7fd923c00640 1 -- start start 2024-04-26T20:56:58.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.833+0000 7fb640000640 1 -- 172.21.15.28:0/2505698047 shutdown_connections 2024-04-26T20:56:58.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.834+0000 7f1f0b600640 1 -- 172.21.15.28:0/2155817807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f040b43c0 msgr2=0x7f1f040d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.834+0000 7f1f0b600640 1 --2- 172.21.15.28:0/2155817807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f040b43c0 0x7f1f040d00f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009d90 tx=0x7f1ef402a5d0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.833+0000 7fb640000640 1 -- 172.21.15.28:0/2505698047 wait complete. 2024-04-26T20:56:58.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.834+0000 7f1f0b600640 1 -- 172.21.15.28:0/2155817807 shutdown_connections 2024-04-26T20:56:58.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.834+0000 7f1f0b600640 1 --2- 172.21.15.28:0/2155817807 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f041607e0 0x7f1f04163cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.834+0000 7f1f0b600640 1 --2- 172.21.15.28:0/2155817807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f040b43c0 0x7f1f040d00f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.834+0000 7f1f0b600640 1 -- 172.21.15.28:0/2155817807 >> 172.21.15.28:0/2155817807 conn(0x7f1f040d1b10 msgr2=0x7f1f040d1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.834+0000 7f1f0b600640 1 -- 172.21.15.28:0/2155817807 shutdown_connections 2024-04-26T20:56:58.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.834+0000 7f1f0b600640 1 -- 172.21.15.28:0/2155817807 wait complete. 2024-04-26T20:56:58.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.834+0000 7efdc9800640 1 Processor -- start 2024-04-26T20:56:58.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.835+0000 7fb640000640 1 Processor -- start 2024-04-26T20:56:58.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.835+0000 7fb640000640 1 -- start start 2024-04-26T20:56:58.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.835+0000 7efdc9800640 1 -- start start 2024-04-26T20:56:58.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.836+0000 7fadf6a00640 1 Processor -- start 2024-04-26T20:56:58.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.835+0000 7fc5ab200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc5ac0b7020 0x7fc5ac159a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.835+0000 7fc5ab200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5ac0d0370 con 0x7fc5ac0cfb20 2024-04-26T20:56:58.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.835+0000 7fc5ab200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5ac0d0650 con 0x7fc5ac163e90 2024-04-26T20:56:58.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.835+0000 7fc5ab200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fc5ac0d0930 con 0x7fc5ac0b7020 2024-04-26T20:56:58.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.836+0000 7fadf6a00640 1 -- start start 2024-04-26T20:56:58.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.837+0000 7f1f0b600640 1 Processor -- start 2024-04-26T20:56:58.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.837+0000 7f1f0b600640 1 -- start start 2024-04-26T20:56:58.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.837+0000 7fc5a9400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc5ac0cfb20 0x7fc5ac0b6910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57808/0 (socket says 172.21.15.28:57808) 2024-04-26T20:56:58.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.837+0000 7fc5a9400640 1 -- 172.21.15.28:0/1809417278 learned_addr learned my addr 172.21.15.28:0/1809417278 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:58.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.837+0000 7fb640000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6380b43c0 0x7fb63815eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.837+0000 7fb640000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6380b6b10 0x7fb6380c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.837+0000 7fb640000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6381607e0 0x7fb6380b8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.837+0000 7fb640000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb638163790 con 0x7fb6380b6b10 2024-04-26T20:56:58.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.837+0000 7fb640000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb6381634b0 con 0x7fb6380b43c0 2024-04-26T20:56:58.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.837+0000 7fb640000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb6380d3880 con 0x7fb6381607e0 2024-04-26T20:56:58.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.837+0000 7fb63cc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6380b6b10 0x7fb6380c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.837+0000 7fb63cc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6380b6b10 0x7fb6380c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42232/0 (socket says 172.21.15.28:42232) 2024-04-26T20:56:58.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.837+0000 7fb63cc00640 1 -- 172.21.15.28:0/3325219525 learned_addr learned my addr 172.21.15.28:0/3325219525 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.838+0000 7fb63d600640 1 --2- 172.21.15.28:0/3325219525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6380b43c0 0x7fb63815eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.838+0000 7fb63e000640 1 --2- 172.21.15.28:0/3325219525 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6381607e0 0x7fb6380b8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.837+0000 7fc5aa800640 1 --2- 172.21.15.28:0/1809417278 >> v2:172.21.15.28:3300/0 conn(0x7fc5ac0b7020 0x7fc5ac159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.838+0000 7fc5a8a00640 1 -- 172.21.15.28:0/1809417278 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 742679354 0 0) 0x7fc5ac0d0370 con 0x7fc5ac0cfb20 2024-04-26T20:56:58.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.838+0000 7fa52a800640 1 Processor -- start 2024-04-26T20:56:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.838+0000 7fc5a8a00640 1 -- 172.21.15.28:0/1809417278 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc590003520 con 0x7fc5ac0cfb20 2024-04-26T20:56:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.838+0000 7f1f0b600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f040b43c0 0x7f1f040cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.838+0000 7f1f0b600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f040b6b10 0x7f1f040c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.838+0000 7f1f0b600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f041607e0 0x7f1f04153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.838+0000 7f1f0b600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1f04160c60 con 0x7f1f040b43c0 2024-04-26T20:56:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.838+0000 7f1f0b600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1f041641f0 con 0x7f1f041607e0 2024-04-26T20:56:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.838+0000 7f1f0b600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f1f040d3f90 con 0x7f1f040b6b10 2024-04-26T20:56:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.838+0000 7fa52a800640 1 -- start start 2024-04-26T20:56:58.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.838+0000 7fc5a8a00640 1 -- 172.21.15.28:0/1809417278 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2065647283 0 0) 0x7fc590003520 con 0x7fc5ac0cfb20 2024-04-26T20:56:58.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.838+0000 7fc5a8a00640 1 -- 172.21.15.28:0/1809417278 >> v2:172.21.15.28:3300/0 conn(0x7fc5ac0b7020 msgr2=0x7fc5ac159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.838+0000 7fc5a8a00640 1 --2- 172.21.15.28:0/1809417278 >> v2:172.21.15.28:3300/0 conn(0x7fc5ac0b7020 0x7fc5ac159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.838+0000 7fc5a8a00640 1 -- 172.21.15.28:0/1809417278 >> v1:172.21.15.103:6789/0 conn(0x7fc5ac163e90 legacy=0x7fc5ac164270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.838+0000 7fc5a8a00640 1 -- 172.21.15.28:0/1809417278 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac0d0c10 con 0x7fc5ac0cfb20 2024-04-26T20:56:58.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7fb63cc00640 1 -- 172.21.15.28:0/3325219525 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6381607e0 msgr2=0x7fb6380b8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7fadf6a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadf01612d0 0x7fadf00b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7fadf6a00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fadf00b61c0 0x7fadf0165330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7fadf6a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadf00b4740 con 0x7fadf0163c00 2024-04-26T20:56:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7fadf6a00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fadf00b4a20 con 0x7fadf00b61c0 2024-04-26T20:56:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7f1f08c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f040b43c0 0x7f1f040cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7f1f03e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f040b6b10 0x7f1f040c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7fadf6a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fadf00b4d00 con 0x7fadf01612d0 2024-04-26T20:56:58.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7f1f09600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f041607e0 0x7f1f04153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7fc5a8a00640 1 -- 172.21.15.28:0/1809417278 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fc5a40019f0 con 0x7fc5ac0cfb20 2024-04-26T20:56:58.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7fc5a8a00640 1 -- 172.21.15.28:0/1809417278 >> v1:172.21.15.28:6789/0 conn(0x7fc5ac0cfb20 legacy=0x7fc5ac0b6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7fc5a8a00640 1 --2- 172.21.15.28:0/1809417278 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5900039b0 0x7fc590003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7fc5a8a00640 1 --2- 172.21.15.28:0/1809417278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc590004800 0x7fc590004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7fc5a8a00640 1 --2- 172.21.15.28:0/1809417278 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5900051c0 0x7fc59000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7fc5a8a00640 1 -- 172.21.15.28:0/1809417278 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac0d0930 con 0x7fc5900039b0 2024-04-26T20:56:58.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7fc5a8a00640 1 -- 172.21.15.28:0/1809417278 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc59000c010 con 0x7fc590004800 2024-04-26T20:56:58.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7fc5a8a00640 1 -- 172.21.15.28:0/1809417278 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc59000c210 con 0x7fc5900051c0 2024-04-26T20:56:58.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7f1f09600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f041607e0 0x7f1f04153030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.28:35690/0 (socket says 172.21.15.28:35690) 2024-04-26T20:56:58.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7f1f08c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f040b43c0 0x7f1f040cba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42248/0 (socket says 172.21.15.28:42248) 2024-04-26T20:56:58.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7f1f09600640 1 -- 172.21.15.28:0/2401730747 learned_addr learned my addr 172.21.15.28:0/2401730747 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7fb63cc00640 1 --2- 172.21.15.28:0/3325219525 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6381607e0 0x7fb6380b8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7fb63cc00640 1 -- 172.21.15.28:0/3325219525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6380b43c0 msgr2=0x7fb63815eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7fc5aa800640 1 --2- 172.21.15.28:0/1809417278 >> v2:172.21.15.28:3300/0 conn(0x7fc5ac0b7020 0x7fc5ac159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:56:58.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7fb63cc00640 1 --2- 172.21.15.28:0/3325219525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6380b43c0 0x7fb63815eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.840+0000 7fb63cc00640 1 -- 172.21.15.28:0/3325219525 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6380c9190 con 0x7fb6380b6b10 2024-04-26T20:56:58.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.840+0000 7fadefe00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadf01612d0 0x7fadf00b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.840+0000 7fadefe00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadf01612d0 0x7fadf00b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.28:59324/0 (socket says 172.21.15.28:59324) 2024-04-26T20:56:58.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.840+0000 7fadefe00640 1 -- 172.21.15.28:0/3784799291 learned_addr learned my addr 172.21.15.28:0/3784799291 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.840+0000 7fb63d600640 1 --2- 172.21.15.28:0/3325219525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6380b43c0 0x7fb63815eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:56:58.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.840+0000 7fb63e000640 1 --2- 172.21.15.28:0/3325219525 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6381607e0 0x7fb6380b8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:56:58.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.840+0000 7f1f08c00640 1 -- 172.21.15.28:0/2401730747 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f040b6b10 msgr2=0x7f1f040c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.840+0000 7f1f08c00640 1 --2- 172.21.15.28:0/2401730747 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f040b6b10 0x7f1f040c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.840+0000 7f1f08c00640 1 -- 172.21.15.28:0/2401730747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f041607e0 msgr2=0x7f1f04153030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.840+0000 7f1f08c00640 1 --2- 172.21.15.28:0/2401730747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f041607e0 0x7f1f04153030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.840+0000 7f1f08c00640 1 -- 172.21.15.28:0/2401730747 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f040ca9f0 con 0x7f1f040b43c0 2024-04-26T20:56:58.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.839+0000 7fc5a9e00640 1 --2- 172.21.15.28:0/1809417278 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5900039b0 0x7fc590003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.840+0000 7fb63cc00640 1 --2- 172.21.15.28:0/3325219525 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6380b6b10 0x7fb6380c8740 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00b3f0 tx=0x7fb62c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.840+0000 7fadf4a00640 1 --2- 172.21.15.28:0/3784799291 >> v2:172.21.15.103:3300/0 conn(0x7fadf00b61c0 0x7fadf0165330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7efdc9800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efdc415f070 0x7efdc40d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7efdc9800640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7efdc40d1f10 0x7efdc40b7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7efdc9800640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdc40d4500 con 0x7efdc40ba710 2024-04-26T20:56:58.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7efdc9800640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7efdc415f450 con 0x7efdc40d1f10 2024-04-26T20:56:58.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7efdc9800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7efdc40d4a80 con 0x7efdc415f070 2024-04-26T20:56:58.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7f1f08c00640 1 --2- 172.21.15.28:0/2401730747 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f040b43c0 0x7f1f040cba90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1f0408d820 tx=0x7f1ef4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7fa52a800640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa52415f200 0x7fa5240bacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7fa52a800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5240cfe70 0x7fa5240d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7fa52a800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5240b8240 con 0x7fa52415f5e0 2024-04-26T20:56:58.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7fa52a800640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fa5240d3810 con 0x7fa52415f200 2024-04-26T20:56:58.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7fa52a800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa524163760 con 0x7fa5240cfe70 2024-04-26T20:56:58.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7fb635600640 1 -- 172.21.15.28:0/3325219525 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c015020 con 0x7fb6380b6b10 2024-04-26T20:56:58.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.835+0000 7fd923c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd91c0d00f0 0x7fd91c1607e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.835+0000 7fd921200640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd91c0b43c0 0x7fd91c163cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57798/0 (socket says 172.21.15.28:57798) 2024-04-26T20:56:58.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7fb635600640 1 -- 172.21.15.28:0/3325219525 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb62c011830 con 0x7fb6380b6b10 2024-04-26T20:56:58.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7fb635600640 1 -- 172.21.15.28:0/3325219525 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb62c011c20 con 0x7fb6380b6b10 2024-04-26T20:56:58.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7fadefe00640 1 -- 172.21.15.28:0/3784799291 >> v2:172.21.15.103:3300/0 conn(0x7fadf00b61c0 msgr2=0x7fadf0165330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7fb640000640 1 -- 172.21.15.28:0/3325219525 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6380c9470 con 0x7fb6380b6b10 2024-04-26T20:56:58.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7efdc3e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efdc415f070 0x7efdc40d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7efdc3400640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7efdc40ba710 0x7efdc40d1800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.28:36100/0 (socket says 172.21.15.28:36100) 2024-04-26T20:56:58.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7efdc3400640 1 -- 172.21.15.28:0/185421119 learned_addr learned my addr 172.21.15.28:0/185421119 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:58.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7fadefe00640 1 --2- 172.21.15.28:0/3784799291 >> v2:172.21.15.103:3300/0 conn(0x7fadf00b61c0 0x7fadf0165330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7fadefe00640 1 -- 172.21.15.28:0/3784799291 >> v1:172.21.15.28:6789/0 conn(0x7fadf0163c00 legacy=0x7fadf00b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7fadefe00640 1 -- 172.21.15.28:0/3784799291 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf0164e10 con 0x7fadf01612d0 2024-04-26T20:56:58.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.841+0000 7fadf4a00640 1 --2- 172.21.15.28:0/3784799291 >> v2:172.21.15.103:3300/0 conn(0x7fadf00b61c0 0x7fadf0165330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:56:58.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.842+0000 7fd921200640 1 -- 172.21.15.28:0/2361792812 learned_addr learned my addr 172.21.15.28:0/2361792812 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:58.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.842+0000 7fd923c00640 1 --2- 172.21.15.28:0/2361792812 >> v2:172.21.15.28:3300/0 conn(0x7fd91c0b6b10 0x7fd91c0b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.842+0000 7fb640000640 1 -- 172.21.15.28:0/3325219525 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6380c9a30 con 0x7fb6380b6b10 2024-04-26T20:56:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.842+0000 7fd91be00640 1 --2- 172.21.15.28:0/2361792812 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd91c0d00f0 0x7fd91c1607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.842+0000 7efdc2a00640 1 -- 172.21.15.28:0/185421119 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 249199956 0 0) 0x7efdc40d4500 con 0x7efdc40ba710 2024-04-26T20:56:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.842+0000 7fa523400640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa52415f200 0x7fa5240bacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.842+0000 7efdc2a00640 1 -- 172.21.15.28:0/185421119 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efda8003520 con 0x7efdc40ba710 2024-04-26T20:56:58.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.843+0000 7efdc3e00640 1 -- 172.21.15.28:0/185421119 >> v2:172.21.15.103:3300/0 conn(0x7efdc40d1f10 msgr2=0x7efdc40b7f20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T20:56:58.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.843+0000 7efdc3e00640 1 --2- 172.21.15.28:0/185421119 >> v2:172.21.15.103:3300/0 conn(0x7efdc40d1f10 0x7efdc40b7f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.843+0000 7efdc3e00640 1 -- 172.21.15.28:0/185421119 >> v1:172.21.15.103:6789/0 conn(0x7efdc40ba710 legacy=0x7efdc40d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.843+0000 7efdc3e00640 1 -- 172.21.15.28:0/185421119 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc40d01c0 con 0x7efdc415f070 2024-04-26T20:56:58.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.843+0000 7fa523e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5240cfe70 0x7fa5240d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.843+0000 7fa523e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5240cfe70 0x7fa5240d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.28:59364/0 (socket says 172.21.15.28:59364) 2024-04-26T20:56:58.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.843+0000 7fa523e00640 1 -- 172.21.15.28:0/84473410 learned_addr learned my addr 172.21.15.28:0/84473410 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.843+0000 7fd923c00640 1 -- 172.21.15.28:0/2361792812 --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd91c0b47a0 con 0x7fd91c0b43c0 2024-04-26T20:56:58.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.840+0000 7fc5a9400640 1 --2- 172.21.15.28:0/1809417278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc590004800 0x7fc590004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.842+0000 7fc5a9400640 1 -- 172.21.15.28:0/1809417278 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5900051c0 msgr2=0x7fc59000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.842+0000 7fc5a9400640 1 --2- 172.21.15.28:0/1809417278 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5900051c0 0x7fc59000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.842+0000 7fc5a9400640 1 -- 172.21.15.28:0/1809417278 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5900039b0 msgr2=0x7fc590003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.842+0000 7fc5a9400640 1 --2- 172.21.15.28:0/1809417278 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5900039b0 0x7fc590003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.842+0000 7fc5a9400640 1 -- 172.21.15.28:0/1809417278 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc590003520 con 0x7fc590004800 2024-04-26T20:56:58.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.842+0000 7fc5aa800640 1 -- 172.21.15.28:0/1809417278 reap_dead start 2024-04-26T20:56:58.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.843+0000 7fadefe00640 1 --2- 172.21.15.28:0/3784799291 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadf01612d0 0x7fadf00b3df0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fade8009e50 tx=0x7fade802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f271e05e5abf01a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.843+0000 7f1f00c00640 1 -- 172.21.15.28:0/2401730747 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4040070 con 0x7f1f040b43c0 2024-04-26T20:56:58.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.843+0000 7fb640000640 1 -- 172.21.15.28:0/3325219525 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb63806a400 con 0x7fb6380b6b10 2024-04-26T20:56:58.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.843+0000 7f1f00c00640 1 -- 172.21.15.28:0/2401730747 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ef4004260 con 0x7f1f040b43c0 2024-04-26T20:56:58.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.843+0000 7f1f00c00640 1 -- 172.21.15.28:0/2401730747 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4004650 con 0x7f1f040b43c0 2024-04-26T20:56:58.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.843+0000 7fd921c00640 1 --2- 172.21.15.28:0/2361792812 >> v2:172.21.15.28:3300/0 conn(0x7fd91c0b6b10 0x7fd91c0b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.844+0000 7fc5a9400640 1 --2- 172.21.15.28:0/1809417278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc590004800 0x7fc590004c80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400a920 tx=0x7fc590005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22a296bab621a83c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.844+0000 7fadeea00640 1 -- 172.21.15.28:0/3784799291 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade8036070 con 0x7fadf01612d0 2024-04-26T20:56:58.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.844+0000 7fadeea00640 1 -- 172.21.15.28:0/3784799291 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fade8031600 con 0x7fadf01612d0 2024-04-26T20:56:58.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.844+0000 7fadeea00640 1 -- 172.21.15.28:0/3784799291 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade80319d0 con 0x7fadf01612d0 2024-04-26T20:56:58.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.844+0000 7f1f0b600640 1 -- 172.21.15.28:0/2401730747 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f040cacd0 con 0x7f1f040b43c0 2024-04-26T20:56:58.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.844+0000 7f1f0b600640 1 -- 172.21.15.28:0/2401730747 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f040cb290 con 0x7f1f040b43c0 2024-04-26T20:56:58.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.844+0000 7fa523400640 1 -- 172.21.15.28:0/84473410 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5240cfe70 msgr2=0x7fa5240d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.844+0000 7fa523400640 1 --2- 172.21.15.28:0/84473410 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5240cfe70 0x7fa5240d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.844+0000 7fa523400640 1 -- 172.21.15.28:0/84473410 >> v1:172.21.15.28:6789/0 conn(0x7fa52415f5e0 legacy=0x7fa524162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.844+0000 7fa523400640 1 -- 172.21.15.28:0/84473410 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa524163a40 con 0x7fa52415f200 2024-04-26T20:56:58.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.845+0000 7fadf6a00640 1 -- 172.21.15.28:0/3784799291 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadf01612d0 msgr2=0x7fadf00b3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.845+0000 7fadf6a00640 1 --2- 172.21.15.28:0/3784799291 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadf01612d0 0x7fadf00b3df0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fade8009e50 tx=0x7fade802a710 comp rx=0 tx=0).stop 2024-04-26T20:56:58.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.845+0000 7fadf6a00640 1 -- 172.21.15.28:0/3784799291 shutdown_connections 2024-04-26T20:56:58.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.845+0000 7fadf6a00640 1 --2- 172.21.15.28:0/3784799291 >> v2:172.21.15.103:3300/0 conn(0x7fadf00b61c0 0x7fadf0165330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.845+0000 7fadf6a00640 1 --2- 172.21.15.28:0/3784799291 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadf01612d0 0x7fadf00b3df0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.845+0000 7fadf6a00640 1 -- 172.21.15.28:0/3784799291 >> 172.21.15.28:0/3784799291 conn(0x7fadf006e220 msgr2=0x7fadf006e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.844+0000 7fc5a8a00640 1 -- 172.21.15.28:0/1809417278 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a400a4c0 con 0x7fc590004800 2024-04-26T20:56:58.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.844+0000 7fc5a8a00640 1 -- 172.21.15.28:0/1809417278 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a402f070 con 0x7fc590004800 2024-04-26T20:56:58.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.844+0000 7fc5a8a00640 1 -- 172.21.15.28:0/1809417278 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a402bcf0 con 0x7fc590004800 2024-04-26T20:56:58.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.845+0000 7efdc3e00640 1 --2- 172.21.15.28:0/185421119 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efdc415f070 0x7efdc40d36b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7efdc415f630 tx=0x7efdb402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0025942f45d860a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.846+0000 7efdc2a00640 1 -- 172.21.15.28:0/185421119 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb4037070 con 0x7efdc415f070 2024-04-26T20:56:58.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.846+0000 7efdc2a00640 1 -- 172.21.15.28:0/185421119 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb4032440 con 0x7efdc415f070 2024-04-26T20:56:58.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.846+0000 7efdc2a00640 1 -- 172.21.15.28:0/185421119 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb402ae60 con 0x7efdc415f070 2024-04-26T20:56:58.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.847+0000 7efdc9800640 1 -- 172.21.15.28:0/185421119 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efdc415f070 msgr2=0x7efdc40d36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.847+0000 7efdc9800640 1 --2- 172.21.15.28:0/185421119 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efdc415f070 0x7efdc40d36b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7efdc415f630 tx=0x7efdb402a110 comp rx=0 tx=0).stop 2024-04-26T20:56:58.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.847+0000 7f1f0b600640 1 -- 172.21.15.28:0/2401730747 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f0406a400 con 0x7f1f040b43c0 2024-04-26T20:56:58.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.847+0000 7fa523400640 1 --2- 172.21.15.28:0/84473410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa52415f200 0x7fa5240bacc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa5100097a0 tx=0x7fa51002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd4b73b19406faf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.846+0000 7fd923c00640 1 -- 172.21.15.28:0/2361792812 --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd91c0b74c0 con 0x7fd91c0b6b10 2024-04-26T20:56:58.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.846+0000 7fd923c00640 1 -- 172.21.15.28:0/2361792812 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fd91c160d20 con 0x7fd91c0d00f0 2024-04-26T20:56:58.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.847+0000 7fadf6a00640 1 -- 172.21.15.28:0/3784799291 shutdown_connections 2024-04-26T20:56:58.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.847+0000 7fadf6a00640 1 -- 172.21.15.28:0/3784799291 wait complete. 2024-04-26T20:56:58.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.847+0000 7fb635600640 1 -- 172.21.15.28:0/3325219525 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb62c018020 con 0x7fb6380b6b10 2024-04-26T20:56:58.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.847+0000 7fa522000640 1 -- 172.21.15.28:0/84473410 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510037070 con 0x7fa52415f200 2024-04-26T20:56:58.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.848+0000 7fa522000640 1 -- 172.21.15.28:0/84473410 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa510032440 con 0x7fa52415f200 2024-04-26T20:56:58.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.848+0000 7fa522000640 1 -- 172.21.15.28:0/84473410 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51002ae60 con 0x7fa52415f200 2024-04-26T20:56:58.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.849+0000 7fadf6a00640 1 Processor -- start 2024-04-26T20:56:58.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.849+0000 7efdc9800640 1 -- 172.21.15.28:0/185421119 shutdown_connections 2024-04-26T20:56:58.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.849+0000 7fadf6a00640 1 -- start start 2024-04-26T20:56:58.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.849+0000 7efdc9800640 1 --2- 172.21.15.28:0/185421119 >> v2:172.21.15.103:3300/0 conn(0x7efdc40d1f10 0x7efdc40b7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.849+0000 7efdc9800640 1 --2- 172.21.15.28:0/185421119 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efdc415f070 0x7efdc40d36b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.849+0000 7efdc9800640 1 -- 172.21.15.28:0/185421119 >> 172.21.15.28:0/185421119 conn(0x7efdc406c980 msgr2=0x7efdc4072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.849+0000 7fa52a800640 1 -- 172.21.15.28:0/84473410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa52415f200 msgr2=0x7fa5240bacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.849+0000 7fa52a800640 1 --2- 172.21.15.28:0/84473410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa52415f200 0x7fa5240bacc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa5100097a0 tx=0x7fa51002a110 comp rx=0 tx=0).stop 2024-04-26T20:56:58.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.847+0000 7fd91be00640 1 -- 172.21.15.28:0/2361792812 >> v2:172.21.15.28:3300/0 conn(0x7fd91c0b6b10 msgr2=0x7fd91c0b6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.847+0000 7fd91be00640 1 --2- 172.21.15.28:0/2361792812 >> v2:172.21.15.28:3300/0 conn(0x7fd91c0b6b10 0x7fd91c0b6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.847+0000 7fd91be00640 1 -- 172.21.15.28:0/2361792812 >> v1:172.21.15.28:6789/0 conn(0x7fd91c0b43c0 legacy=0x7fd91c163cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.847+0000 7fd91be00640 1 -- 172.21.15.28:0/2361792812 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91c0d0740 con 0x7fd91c0d00f0 2024-04-26T20:56:58.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.845+0000 7fc5ab200640 1 -- 172.21.15.28:0/1809417278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc590004800 msgr2=0x7fc590004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.846+0000 7fc5ab200640 1 --2- 172.21.15.28:0/1809417278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc590004800 0x7fc590004c80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400a920 tx=0x7fc590005830 comp rx=0 tx=0).stop 2024-04-26T20:56:58.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.846+0000 7fc5ab200640 1 -- 172.21.15.28:0/1809417278 shutdown_connections 2024-04-26T20:56:58.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.846+0000 7fc5ab200640 1 --2- 172.21.15.28:0/1809417278 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc590004800 0x7fc590004c80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.846+0000 7fc5ab200640 1 -- 172.21.15.28:0/1809417278 >> 172.21.15.28:0/1809417278 conn(0x7fc5ac06c980 msgr2=0x7fc5ac0d4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.846+0000 7fc5ab200640 1 -- 172.21.15.28:0/1809417278 shutdown_connections 2024-04-26T20:56:58.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.846+0000 7fc5ab200640 1 -- 172.21.15.28:0/1809417278 wait complete. 2024-04-26T20:56:58.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.849+0000 7fc5ab200640 1 Processor -- start 2024-04-26T20:56:58.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.849+0000 7fc5ab200640 1 -- start start 2024-04-26T20:56:58.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.850+0000 7efdc9800640 1 -- 172.21.15.28:0/185421119 shutdown_connections 2024-04-26T20:56:58.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.850+0000 7fa52a800640 1 -- 172.21.15.28:0/84473410 shutdown_connections 2024-04-26T20:56:58.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.850+0000 7fa52a800640 1 --2- 172.21.15.28:0/84473410 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5240cfe70 0x7fa5240d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.850+0000 7fa52a800640 1 --2- 172.21.15.28:0/84473410 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa52415f200 0x7fa5240bacc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.850+0000 7fa52a800640 1 -- 172.21.15.28:0/84473410 >> 172.21.15.28:0/84473410 conn(0x7fa5240b6bb0 msgr2=0x7fa5240b6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.850+0000 7fadf6a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadf00b61c0 0x7fadf015ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.850+0000 7fadf6a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fadf01612d0 0x7fadf015d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.850+0000 7fadf6a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadf0163c00 0x7fadf00731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.850+0000 7fadf6a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fadf01659f0 con 0x7fadf01612d0 2024-04-26T20:56:58.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.850+0000 7fadf6a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fadf0165870 con 0x7fadf00b61c0 2024-04-26T20:56:58.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.850+0000 7fadf6a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fadf0165b70 con 0x7fadf0163c00 2024-04-26T20:56:58.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.850+0000 7fb635600640 1 --2- 172.21.15.28:0/3325219525 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb61806c940 0x7fb61806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.850+0000 7fa52a800640 1 -- 172.21.15.28:0/84473410 shutdown_connections 2024-04-26T20:56:58.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.851+0000 7fadf4a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadf0163c00 0x7fadf00731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.851+0000 7fadefe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadf00b61c0 0x7fadf015ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.851+0000 7fadefe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadf00b61c0 0x7fadf015ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.28:35710/0 (socket says 172.21.15.28:35710) 2024-04-26T20:56:58.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.851+0000 7fadefe00640 1 -- 172.21.15.28:0/1497154816 learned_addr learned my addr 172.21.15.28:0/1497154816 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.852+0000 7fd91be00640 1 --2- 172.21.15.28:0/2361792812 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd91c0d00f0 0x7fd91c1607e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd9080097a0 tx=0x7fd90802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41461e1cbfae934a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.850+0000 7fb635600640 1 -- 172.21.15.28:0/3325219525 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb62c08ba80 con 0x7fb6380b6b10 2024-04-26T20:56:58.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.851+0000 7fb63d600640 1 --2- 172.21.15.28:0/3325219525 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb61806c940 0x7fb61806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.852+0000 7fb63d600640 1 --2- 172.21.15.28:0/3325219525 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb61806c940 0x7fb61806ee00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb624034000 tx=0x7fb62402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.852+0000 7fadef400640 1 --2- 172.21.15.28:0/1497154816 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fadf01612d0 0x7fadf015d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.852+0000 7fc5ab200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc590004800 0x7fc5ac159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.852+0000 7fd91b400640 1 -- 172.21.15.28:0/2361792812 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908037070 con 0x7fd91c0d00f0 2024-04-26T20:56:58.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.853+0000 7fd91b400640 1 -- 172.21.15.28:0/2361792812 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd908032440 con 0x7fd91c0d00f0 2024-04-26T20:56:58.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.853+0000 7fd91b400640 1 -- 172.21.15.28:0/2361792812 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd908032850 con 0x7fd91c0d00f0 2024-04-26T20:56:58.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.853+0000 7fadefe00640 1 -- 172.21.15.28:0/1497154816 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadf0163c00 msgr2=0x7fadf00731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.853+0000 7fadefe00640 1 --2- 172.21.15.28:0/1497154816 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadf0163c00 0x7fadf00731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.853+0000 7fadefe00640 1 -- 172.21.15.28:0/1497154816 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fadf01612d0 msgr2=0x7fadf015d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.853+0000 7fadefe00640 1 --2- 172.21.15.28:0/1497154816 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fadf01612d0 0x7fadf015d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.853+0000 7fadefe00640 1 -- 172.21.15.28:0/1497154816 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf015e020 con 0x7fadf00b61c0 2024-04-26T20:56:58.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.853+0000 7fc5ab200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5ac15e250 0x7fc5ac1605e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.853+0000 7fc5ab200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5ac162be0 0x7fc5ac152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.853+0000 7fc5ab200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac0b6a90 con 0x7fc5ac162be0 2024-04-26T20:56:58.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.853+0000 7fc5ab200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac0b6910 con 0x7fc5ac15e250 2024-04-26T20:56:58.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.853+0000 7fc5ab200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc5ac0b6c10 con 0x7fc590004800 2024-04-26T20:56:58.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.850+0000 7efdc9800640 1 -- 172.21.15.28:0/185421119 wait complete. 2024-04-26T20:56:58.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.853+0000 7efdc9800640 1 Processor -- start 2024-04-26T20:56:58.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.853+0000 7fc5a9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5ac15e250 0x7fc5ac1605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.853+0000 7fc5aa800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5ac162be0 0x7fc5ac152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.853+0000 7fc5aa800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5ac162be0 0x7fc5ac152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42282/0 (socket says 172.21.15.28:42282) 2024-04-26T20:56:58.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.853+0000 7fc5aa800640 1 -- 172.21.15.28:0/84208748 learned_addr learned my addr 172.21.15.28:0/84208748 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.851+0000 7fa52a800640 1 -- 172.21.15.28:0/84473410 wait complete. 2024-04-26T20:56:58.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.853+0000 7fc5a9e00640 1 --2- 172.21.15.28:0/84208748 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc590004800 0x7fc5ac159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.853+0000 7fa52a800640 1 Processor -- start 2024-04-26T20:56:58.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.854+0000 7fd923c00640 1 -- 172.21.15.28:0/2361792812 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd91c0d00f0 msgr2=0x7fd91c1607e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.854+0000 7fd923c00640 1 --2- 172.21.15.28:0/2361792812 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd91c0d00f0 0x7fd91c1607e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd9080097a0 tx=0x7fd90802a130 comp rx=0 tx=0).stop 2024-04-26T20:56:58.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.854+0000 7fadefe00640 1 --2- 172.21.15.28:0/1497154816 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadf00b61c0 0x7fadf015ef10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fade8009f70 tx=0x7fade8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.854+0000 7fa52a800640 1 -- start start 2024-04-26T20:56:58.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.854+0000 7efdc9800640 1 -- start start 2024-04-26T20:56:58.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.854+0000 7fc5a9400640 1 -- 172.21.15.28:0/84208748 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc590004800 msgr2=0x7fc5ac159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.854+0000 7fc5a9400640 1 --2- 172.21.15.28:0/84208748 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc590004800 0x7fc5ac159b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.854+0000 7fc5a9400640 1 -- 172.21.15.28:0/84208748 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5ac162be0 msgr2=0x7fc5ac152c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.854+0000 7fc5a9400640 1 --2- 172.21.15.28:0/84208748 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5ac162be0 0x7fc5ac152c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.854+0000 7fc5a9400640 1 -- 172.21.15.28:0/84208748 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac159380 con 0x7fc5ac15e250 2024-04-26T20:56:58.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.854+0000 7fd923c00640 1 -- 172.21.15.28:0/2361792812 shutdown_connections 2024-04-26T20:56:58.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.854+0000 7fade7e00640 1 -- 172.21.15.28:0/1497154816 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade8041020 con 0x7fadf00b61c0 2024-04-26T20:56:58.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.854+0000 7fade7e00640 1 -- 172.21.15.28:0/1497154816 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fade80087b0 con 0x7fadf00b61c0 2024-04-26T20:56:58.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.854+0000 7fade7e00640 1 -- 172.21.15.28:0/1497154816 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade8036040 con 0x7fadf00b61c0 2024-04-26T20:56:58.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.854+0000 7fadf6a00640 1 -- 172.21.15.28:0/1497154816 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf015e300 con 0x7fadf00b61c0 2024-04-26T20:56:58.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.854+0000 7fd923c00640 1 --2- 172.21.15.28:0/2361792812 >> v2:172.21.15.28:3300/0 conn(0x7fd91c0b6b10 0x7fd91c0b6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.854+0000 7fd923c00640 1 --2- 172.21.15.28:0/2361792812 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd91c0d00f0 0x7fd91c1607e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.854+0000 7fd923c00640 1 -- 172.21.15.28:0/2361792812 >> 172.21.15.28:0/2361792812 conn(0x7fd91c0d1b10 msgr2=0x7fd91c0d1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.855+0000 7fd923c00640 1 -- 172.21.15.28:0/2361792812 shutdown_connections 2024-04-26T20:56:58.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.855+0000 7fa52a800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5240cfe70 0x7fa5241f75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.855+0000 7fa52a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa52415f200 0x7fa52415dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.855+0000 7fc5a9400640 1 --2- 172.21.15.28:0/84208748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5ac15e250 0x7fc5ac1605e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc59000c1e0 tx=0x7fc5a402d070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.855+0000 7efdc9800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efdc40ba710 0x7efdc41f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.855+0000 7efdc9800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc40d1f10 0x7efdc41f6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.855+0000 7efdc9800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdc415f070 0x7efdc41f6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.855+0000 7efdc9800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efdc40d3bf0 con 0x7efdc40d1f10 2024-04-26T20:56:58.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.855+0000 7efdc9800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efdc40d04a0 con 0x7efdc415f070 2024-04-26T20:56:58.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.855+0000 7efdc9800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7efdc40d3d70 con 0x7efdc40ba710 2024-04-26T20:56:58.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.856+0000 7fd923c00640 1 -- 172.21.15.28:0/2361792812 wait complete. 2024-04-26T20:56:58.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.855+0000 7fadf6a00640 1 -- 172.21.15.28:0/1497154816 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf00aa550 con 0x7fadf00b61c0 2024-04-26T20:56:58.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.856+0000 7fb635600640 1 -- 172.21.15.28:0/3325219525 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb62c01f030 con 0x7fb6380b6b10 2024-04-26T20:56:58.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.855+0000 7fa52a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa52415f5e0 0x7fa5240ce610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.855+0000 7fa52a800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa5240d3ca0 con 0x7fa52415f200 2024-04-26T20:56:58.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.855+0000 7fa52a800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa5240d34a0 con 0x7fa52415f5e0 2024-04-26T20:56:58.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.855+0000 7fa52a800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa5240d3e20 con 0x7fa5240cfe70 2024-04-26T20:56:58.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.856+0000 7fa523e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa52415f5e0 0x7fa5240ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.856+0000 7fa523e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa52415f5e0 0x7fa5240ce610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.28:35728/0 (socket says 172.21.15.28:35728) 2024-04-26T20:56:58.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.856+0000 7fa523e00640 1 -- 172.21.15.28:0/3583618702 learned_addr learned my addr 172.21.15.28:0/3583618702 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.856+0000 7fa523400640 1 --2- 172.21.15.28:0/3583618702 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5240cfe70 0x7fa5241f75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.856+0000 7efdc3e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efdc40ba710 0x7efdc41f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.856+0000 7efdc3e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efdc40ba710 0x7efdc41f5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.28:59396/0 (socket says 172.21.15.28:59396) 2024-04-26T20:56:58.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.856+0000 7efdc3e00640 1 -- 172.21.15.28:0/1734534095 learned_addr learned my addr 172.21.15.28:0/1734534095 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.856+0000 7fa522a00640 1 --2- 172.21.15.28:0/3583618702 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa52415f200 0x7fa52415dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.856+0000 7fadf6a00640 1 -- 172.21.15.28:0/1497154816 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadf006a400 con 0x7fadf00b61c0 2024-04-26T20:56:58.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.857+0000 7fa523e00640 1 -- 172.21.15.28:0/3583618702 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5240cfe70 msgr2=0x7fa5241f75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.857+0000 7fa523e00640 1 --2- 172.21.15.28:0/3583618702 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5240cfe70 0x7fa5241f75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.856+0000 7efdc3400640 1 --2- 172.21.15.28:0/1734534095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc40d1f10 0x7efdc41f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.857+0000 7fa523e00640 1 -- 172.21.15.28:0/3583618702 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa52415f200 msgr2=0x7fa52415dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.857+0000 7efdc3e00640 1 -- 172.21.15.28:0/1734534095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdc415f070 msgr2=0x7efdc41f6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.857+0000 7efdc3e00640 1 --2- 172.21.15.28:0/1734534095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdc415f070 0x7efdc41f6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.857+0000 7efdc3e00640 1 -- 172.21.15.28:0/1734534095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc40d1f10 msgr2=0x7efdc41f6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.857+0000 7fa523e00640 1 --2- 172.21.15.28:0/3583618702 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa52415f200 0x7fa52415dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.857+0000 7efdc3e00640 1 --2- 172.21.15.28:0/1734534095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc40d1f10 0x7efdc41f6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.857+0000 7efdc3e00640 1 -- 172.21.15.28:0/1734534095 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc4201ce0 con 0x7efdc40ba710 2024-04-26T20:56:58.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.858+0000 7efdc3400640 1 --2- 172.21.15.28:0/1734534095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc40d1f10 0x7efdc41f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:56:58.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.858+0000 7fa523e00640 1 -- 172.21.15.28:0/3583618702 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5241f5b50 con 0x7fa52415f5e0 2024-04-26T20:56:58.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.858+0000 7fa522a00640 1 --2- 172.21.15.28:0/3583618702 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa52415f200 0x7fa52415dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:56:58.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.856+0000 7fc5a2000640 1 -- 172.21.15.28:0/84208748 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a400a4c0 con 0x7fc5ac15e250 2024-04-26T20:56:58.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.857+0000 7fc5a2000640 1 -- 172.21.15.28:0/84208748 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a4034880 con 0x7fc5ac15e250 2024-04-26T20:56:58.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.857+0000 7fc5a2000640 1 -- 172.21.15.28:0/84208748 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4034c30 con 0x7fc5ac15e250 2024-04-26T20:56:58.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.857+0000 7fc5ab200640 1 -- 172.21.15.28:0/84208748 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5ac15e7c0 con 0x7fc5ac15e250 2024-04-26T20:56:58.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.857+0000 7fc5ab200640 1 -- 172.21.15.28:0/84208748 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5ac159620 con 0x7fc5ac15e250 2024-04-26T20:56:58.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.858+0000 7fd923c00640 1 Processor -- start 2024-04-26T20:56:58.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.858+0000 7efdc3e00640 1 --2- 172.21.15.28:0/1734534095 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efdc40ba710 0x7efdc41f5cc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efdb402a700 tx=0x7efdb40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.859+0000 7fd923c00640 1 -- start start 2024-04-26T20:56:58.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.859+0000 7fa523e00640 1 --2- 172.21.15.28:0/3583618702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa52415f5e0 0x7fa5240ce610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009fd0 tx=0x7fa51c007f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.860+0000 7fd923c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd91c0b43c0 0x7fd91c159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.859+0000 7efdbbe00640 1 -- 172.21.15.28:0/1734534095 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb4042020 con 0x7efdc40ba710 2024-04-26T20:56:58.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.859+0000 7efdbbe00640 1 -- 172.21.15.28:0/1734534095 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb403d730 con 0x7efdc40ba710 2024-04-26T20:56:58.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.859+0000 7efdbbe00640 1 -- 172.21.15.28:0/1734534095 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb4037040 con 0x7efdc40ba710 2024-04-26T20:56:58.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.860+0000 7efdc9800640 1 -- 172.21.15.28:0/1734534095 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc4201fc0 con 0x7efdc40ba710 2024-04-26T20:56:58.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.861+0000 7efdc9800640 1 -- 172.21.15.28:0/1734534095 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc4202580 con 0x7efdc40ba710 2024-04-26T20:56:58.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.861+0000 7fc5ab200640 1 -- 172.21.15.28:0/84208748 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5ac06a360 con 0x7fc5ac15e250 2024-04-26T20:56:58.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.860+0000 7fd923c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd91c0b6b10 0x7fd91c0cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.860+0000 7fd923c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd91c0d00f0 0x7fd91c0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.861+0000 7fd921200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd91c0b43c0 0x7fd91c159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.861+0000 7fd921c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd91c0d00f0 0x7fd91c0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.861+0000 7fd921c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd91c0d00f0 0x7fd91c0cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42324/0 (socket says 172.21.15.28:42324) 2024-04-26T20:56:58.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.859+0000 7fade7e00640 1 -- 172.21.15.28:0/1497154816 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fade8044020 con 0x7fadf00b61c0 2024-04-26T20:56:58.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.861+0000 7fade7e00640 1 --2- 172.21.15.28:0/1497154816 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fadcc06c9c0 0x7fadcc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.861+0000 7fade7e00640 1 -- 172.21.15.28:0/1497154816 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fade80acec0 con 0x7fadf00b61c0 2024-04-26T20:56:58.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.860+0000 7f1f00c00640 1 -- 172.21.15.28:0/2401730747 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ef4039040 con 0x7f1f040b43c0 2024-04-26T20:56:58.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.861+0000 7fd921200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd91c0b43c0 0x7fd91c159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.28:59398/0 (socket says 172.21.15.28:59398) 2024-04-26T20:56:58.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.861+0000 7fd921c00640 1 -- 172.21.15.28:0/719608262 learned_addr learned my addr 172.21.15.28:0/719608262 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.861+0000 7fd923c00640 1 -- 172.21.15.28:0/719608262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c163790 con 0x7fd91c0d00f0 2024-04-26T20:56:58.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.861+0000 7fd923c00640 1 -- 172.21.15.28:0/719608262 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd91c1634b0 con 0x7fd91c0b6b10 2024-04-26T20:56:58.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.861+0000 7fd923c00640 1 -- 172.21.15.28:0/719608262 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fd91c0d3880 con 0x7fd91c0b43c0 2024-04-26T20:56:58.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.862+0000 7fd91be00640 1 --2- 172.21.15.28:0/719608262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd91c0b6b10 0x7fd91c0cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.862+0000 7fd921c00640 1 -- 172.21.15.28:0/719608262 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd91c0b43c0 msgr2=0x7fd91c159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.862+0000 7fd921c00640 1 --2- 172.21.15.28:0/719608262 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd91c0b43c0 0x7fd91c159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.862+0000 7fd921c00640 1 -- 172.21.15.28:0/719608262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd91c0b6b10 msgr2=0x7fd91c0cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.862+0000 7fd921c00640 1 --2- 172.21.15.28:0/719608262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd91c0b6b10 0x7fd91c0cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.862+0000 7fd921c00640 1 -- 172.21.15.28:0/719608262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91c0c8dd0 con 0x7fd91c0d00f0 2024-04-26T20:56:58.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.862+0000 7fadef400640 1 --2- 172.21.15.28:0/1497154816 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fadcc06c9c0 0x7fadcc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.862+0000 7fd921c00640 1 --2- 172.21.15.28:0/719608262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd91c0d00f0 0x7fd91c0cb790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd91400e190 tx=0x7fd91400e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.862+0000 7efdc9800640 1 -- 172.21.15.28:0/1734534095 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdc406a360 con 0x7efdc40ba710 2024-04-26T20:56:58.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.863+0000 7fadef400640 1 --2- 172.21.15.28:0/1497154816 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fadcc06c9c0 0x7fadcc06ee80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fadf00b04e0 tx=0x7fadd8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.863+0000 7f1f00c00640 1 --2- 172.21.15.28:0/2401730747 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1ee006c940 0x7f1ee006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.863+0000 7f1f00c00640 1 -- 172.21.15.28:0/2401730747 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1ef40acfa0 con 0x7f1f040b43c0 2024-04-26T20:56:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.864+0000 7f1f03e00640 1 --2- 172.21.15.28:0/2401730747 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1ee006c940 0x7f1ee006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.864+0000 7f1f03e00640 1 --2- 172.21.15.28:0/2401730747 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1ee006c940 0x7f1ee006ee00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1f040cf2f0 tx=0x7f1efc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.865+0000 7efdbbe00640 1 -- 172.21.15.28:0/1734534095 <== mon.2 v2:172.21.15.103:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdb4045020 con 0x7efdc40ba710 2024-04-26T20:56:58.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.866+0000 7fd918c00640 1 -- 172.21.15.28:0/719608262 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd914014620 con 0x7fd91c0d00f0 2024-04-26T20:56:58.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.866+0000 7fd918c00640 1 -- 172.21.15.28:0/719608262 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd914019070 con 0x7fd91c0d00f0 2024-04-26T20:56:58.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.866+0000 7fd918c00640 1 -- 172.21.15.28:0/719608262 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9140149c0 con 0x7fd91c0d00f0 2024-04-26T20:56:58.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.866+0000 7fd923c00640 1 -- 172.21.15.28:0/719608262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd91c0c90b0 con 0x7fd91c0d00f0 2024-04-26T20:56:58.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.867+0000 7fd923c00640 1 -- 172.21.15.28:0/719608262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd91c159880 con 0x7fd91c0d00f0 2024-04-26T20:56:58.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.867+0000 7efdbbe00640 1 --2- 172.21.15.28:0/1734534095 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efda406c8f0 0x7efda406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.867+0000 7efdbbe00640 1 -- 172.21.15.28:0/1734534095 <== mon.2 v2:172.21.15.103:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efdb40ac940 con 0x7efdc40ba710 2024-04-26T20:56:58.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.867+0000 7f9ff4e00640 1 Processor -- start 2024-04-26T20:56:58.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.867+0000 7f1f00c00640 1 -- 172.21.15.28:0/2401730747 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ef407af10 con 0x7f1f040b43c0 2024-04-26T20:56:58.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.868+0000 7efdc3400640 1 --2- 172.21.15.28:0/1734534095 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efda406c8f0 0x7efda406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.867+0000 7f9ff4e00640 1 -- start start 2024-04-26T20:56:58.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.868+0000 7fd923c00640 1 -- 172.21.15.28:0/719608262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd91c06a400 con 0x7fd91c0d00f0 2024-04-26T20:56:58.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.869+0000 7efdc3400640 1 --2- 172.21.15.28:0/1734534095 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efda406c8f0 0x7efda406edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efdc40d30d0 tx=0x7efdb0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.869+0000 7f9ff4e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9ff00b43c0 0x7f9ff00d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.869+0000 7f9ff4e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9ff00b6b10 0x7f9ff00b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.869+0000 7f9ff4e00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ff00b47a0 con 0x7f9ff01607e0 2024-04-26T20:56:58.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.869+0000 7f9ff4e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f9ff00b74a0 con 0x7f9ff00b43c0 2024-04-26T20:56:58.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.869+0000 7f9ff4e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff0160c90 con 0x7f9ff00b6b10 2024-04-26T20:56:58.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.870+0000 7f9fede00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9ff00b43c0 0x7f9ff00d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.870+0000 7fd918c00640 1 -- 172.21.15.28:0/719608262 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd914014b60 con 0x7fd91c0d00f0 2024-04-26T20:56:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.870+0000 7f9fede00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9ff00b43c0 0x7f9ff00d00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42330/0 (socket says 172.21.15.28:42330) 2024-04-26T20:56:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.870+0000 7f9fede00640 1 -- 172.21.15.28:0/3311569965 learned_addr learned my addr 172.21.15.28:0/3311569965 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.870+0000 7f9fee800640 1 --2- 172.21.15.28:0/3311569965 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9ff00b6b10 0x7f9ff00b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.870+0000 7f9fede00640 1 -- 172.21.15.28:0/3311569965 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9ff00b6b10 msgr2=0x7f9ff00b6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.870+0000 7f9fede00640 1 --2- 172.21.15.28:0/3311569965 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9ff00b6b10 0x7f9ff00b6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.870+0000 7f9fede00640 1 -- 172.21.15.28:0/3311569965 >> v1:172.21.15.103:6789/0 conn(0x7f9ff01607e0 legacy=0x7f9ff0163cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:58.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.870+0000 7f9fede00640 1 -- 172.21.15.28:0/3311569965 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff00d0740 con 0x7f9ff00b43c0 2024-04-26T20:56:58.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.871+0000 7fd918c00640 1 --2- 172.21.15.28:0/719608262 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd8fc06c8f0 0x7fd8fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.871+0000 7fd918c00640 1 -- 172.21.15.28:0/719608262 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd91408c3c0 con 0x7fd91c0d00f0 2024-04-26T20:56:58.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.871+0000 7fd921200640 1 --2- 172.21.15.28:0/719608262 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd8fc06c8f0 0x7fd8fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.872+0000 7fd921200640 1 --2- 172.21.15.28:0/719608262 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd8fc06c8f0 0x7fd8fc06edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd910005880 tx=0x7fd910009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.873+0000 7f9fede00640 1 --2- 172.21.15.28:0/3311569965 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ff00b43c0 0x7f9ff00d00f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009870 tx=0x7f9fe002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1473d641b0200bb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.875+0000 7f9feca00640 1 -- 172.21.15.28:0/3311569965 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0037070 con 0x7f9ff00b43c0 2024-04-26T20:56:58.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.875+0000 7f9feca00640 1 -- 172.21.15.28:0/3311569965 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fe0032440 con 0x7f9ff00b43c0 2024-04-26T20:56:58.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.875+0000 7f9feca00640 1 -- 172.21.15.28:0/3311569965 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0032810 con 0x7f9ff00b43c0 2024-04-26T20:56:58.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.876+0000 7fd918c00640 1 -- 172.21.15.28:0/719608262 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd91405a880 con 0x7fd91c0d00f0 2024-04-26T20:56:58.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.876+0000 7fa51b400640 1 -- 172.21.15.28:0/3583618702 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c018070 con 0x7fa52415f5e0 2024-04-26T20:56:58.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.876+0000 7fa51b400640 1 -- 172.21.15.28:0/3583618702 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa51c00d860 con 0x7fa52415f5e0 2024-04-26T20:56:58.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.876+0000 7fa51b400640 1 -- 172.21.15.28:0/3583618702 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c003e60 con 0x7fa52415f5e0 2024-04-26T20:56:58.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.876+0000 7fa52a800640 1 -- 172.21.15.28:0/3583618702 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5241f5e30 con 0x7fa52415f5e0 2024-04-26T20:56:58.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.877+0000 7fa52a800640 1 -- 172.21.15.28:0/3583618702 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5241f63f0 con 0x7fa52415f5e0 2024-04-26T20:56:58.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.876+0000 7fade7e00640 1 -- 172.21.15.28:0/1497154816 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fade804b070 con 0x7fadf00b61c0 2024-04-26T20:56:58.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.877+0000 7f9ff4e00640 1 -- 172.21.15.28:0/3311569965 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ff00b43c0 msgr2=0x7f9ff00d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.877+0000 7f9ff4e00640 1 --2- 172.21.15.28:0/3311569965 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ff00b43c0 0x7f9ff00d00f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0009870 tx=0x7f9fe002a180 comp rx=0 tx=0).stop 2024-04-26T20:56:58.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.878+0000 7f9ff4e00640 1 -- 172.21.15.28:0/3311569965 shutdown_connections 2024-04-26T20:56:58.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.878+0000 7f9ff4e00640 1 --2- 172.21.15.28:0/3311569965 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9ff00b6b10 0x7f9ff00b6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.878+0000 7f9ff4e00640 1 --2- 172.21.15.28:0/3311569965 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ff00b43c0 0x7f9ff00d00f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.878+0000 7f9ff4e00640 1 -- 172.21.15.28:0/3311569965 >> 172.21.15.28:0/3311569965 conn(0x7f9ff00d1b10 msgr2=0x7f9ff00d1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:58.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.878+0000 7fa52a800640 1 -- 172.21.15.28:0/3583618702 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa52406a400 con 0x7fa52415f5e0 2024-04-26T20:56:58.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.878+0000 7f9ff4e00640 1 -- 172.21.15.28:0/3311569965 shutdown_connections 2024-04-26T20:56:58.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.878+0000 7f9ff4e00640 1 -- 172.21.15.28:0/3311569965 wait complete. 2024-04-26T20:56:58.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.879+0000 7fc5a2000640 1 -- 172.21.15.28:0/84208748 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5a4049430 con 0x7fc5ac15e250 2024-04-26T20:56:58.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.880+0000 7fc5a2000640 1 --2- 172.21.15.28:0/84208748 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc58806c9f0 0x7fc58806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.880+0000 7fc5a2000640 1 -- 172.21.15.28:0/84208748 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc5a40ac8b0 con 0x7fc5ac15e250 2024-04-26T20:56:58.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.880+0000 7fc5a9e00640 1 --2- 172.21.15.28:0/84208748 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc58806c9f0 0x7fc58806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.881+0000 7f9ff4e00640 1 Processor -- start 2024-04-26T20:56:58.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.881+0000 7fc5a9e00640 1 --2- 172.21.15.28:0/84208748 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc58806c9f0 0x7fc58806eeb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac1634c0 tx=0x7fc594001250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.881+0000 7f9ff4e00640 1 -- start start 2024-04-26T20:56:58.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.882+0000 7efdbbe00640 1 -- 172.21.15.28:0/1734534095 <== mon.2 v2:172.21.15.103:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efdb404c030 con 0x7efdc40ba710 2024-04-26T20:56:58.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.883+0000 7f9ff4e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ff00b43c0 0x7f9ff00c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.883+0000 7f9ff4e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ff00b6b10 0x7f9ff00c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.883+0000 7f9ff4e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9ff01607e0 0x7f9ff015f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.883+0000 7f9ff4e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0163790 con 0x7f9ff00b43c0 2024-04-26T20:56:58.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.883+0000 7f9ff4e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff01634b0 con 0x7f9ff00b6b10 2024-04-26T20:56:58.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.883+0000 7f9ff4e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff00d3880 con 0x7f9ff01607e0 2024-04-26T20:56:58.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.883+0000 7f9fede00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ff00b43c0 0x7f9ff00c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.884+0000 7f9fede00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ff00b43c0 0x7f9ff00c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42342/0 (socket says 172.21.15.28:42342) 2024-04-26T20:56:58.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.884+0000 7f9fede00640 1 -- 172.21.15.28:0/625825263 learned_addr learned my addr 172.21.15.28:0/625825263 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:58.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.884+0000 7f9fee800640 1 --2- 172.21.15.28:0/625825263 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9ff01607e0 0x7f9ff015f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.884+0000 7f9fed400640 1 --2- 172.21.15.28:0/625825263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ff00b6b10 0x7f9ff00c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.884+0000 7f9fede00640 1 -- 172.21.15.28:0/625825263 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9ff01607e0 msgr2=0x7f9ff015f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.884+0000 7f9fede00640 1 --2- 172.21.15.28:0/625825263 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9ff01607e0 0x7f9ff015f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.884+0000 7f9fede00640 1 -- 172.21.15.28:0/625825263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ff00b6b10 msgr2=0x7f9ff00c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:58.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.884+0000 7f9fede00640 1 --2- 172.21.15.28:0/625825263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ff00b6b10 0x7f9ff00c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:58.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.884+0000 7f9fede00640 1 -- 172.21.15.28:0/625825263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff00c8db0 con 0x7f9ff00b43c0 2024-04-26T20:56:58.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.885+0000 7f9fede00640 1 --2- 172.21.15.28:0/625825263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ff00b43c0 0x7f9ff00c8740 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0000c00 tx=0x7f9fe00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.885+0000 7f9fe6000640 1 -- 172.21.15.28:0/625825263 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0041020 con 0x7f9ff00b43c0 2024-04-26T20:56:58.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.885+0000 7f9fe6000640 1 -- 172.21.15.28:0/625825263 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fe00306c0 con 0x7f9ff00b43c0 2024-04-26T20:56:58.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.885+0000 7f9fe6000640 1 -- 172.21.15.28:0/625825263 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0037040 con 0x7f9ff00b43c0 2024-04-26T20:56:58.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.885+0000 7f9ff4e00640 1 -- 172.21.15.28:0/625825263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff00c9090 con 0x7f9ff00b43c0 2024-04-26T20:56:58.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.885+0000 7f9ff4e00640 1 -- 172.21.15.28:0/625825263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff00c9650 con 0x7f9ff00b43c0 2024-04-26T20:56:58.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.887+0000 7f9ff4e00640 1 -- 172.21.15.28:0/625825263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff00d17e0 con 0x7f9ff00b43c0 2024-04-26T20:56:58.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.890+0000 7f9fe6000640 1 -- 172.21.15.28:0/625825263 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9fe0044020 con 0x7f9ff00b43c0 2024-04-26T20:56:58.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.891+0000 7f9fe6000640 1 --2- 172.21.15.28:0/625825263 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9fd006c940 0x7f9fd006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.891+0000 7f9fe6000640 1 -- 172.21.15.28:0/625825263 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9fe00ac800 con 0x7f9ff00b43c0 2024-04-26T20:56:58.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.892+0000 7fc5a2000640 1 -- 172.21.15.28:0/84208748 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc5a407acc0 con 0x7fc5ac15e250 2024-04-26T20:56:58.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.893+0000 7f9fed400640 1 --2- 172.21.15.28:0/625825263 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9fd006c940 0x7f9fd006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.894+0000 7f9fed400640 1 --2- 172.21.15.28:0/625825263 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9fd006c940 0x7f9fd006ee00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc006fd0 tx=0x7f9fdc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.895+0000 7fa51b400640 1 -- 172.21.15.28:0/3583618702 <== mon.1 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa51c01b070 con 0x7fa52415f5e0 2024-04-26T20:56:58.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.896+0000 7fa51b400640 1 --2- 172.21.15.28:0/3583618702 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa50406c8f0 0x7fa50406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:58.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.897+0000 7fa51b400640 1 -- 172.21.15.28:0/3583618702 <== mon.1 v2:172.21.15.103:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa51c08b990 con 0x7fa52415f5e0 2024-04-26T20:56:58.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.897+0000 7fa523400640 1 --2- 172.21.15.28:0/3583618702 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa50406c8f0 0x7fa50406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:58.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.900+0000 7f9fe6000640 1 -- 172.21.15.28:0/625825263 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9ff00d17e0 con 0x7f9ff00b43c0 2024-04-26T20:56:58.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.900+0000 7fa523400640 1 --2- 172.21.15.28:0/3583618702 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa50406c8f0 0x7fa50406edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa510034000 tx=0x7fa51002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:58.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:58.908+0000 7fa51b400640 1 -- 172.21.15.28:0/3583618702 <== mon.1 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa51c022020 con 0x7fa52415f5e0 2024-04-26T20:56:59.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.300+0000 7f1f0b600640 1 -- 172.21.15.28:0/2401730747 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f1f040cfb00 con 0x7f1f040b43c0 2024-04-26T20:56:59.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.301+0000 7f1f00c00640 1 -- 172.21.15.28:0/2401730747 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1ef407f210 con 0x7f1f040b43c0 2024-04-26T20:56:59.302 INFO:teuthology.orchestra.run.smithi028.stdout:55834574853 2024-04-26T20:56:59.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.305+0000 7f1f0b600640 1 -- 172.21.15.28:0/2401730747 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1ee006c940 msgr2=0x7f1ee006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.305+0000 7f1f0b600640 1 --2- 172.21.15.28:0/2401730747 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1ee006c940 0x7f1ee006ee00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1f040cf2f0 tx=0x7f1efc008040 comp rx=0 tx=0).stop 2024-04-26T20:56:59.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.305+0000 7f1f0b600640 1 -- 172.21.15.28:0/2401730747 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f040b43c0 msgr2=0x7f1f040cba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.305+0000 7f1f0b600640 1 --2- 172.21.15.28:0/2401730747 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f040b43c0 0x7f1f040cba90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f1f0408d820 tx=0x7f1ef4004a40 comp rx=0 tx=0).stop 2024-04-26T20:56:59.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.306+0000 7f1f0b600640 1 -- 172.21.15.28:0/2401730747 shutdown_connections 2024-04-26T20:56:59.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.306+0000 7f1f0b600640 1 --2- 172.21.15.28:0/2401730747 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1ee006c940 0x7f1ee006ee00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.306+0000 7f1f0b600640 1 --2- 172.21.15.28:0/2401730747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1f041607e0 0x7f1f04153030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.306+0000 7f1f0b600640 1 --2- 172.21.15.28:0/2401730747 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1f040b6b10 0x7f1f040c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.306+0000 7f1f0b600640 1 --2- 172.21.15.28:0/2401730747 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f040b43c0 0x7f1f040cba90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.306+0000 7f1f0b600640 1 -- 172.21.15.28:0/2401730747 >> 172.21.15.28:0/2401730747 conn(0x7f1f040d1b10 msgr2=0x7f1f040b89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:59.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.306+0000 7f1f0b600640 1 -- 172.21.15.28:0/2401730747 shutdown_connections 2024-04-26T20:56:59.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.307+0000 7fb640000640 1 -- 172.21.15.28:0/3325219525 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fb6381616b0 con 0x7fb6380b6b10 2024-04-26T20:56:59.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.306+0000 7f1f0b600640 1 -- 172.21.15.28:0/2401730747 wait complete. 2024-04-26T20:56:59.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.307+0000 7fb635600640 1 -- 172.21.15.28:0/3325219525 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb62c059940 con 0x7fb6380b6b10 2024-04-26T20:56:59.309 INFO:teuthology.orchestra.run.smithi028.stdout:55834574852 2024-04-26T20:56:59.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.311+0000 7fb640000640 1 -- 172.21.15.28:0/3325219525 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb61806c940 msgr2=0x7fb61806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.311+0000 7fb640000640 1 --2- 172.21.15.28:0/3325219525 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb61806c940 0x7fb61806ee00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb624034000 tx=0x7fb62402e000 comp rx=0 tx=0).stop 2024-04-26T20:56:59.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.311+0000 7fb640000640 1 -- 172.21.15.28:0/3325219525 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6380b6b10 msgr2=0x7fb6380c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.311+0000 7fb640000640 1 --2- 172.21.15.28:0/3325219525 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6380b6b10 0x7fb6380c8740 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb62c00b3f0 tx=0x7fb62c00b8b0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.312+0000 7fb640000640 1 -- 172.21.15.28:0/3325219525 shutdown_connections 2024-04-26T20:56:59.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.312+0000 7fb640000640 1 --2- 172.21.15.28:0/3325219525 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb61806c940 0x7fb61806ee00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.312+0000 7fb640000640 1 --2- 172.21.15.28:0/3325219525 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6381607e0 0x7fb6380b8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.312+0000 7fb640000640 1 --2- 172.21.15.28:0/3325219525 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6380b6b10 0x7fb6380c8740 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.312+0000 7fb640000640 1 --2- 172.21.15.28:0/3325219525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6380b43c0 0x7fb63815eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.312+0000 7fb640000640 1 -- 172.21.15.28:0/3325219525 >> 172.21.15.28:0/3325219525 conn(0x7fb6380d1b10 msgr2=0x7fb638160ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:59.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.313+0000 7fb640000640 1 -- 172.21.15.28:0/3325219525 shutdown_connections 2024-04-26T20:56:59.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.313+0000 7fb640000640 1 -- 172.21.15.28:0/3325219525 wait complete. 2024-04-26T20:56:59.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.325+0000 7fd923c00640 1 -- 172.21.15.28:0/719608262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fd91c0b68f0 con 0x7fd91c0d00f0 2024-04-26T20:56:59.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.326+0000 7fd918c00640 1 -- 172.21.15.28:0/719608262 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd91405a180 con 0x7fd91c0d00f0 2024-04-26T20:56:59.327 INFO:teuthology.orchestra.run.smithi028.stdout:55834574852 2024-04-26T20:56:59.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.326+0000 7efdc9800640 1 -- 172.21.15.28:0/1734534095 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7efdc40d2e20 con 0x7efdc40ba710 2024-04-26T20:56:59.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.327+0000 7efdbbe00640 1 -- 172.21.15.28:0/1734534095 <== mon.2 v2:172.21.15.103:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efdb407aee0 con 0x7efdc40ba710 2024-04-26T20:56:59.328 INFO:teuthology.orchestra.run.smithi028.stdout:55834574853 2024-04-26T20:56:59.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.329+0000 7fadf6a00640 1 -- 172.21.15.28:0/1497154816 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fadf0167d20 con 0x7fadf00b61c0 2024-04-26T20:56:59.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.330+0000 7fd923c00640 1 -- 172.21.15.28:0/719608262 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd8fc06c8f0 msgr2=0x7fd8fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.330+0000 7fd923c00640 1 --2- 172.21.15.28:0/719608262 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd8fc06c8f0 0x7fd8fc06edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd910005880 tx=0x7fd910009040 comp rx=0 tx=0).stop 2024-04-26T20:56:59.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.330+0000 7fd923c00640 1 -- 172.21.15.28:0/719608262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd91c0d00f0 msgr2=0x7fd91c0cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.330+0000 7fd923c00640 1 --2- 172.21.15.28:0/719608262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd91c0d00f0 0x7fd91c0cb790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd91400e190 tx=0x7fd91400e650 comp rx=0 tx=0).stop 2024-04-26T20:56:59.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.331+0000 7fade7e00640 1 -- 172.21.15.28:0/1497154816 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fade807ae30 con 0x7fadf00b61c0 2024-04-26T20:56:59.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.331+0000 7fd923c00640 1 -- 172.21.15.28:0/719608262 shutdown_connections 2024-04-26T20:56:59.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.331+0000 7fd923c00640 1 --2- 172.21.15.28:0/719608262 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd8fc06c8f0 0x7fd8fc06edb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.331+0000 7fd923c00640 1 --2- 172.21.15.28:0/719608262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd91c0d00f0 0x7fd91c0cb790 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.331+0000 7fd923c00640 1 --2- 172.21.15.28:0/719608262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd91c0b6b10 0x7fd91c0cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.331+0000 7fd923c00640 1 --2- 172.21.15.28:0/719608262 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd91c0b43c0 0x7fd91c159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.331+0000 7fd923c00640 1 -- 172.21.15.28:0/719608262 >> 172.21.15.28:0/719608262 conn(0x7fd91c0d1b10 msgr2=0x7fd91c160ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:59.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.331+0000 7fd923c00640 1 -- 172.21.15.28:0/719608262 shutdown_connections 2024-04-26T20:56:59.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.331+0000 7fd923c00640 1 -- 172.21.15.28:0/719608262 wait complete. 2024-04-26T20:56:59.335 INFO:teuthology.orchestra.run.smithi028.stdout:55834574852 2024-04-26T20:56:59.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.331+0000 7efdc9800640 1 -- 172.21.15.28:0/1734534095 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efda406c8f0 msgr2=0x7efda406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.331+0000 7efdc9800640 1 --2- 172.21.15.28:0/1734534095 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efda406c8f0 0x7efda406edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efdc40d30d0 tx=0x7efdb0009040 comp rx=0 tx=0).stop 2024-04-26T20:56:59.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.332+0000 7efdc9800640 1 -- 172.21.15.28:0/1734534095 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efdc40ba710 msgr2=0x7efdc41f5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.332+0000 7efdc9800640 1 --2- 172.21.15.28:0/1734534095 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efdc40ba710 0x7efdc41f5cc0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efdb402a700 tx=0x7efdb40046c0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.332+0000 7efdc9800640 1 -- 172.21.15.28:0/1734534095 shutdown_connections 2024-04-26T20:56:59.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.332+0000 7efdc9800640 1 --2- 172.21.15.28:0/1734534095 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efda406c8f0 0x7efda406edb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.332+0000 7efdc9800640 1 --2- 172.21.15.28:0/1734534095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efdc415f070 0x7efdc41f6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.332+0000 7efdc9800640 1 --2- 172.21.15.28:0/1734534095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efdc40d1f10 0x7efdc41f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.332+0000 7efdc9800640 1 --2- 172.21.15.28:0/1734534095 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efdc40ba710 0x7efdc41f5cc0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.332+0000 7efdc9800640 1 -- 172.21.15.28:0/1734534095 >> 172.21.15.28:0/1734534095 conn(0x7efdc406c980 msgr2=0x7efdc40cbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:59.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.332+0000 7efdc9800640 1 -- 172.21.15.28:0/1734534095 shutdown_connections 2024-04-26T20:56:59.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.332+0000 7efdc9800640 1 -- 172.21.15.28:0/1734534095 wait complete. 2024-04-26T20:56:59.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.335+0000 7fadf6a00640 1 -- 172.21.15.28:0/1497154816 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fadcc06c9c0 msgr2=0x7fadcc06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.335+0000 7fadf6a00640 1 --2- 172.21.15.28:0/1497154816 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fadcc06c9c0 0x7fadcc06ee80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fadf00b04e0 tx=0x7fadd8009040 comp rx=0 tx=0).stop 2024-04-26T20:56:59.338 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-04-26T20:56:59.338 DEBUG:teuthology.parallel:result is None 2024-04-26T20:56:59.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.336+0000 7fadf6a00640 1 -- 172.21.15.28:0/1497154816 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadf00b61c0 msgr2=0x7fadf015ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.336+0000 7fadf6a00640 1 --2- 172.21.15.28:0/1497154816 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadf00b61c0 0x7fadf015ef10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fade8009f70 tx=0x7fade8004a40 comp rx=0 tx=0).stop 2024-04-26T20:56:59.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.336+0000 7fadf6a00640 1 -- 172.21.15.28:0/1497154816 shutdown_connections 2024-04-26T20:56:59.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.336+0000 7fadf6a00640 1 --2- 172.21.15.28:0/1497154816 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fadcc06c9c0 0x7fadcc06ee80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.336+0000 7fadf6a00640 1 --2- 172.21.15.28:0/1497154816 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fadf0163c00 0x7fadf00731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.336+0000 7fadf6a00640 1 --2- 172.21.15.28:0/1497154816 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fadf01612d0 0x7fadf015d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.336+0000 7fadf6a00640 1 --2- 172.21.15.28:0/1497154816 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fadf00b61c0 0x7fadf015ef10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.336+0000 7fadf6a00640 1 -- 172.21.15.28:0/1497154816 >> 172.21.15.28:0/1497154816 conn(0x7fadf006e220 msgr2=0x7fadf00cf810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:59.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.336+0000 7fadf6a00640 1 -- 172.21.15.28:0/1497154816 shutdown_connections 2024-04-26T20:56:59.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.336+0000 7fadf6a00640 1 -- 172.21.15.28:0/1497154816 wait complete. 2024-04-26T20:56:59.340 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.0 2024-04-26T20:56:59.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.339+0000 7fc5ab200640 1 -- 172.21.15.28:0/84208748 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fc5ac069840 con 0x7fc5ac15e250 2024-04-26T20:56:59.341 DEBUG:teuthology.parallel:result is None 2024-04-26T20:56:59.341 INFO:teuthology.orchestra.run.smithi028.stdout:55834574852 2024-04-26T20:56:59.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.340+0000 7fc5a2000640 1 -- 172.21.15.28:0/84208748 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc5a407a5c0 con 0x7fc5ac15e250 2024-04-26T20:56:59.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.342+0000 7fc5ab200640 1 -- 172.21.15.28:0/84208748 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc58806c9f0 msgr2=0x7fc58806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.342+0000 7fc5ab200640 1 --2- 172.21.15.28:0/84208748 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc58806c9f0 0x7fc58806eeb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac1634c0 tx=0x7fc594001250 comp rx=0 tx=0).stop 2024-04-26T20:56:59.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.342+0000 7fc5ab200640 1 -- 172.21.15.28:0/84208748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5ac15e250 msgr2=0x7fc5ac1605e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.342+0000 7fc5ab200640 1 --2- 172.21.15.28:0/84208748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5ac15e250 0x7fc5ac1605e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc59000c1e0 tx=0x7fc5a402d070 comp rx=0 tx=0).stop 2024-04-26T20:56:59.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.343+0000 7fc5ab200640 1 -- 172.21.15.28:0/84208748 shutdown_connections 2024-04-26T20:56:59.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.343+0000 7fc5ab200640 1 --2- 172.21.15.28:0/84208748 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc58806c9f0 0x7fc58806eeb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.343+0000 7fc5ab200640 1 --2- 172.21.15.28:0/84208748 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5ac162be0 0x7fc5ac152c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.343+0000 7fc5ab200640 1 --2- 172.21.15.28:0/84208748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5ac15e250 0x7fc5ac1605e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.343+0000 7fc5ab200640 1 --2- 172.21.15.28:0/84208748 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc590004800 0x7fc5ac159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.343+0000 7fc5ab200640 1 -- 172.21.15.28:0/84208748 >> 172.21.15.28:0/84208748 conn(0x7fc5ac06c980 msgr2=0x7fc5ac166e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:59.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.343+0000 7fc5ab200640 1 -- 172.21.15.28:0/84208748 shutdown_connections 2024-04-26T20:56:59.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.343+0000 7fc5ab200640 1 -- 172.21.15.28:0/84208748 wait complete. 2024-04-26T20:56:59.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.352+0000 7f9ff4e00640 1 -- 172.21.15.28:0/625825263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9ff00d14e0 con 0x7f9ff00b43c0 2024-04-26T20:56:59.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.352+0000 7f9fe6000640 1 -- 172.21.15.28:0/625825263 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9ff00d14e0 con 0x7f9ff00b43c0 2024-04-26T20:56:59.353 INFO:teuthology.orchestra.run.smithi028.stdout:55834574853 2024-04-26T20:56:59.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.355+0000 7f9ff4e00640 1 -- 172.21.15.28:0/625825263 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9fd006c940 msgr2=0x7f9fd006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.355+0000 7f9ff4e00640 1 --2- 172.21.15.28:0/625825263 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9fd006c940 0x7f9fd006ee00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc006fd0 tx=0x7f9fdc008040 comp rx=0 tx=0).stop 2024-04-26T20:56:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.355+0000 7f9ff4e00640 1 -- 172.21.15.28:0/625825263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ff00b43c0 msgr2=0x7f9ff00c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.355+0000 7f9ff4e00640 1 --2- 172.21.15.28:0/625825263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ff00b43c0 0x7f9ff00c8740 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0000c00 tx=0x7f9fe00046c0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.355+0000 7f9ff4e00640 1 -- 172.21.15.28:0/625825263 shutdown_connections 2024-04-26T20:56:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.355+0000 7f9ff4e00640 1 --2- 172.21.15.28:0/625825263 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9fd006c940 0x7f9fd006ee00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.355+0000 7f9ff4e00640 1 --2- 172.21.15.28:0/625825263 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9ff01607e0 0x7f9ff015f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.355+0000 7f9ff4e00640 1 --2- 172.21.15.28:0/625825263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9ff00b6b10 0x7f9ff00c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.355+0000 7f9ff4e00640 1 --2- 172.21.15.28:0/625825263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ff00b43c0 0x7f9ff00c8740 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9fe0000c00 tx=0x7f9fe00046c0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.355+0000 7f9ff4e00640 1 -- 172.21.15.28:0/625825263 >> 172.21.15.28:0/625825263 conn(0x7f9ff00d1b10 msgr2=0x7f9ff00ba070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.356+0000 7f9ff4e00640 1 -- 172.21.15.28:0/625825263 shutdown_connections 2024-04-26T20:56:59.358 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.4 2024-04-26T20:56:59.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.356+0000 7f9ff4e00640 1 -- 172.21.15.28:0/625825263 wait complete. 2024-04-26T20:56:59.358 DEBUG:teuthology.parallel:result is None 2024-04-26T20:56:59.361 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.5 2024-04-26T20:56:59.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.360+0000 7fa52a800640 1 -- 172.21.15.28:0/3583618702 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa5240d2d70 con 0x7fa52415f5e0 2024-04-26T20:56:59.361 DEBUG:teuthology.parallel:result is None 2024-04-26T20:56:59.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.361+0000 7fa51b400640 1 -- 172.21.15.28:0/3583618702 <== mon.1 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa51c010070 con 0x7fa52415f5e0 2024-04-26T20:56:59.362 INFO:teuthology.orchestra.run.smithi028.stdout:55834574853 2024-04-26T20:56:59.363 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-04-26T20:56:59.363 DEBUG:teuthology.parallel:result is None 2024-04-26T20:56:59.365 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.6 2024-04-26T20:56:59.366 DEBUG:teuthology.parallel:result is None 2024-04-26T20:56:59.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.363+0000 7fa52a800640 1 -- 172.21.15.28:0/3583618702 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa50406c8f0 msgr2=0x7fa50406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.363+0000 7fa52a800640 1 --2- 172.21.15.28:0/3583618702 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa50406c8f0 0x7fa50406edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa510034000 tx=0x7fa51002e000 comp rx=0 tx=0).stop 2024-04-26T20:56:59.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.363+0000 7fa52a800640 1 -- 172.21.15.28:0/3583618702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa52415f5e0 msgr2=0x7fa5240ce610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.363+0000 7fa52a800640 1 --2- 172.21.15.28:0/3583618702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa52415f5e0 0x7fa5240ce610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa51c009fd0 tx=0x7fa51c007f60 comp rx=0 tx=0).stop 2024-04-26T20:56:59.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.363+0000 7fa52a800640 1 -- 172.21.15.28:0/3583618702 shutdown_connections 2024-04-26T20:56:59.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.363+0000 7fa52a800640 1 --2- 172.21.15.28:0/3583618702 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa50406c8f0 0x7fa50406edb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.363+0000 7fa52a800640 1 --2- 172.21.15.28:0/3583618702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa52415f5e0 0x7fa5240ce610 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.363+0000 7fa52a800640 1 --2- 172.21.15.28:0/3583618702 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa52415f200 0x7fa52415dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.363+0000 7fa52a800640 1 --2- 172.21.15.28:0/3583618702 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa5240cfe70 0x7fa5241f75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.363+0000 7fa52a800640 1 -- 172.21.15.28:0/3583618702 >> 172.21.15.28:0/3583618702 conn(0x7fa5240b6bb0 msgr2=0x7fa5240b8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:59.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.363+0000 7fa52a800640 1 -- 172.21.15.28:0/3583618702 shutdown_connections 2024-04-26T20:56:59.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.363+0000 7fa52a800640 1 -- 172.21.15.28:0/3583618702 wait complete. 2024-04-26T20:56:59.376 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2024-04-26T20:56:59.376 DEBUG:teuthology.parallel:result is None 2024-04-26T20:56:59.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2024-04-26T20:56:59.382 DEBUG:teuthology.parallel:result is None 2024-04-26T20:56:59.382 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T20:56:59.382 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:56:59.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.618+0000 7f54bf200640 1 Processor -- start 2024-04-26T20:56:59.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.618+0000 7f54bf200640 1 -- start start 2024-04-26T20:56:59.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.619+0000 7f54bf200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f54b815c310 0x7f54b815c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:59.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.619+0000 7f54bf200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f54b806df00 0x7f54b815cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:59.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.619+0000 7f54bf200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54b8072e70 con 0x7f54b8072920 2024-04-26T20:56:59.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.619+0000 7f54bf200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f54b80739f0 con 0x7f54b806df00 2024-04-26T20:56:59.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.619+0000 7f54bf200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f54b815d170 con 0x7f54b815c310 2024-04-26T20:56:59.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.620+0000 7f54bd400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f54b8072920 0x7f54b815d840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57836/0 (socket says 172.21.15.28:57836) 2024-04-26T20:56:59.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.620+0000 7f54bd400640 1 -- 172.21.15.28:0/603173909 learned_addr learned my addr 172.21.15.28:0/603173909 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:56:59.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.620+0000 7f54be800640 1 --2- 172.21.15.28:0/603173909 >> v2:172.21.15.28:3300/0 conn(0x7f54b806df00 0x7f54b815cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:59.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.620+0000 7f54bde00640 1 --2- 172.21.15.28:0/603173909 >> v2:172.21.15.103:3300/0 conn(0x7f54b815c310 0x7f54b815c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:59.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.620+0000 7f54bca00640 1 -- 172.21.15.28:0/603173909 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3221008749 0 0) 0x7f54b8072e70 con 0x7f54b8072920 2024-04-26T20:56:59.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.620+0000 7f54bca00640 1 -- 172.21.15.28:0/603173909 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54a0003520 con 0x7f54b8072920 2024-04-26T20:56:59.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.620+0000 7f54be800640 1 -- 172.21.15.28:0/603173909 >> v2:172.21.15.103:3300/0 conn(0x7f54b815c310 msgr2=0x7f54b815c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:56:59.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.620+0000 7f54be800640 1 --2- 172.21.15.28:0/603173909 >> v2:172.21.15.103:3300/0 conn(0x7f54b815c310 0x7f54b815c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.620+0000 7f54be800640 1 -- 172.21.15.28:0/603173909 >> v1:172.21.15.28:6789/0 conn(0x7f54b8072920 legacy=0x7f54b815d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.620+0000 7f54be800640 1 -- 172.21.15.28:0/603173909 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b8073150 con 0x7f54b806df00 2024-04-26T20:56:59.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.620+0000 7f54bde00640 1 --2- 172.21.15.28:0/603173909 >> v2:172.21.15.103:3300/0 conn(0x7f54b815c310 0x7f54b815c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:56:59.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.622+0000 7f54be800640 1 --2- 172.21.15.28:0/603173909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b806df00 0x7f54b815cc30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f54b80745b0 tx=0x7f54b402c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20c6933516f8dce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:59.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.622+0000 7f54bca00640 1 -- 172.21.15.28:0/603173909 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4007740 con 0x7f54b806df00 2024-04-26T20:56:59.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.622+0000 7f54bca00640 1 -- 172.21.15.28:0/603173909 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b40078e0 con 0x7f54b806df00 2024-04-26T20:56:59.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.622+0000 7f54bca00640 1 -- 172.21.15.28:0/603173909 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4007cb0 con 0x7f54b806df00 2024-04-26T20:56:59.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.622+0000 7f54bf200640 1 -- 172.21.15.28:0/603173909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b806df00 msgr2=0x7f54b815cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.622+0000 7f54bf200640 1 --2- 172.21.15.28:0/603173909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b806df00 0x7f54b815cc30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f54b80745b0 tx=0x7f54b402c800 comp rx=0 tx=0).stop 2024-04-26T20:56:59.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.623+0000 7f54bf200640 1 -- 172.21.15.28:0/603173909 shutdown_connections 2024-04-26T20:56:59.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.623+0000 7f54bf200640 1 --2- 172.21.15.28:0/603173909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b806df00 0x7f54b815cc30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.623+0000 7f54bf200640 1 --2- 172.21.15.28:0/603173909 >> v2:172.21.15.103:3300/0 conn(0x7f54b815c310 0x7f54b815c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.623+0000 7f54bf200640 1 -- 172.21.15.28:0/603173909 >> 172.21.15.28:0/603173909 conn(0x7f54b80b2720 msgr2=0x7f54b80b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:59.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.623+0000 7f54bf200640 1 -- 172.21.15.28:0/603173909 shutdown_connections 2024-04-26T20:56:59.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.623+0000 7f54bf200640 1 -- 172.21.15.28:0/603173909 wait complete. 2024-04-26T20:56:59.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.624+0000 7f54bf200640 1 Processor -- start 2024-04-26T20:56:59.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.624+0000 7f54bf200640 1 -- start start 2024-04-26T20:56:59.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.625+0000 7f54bf200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b806df00 0x7f54b81425b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:59.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.625+0000 7f54bf200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54b8072920 0x7f54b815a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:59.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.625+0000 7f54bf200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f54b815c310 0x7f54b81561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:59.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.625+0000 7f54bf200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f54b806fc90 con 0x7f54b806df00 2024-04-26T20:56:59.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.625+0000 7f54bf200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f54b8073710 con 0x7f54b8072920 2024-04-26T20:56:59.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.625+0000 7f54bf200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f54b806fe10 con 0x7f54b815c310 2024-04-26T20:56:59.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.626+0000 7f54bde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b806df00 0x7f54b81425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:59.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.626+0000 7f54bde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b806df00 0x7f54b81425b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42370/0 (socket says 172.21.15.28:42370) 2024-04-26T20:56:59.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.626+0000 7f54bde00640 1 -- 172.21.15.28:0/2825982664 learned_addr learned my addr 172.21.15.28:0/2825982664 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:56:59.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.626+0000 7f54bd400640 1 --2- 172.21.15.28:0/2825982664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54b8072920 0x7f54b815a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:59.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.626+0000 7f54be800640 1 --2- 172.21.15.28:0/2825982664 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f54b815c310 0x7f54b81561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:59.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.626+0000 7f54bde00640 1 -- 172.21.15.28:0/2825982664 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f54b815c310 msgr2=0x7f54b81561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.626+0000 7f54bde00640 1 --2- 172.21.15.28:0/2825982664 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f54b815c310 0x7f54b81561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.626+0000 7f54bde00640 1 -- 172.21.15.28:0/2825982664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54b8072920 msgr2=0x7f54b815a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.626+0000 7f54bde00640 1 --2- 172.21.15.28:0/2825982664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54b8072920 0x7f54b815a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.626+0000 7f54bde00640 1 -- 172.21.15.28:0/2825982664 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b815b0a0 con 0x7f54b806df00 2024-04-26T20:56:59.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.626+0000 7f54be800640 1 --2- 172.21.15.28:0/2825982664 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f54b815c310 0x7f54b81561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:56:59.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.626+0000 7f54bde00640 1 --2- 172.21.15.28:0/2825982664 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b806df00 0x7f54b81425b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00c7f0 tx=0x7f54ac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:59.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.626+0000 7f54b2000640 1 -- 172.21.15.28:0/2825982664 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac015070 con 0x7f54b806df00 2024-04-26T20:56:59.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.627+0000 7f54b2000640 1 -- 172.21.15.28:0/2825982664 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54ac007820 con 0x7f54b806df00 2024-04-26T20:56:59.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.627+0000 7f54b2000640 1 -- 172.21.15.28:0/2825982664 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac007bd0 con 0x7f54b806df00 2024-04-26T20:56:59.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.627+0000 7f54bf200640 1 -- 172.21.15.28:0/2825982664 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b815b380 con 0x7f54b806df00 2024-04-26T20:56:59.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.627+0000 7f54bf200640 1 -- 172.21.15.28:0/2825982664 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b8142af0 con 0x7f54b806df00 2024-04-26T20:56:59.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.628+0000 7f54bf200640 1 -- 172.21.15.28:0/2825982664 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54b806a360 con 0x7f54b806df00 2024-04-26T20:56:59.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.629+0000 7f54b2000640 1 -- 172.21.15.28:0/2825982664 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f54ac018070 con 0x7f54b806df00 2024-04-26T20:56:59.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.630+0000 7f54b2000640 1 --2- 172.21.15.28:0/2825982664 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f549c06c920 0x7f549c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:56:59.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.630+0000 7f54b2000640 1 -- 172.21.15.28:0/2825982664 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f54ac08bc30 con 0x7f54b806df00 2024-04-26T20:56:59.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.630+0000 7f54bd400640 1 --2- 172.21.15.28:0/2825982664 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f549c06c920 0x7f549c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:56:59.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.631+0000 7f54bd400640 1 --2- 172.21.15.28:0/2825982664 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f549c06c920 0x7f549c06ede0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f54b8153d90 tx=0x7f54a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:56:59.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.634+0000 7f54b2000640 1 -- 172.21.15.28:0/2825982664 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f54ac05a050 con 0x7f54b806df00 2024-04-26T20:56:59.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.831+0000 7f54bf200640 1 -- 172.21.15.28:0/2825982664 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f54b80ce3d0 con 0x7f549c06c920 2024-04-26T20:56:59.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.842+0000 7f54b2000640 1 -- 172.21.15.28:0/2825982664 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f54a8001d00 con 0x7f549c06c920 2024-04-26T20:56:59.846 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:56:59.847 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T20:56:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.847+0000 7f54bf200640 1 -- 172.21.15.28:0/2825982664 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f549c06c920 msgr2=0x7f549c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.847+0000 7f54bf200640 1 --2- 172.21.15.28:0/2825982664 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f549c06c920 0x7f549c06ede0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f54b8153d90 tx=0x7f54a8009040 comp rx=0 tx=0).stop 2024-04-26T20:56:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.847+0000 7f54bf200640 1 -- 172.21.15.28:0/2825982664 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b806df00 msgr2=0x7f54b81425b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:56:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.847+0000 7f54bf200640 1 --2- 172.21.15.28:0/2825982664 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b806df00 0x7f54b81425b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00c7f0 tx=0x7f54ac00ccb0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.847+0000 7f54bf200640 1 -- 172.21.15.28:0/2825982664 shutdown_connections 2024-04-26T20:56:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.847+0000 7f54bf200640 1 --2- 172.21.15.28:0/2825982664 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f549c06c920 0x7f549c06ede0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.847+0000 7f54bf200640 1 --2- 172.21.15.28:0/2825982664 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f54b815c310 0x7f54b81561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.847+0000 7f54bf200640 1 --2- 172.21.15.28:0/2825982664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54b8072920 0x7f54b815a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.847+0000 7f54bf200640 1 --2- 172.21.15.28:0/2825982664 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54b806df00 0x7f54b81425b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:56:59.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.847+0000 7f54bf200640 1 -- 172.21.15.28:0/2825982664 >> 172.21.15.28:0/2825982664 conn(0x7f54b80b2720 msgr2=0x7f54b80ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:56:59.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.847+0000 7f54bf200640 1 -- 172.21.15.28:0/2825982664 shutdown_connections 2024-04-26T20:56:59.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:56:59.847+0000 7f54bf200640 1 -- 172.21.15.28:0/2825982664 wait complete. 2024-04-26T20:56:59.866 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-26T20:56:58.706460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235752,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738968,"statfs":{"total":773094113280,"available":772852703232,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.005812"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153718+0000","last_change":"2024-04-26T20:56:51.731207+0000","last_active":"2024-04-26T20:56:53.153718+0000","last_peered":"2024-04-26T20:56:53.153718+0000","last_clean":"2024-04-26T20:56:53.153718+0000","last_became_active":"2024-04-26T20:56:51.730930+0000","last_became_peered":"2024-04-26T20:56:51.730930+0000","last_unstale":"2024-04-26T20:56:53.153718+0000","last_undegraded":"2024-04-26T20:56:53.153718+0000","last_fullsized":"2024-04-26T20:56:53.153718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:06.270681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.714391+0000","last_change":"2024-04-26T20:56:51.719125+0000","last_active":"2024-04-26T20:56:52.714391+0000","last_peered":"2024-04-26T20:56:52.714391+0000","last_clean":"2024-04-26T20:56:52.714391+0000","last_became_active":"2024-04-26T20:56:51.717685+0000","last_became_peered":"2024-04-26T20:56:51.717685+0000","last_unstale":"2024-04-26T20:56:52.714391+0000","last_undegraded":"2024-04-26T20:56:52.714391+0000","last_fullsized":"2024-04-26T20:56:52.714391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.428432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144729+0000","last_change":"2024-04-26T20:56:51.717024+0000","last_active":"2024-04-26T20:56:53.144729+0000","last_peered":"2024-04-26T20:56:53.144729+0000","last_clean":"2024-04-26T20:56:53.144729+0000","last_became_active":"2024-04-26T20:56:51.716486+0000","last_became_peered":"2024-04-26T20:56:51.716486+0000","last_unstale":"2024-04-26T20:56:53.144729+0000","last_undegraded":"2024-04-26T20:56:53.144729+0000","last_fullsized":"2024-04-26T20:56:53.144729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:01.617412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716816+0000","last_change":"2024-04-26T20:56:51.716770+0000","last_active":"2024-04-26T20:56:52.716816+0000","last_peered":"2024-04-26T20:56:52.716816+0000","last_clean":"2024-04-26T20:56:52.716816+0000","last_became_active":"2024-04-26T20:56:51.716443+0000","last_became_peered":"2024-04-26T20:56:51.716443+0000","last_unstale":"2024-04-26T20:56:52.716816+0000","last_undegraded":"2024-04-26T20:56:52.716816+0000","last_fullsized":"2024-04-26T20:56:52.716816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:08.812637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153115+0000","last_change":"2024-04-26T20:56:51.735130+0000","last_active":"2024-04-26T20:56:53.153115+0000","last_peered":"2024-04-26T20:56:53.153115+0000","last_clean":"2024-04-26T20:56:53.153115+0000","last_became_active":"2024-04-26T20:56:51.734775+0000","last_became_peered":"2024-04-26T20:56:51.734775+0000","last_unstale":"2024-04-26T20:56:53.153115+0000","last_undegraded":"2024-04-26T20:56:53.153115+0000","last_fullsized":"2024-04-26T20:56:53.153115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:08.812637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716491+0000","last_change":"2024-04-26T20:56:51.719621+0000","last_active":"2024-04-26T20:56:52.716491+0000","last_peered":"2024-04-26T20:56:52.716491+0000","last_clean":"2024-04-26T20:56:52.716491+0000","last_became_active":"2024-04-26T20:56:51.719312+0000","last_became_peered":"2024-04-26T20:56:51.719312+0000","last_unstale":"2024-04-26T20:56:52.716491+0000","last_undegraded":"2024-04-26T20:56:52.716491+0000","last_fullsized":"2024-04-26T20:56:52.716491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:23.268496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.152560+0000","last_change":"2024-04-26T20:56:51.730495+0000","last_active":"2024-04-26T20:56:53.152560+0000","last_peered":"2024-04-26T20:56:53.152560+0000","last_clean":"2024-04-26T20:56:53.152560+0000","last_became_active":"2024-04-26T20:56:51.729796+0000","last_became_peered":"2024-04-26T20:56:51.729796+0000","last_unstale":"2024-04-26T20:56:53.152560+0000","last_undegraded":"2024-04-26T20:56:53.152560+0000","last_fullsized":"2024-04-26T20:56:53.152560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:52.912874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225874+0000","last_change":"2024-04-26T20:56:51.729824+0000","last_active":"2024-04-26T20:56:53.225874+0000","last_peered":"2024-04-26T20:56:53.225874+0000","last_clean":"2024-04-26T20:56:53.225874+0000","last_became_active":"2024-04-26T20:56:51.729546+0000","last_became_peered":"2024-04-26T20:56:51.729546+0000","last_unstale":"2024-04-26T20:56:53.225874+0000","last_undegraded":"2024-04-26T20:56:53.225874+0000","last_fullsized":"2024-04-26T20:56:53.225874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:18.631055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153639+0000","last_change":"2024-04-26T20:56:51.732556+0000","last_active":"2024-04-26T20:56:53.153639+0000","last_peered":"2024-04-26T20:56:53.153639+0000","last_clean":"2024-04-26T20:56:53.153639+0000","last_became_active":"2024-04-26T20:56:51.732298+0000","last_became_peered":"2024-04-26T20:56:51.732298+0000","last_unstale":"2024-04-26T20:56:53.153639+0000","last_undegraded":"2024-04-26T20:56:53.153639+0000","last_fullsized":"2024-04-26T20:56:53.153639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:07.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730718+0000","last_change":"2024-04-26T20:56:51.724747+0000","last_active":"2024-04-26T20:56:52.730718+0000","last_peered":"2024-04-26T20:56:52.730718+0000","last_clean":"2024-04-26T20:56:52.730718+0000","last_became_active":"2024-04-26T20:56:51.724452+0000","last_became_peered":"2024-04-26T20:56:51.724452+0000","last_unstale":"2024-04-26T20:56:52.730718+0000","last_undegraded":"2024-04-26T20:56:52.730718+0000","last_fullsized":"2024-04-26T20:56:52.730718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.428432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730341+0000","last_change":"2024-04-26T20:56:51.729732+0000","last_active":"2024-04-26T20:56:52.730341+0000","last_peered":"2024-04-26T20:56:52.730341+0000","last_clean":"2024-04-26T20:56:52.730341+0000","last_became_active":"2024-04-26T20:56:51.729464+0000","last_became_peered":"2024-04-26T20:56:51.729464+0000","last_unstale":"2024-04-26T20:56:52.730341+0000","last_undegraded":"2024-04-26T20:56:52.730341+0000","last_fullsized":"2024-04-26T20:56:52.730341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:37.778325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.715425+0000","last_change":"2024-04-26T20:56:51.723870+0000","last_active":"2024-04-26T20:56:52.715425+0000","last_peered":"2024-04-26T20:56:52.715425+0000","last_clean":"2024-04-26T20:56:52.715425+0000","last_became_active":"2024-04-26T20:56:51.722037+0000","last_became_peered":"2024-04-26T20:56:51.722037+0000","last_unstale":"2024-04-26T20:56:52.715425+0000","last_undegraded":"2024-04-26T20:56:52.715425+0000","last_fullsized":"2024-04-26T20:56:52.715425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:42.676745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226737+0000","last_change":"2024-04-26T20:56:51.719947+0000","last_active":"2024-04-26T20:56:53.226737+0000","last_peered":"2024-04-26T20:56:53.226737+0000","last_clean":"2024-04-26T20:56:53.226737+0000","last_became_active":"2024-04-26T20:56:51.719565+0000","last_became_peered":"2024-04-26T20:56:51.719565+0000","last_unstale":"2024-04-26T20:56:53.226737+0000","last_undegraded":"2024-04-26T20:56:53.226737+0000","last_fullsized":"2024-04-26T20:56:53.226737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:36.354765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144673+0000","last_change":"2024-04-26T20:56:51.725826+0000","last_active":"2024-04-26T20:56:53.144673+0000","last_peered":"2024-04-26T20:56:53.144673+0000","last_clean":"2024-04-26T20:56:53.144673+0000","last_became_active":"2024-04-26T20:56:51.723315+0000","last_became_peered":"2024-04-26T20:56:51.723315+0000","last_unstale":"2024-04-26T20:56:53.144673+0000","last_undegraded":"2024-04-26T20:56:53.144673+0000","last_fullsized":"2024-04-26T20:56:53.144673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:38:55.851891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713041+0000","last_change":"2024-04-26T20:56:51.704000+0000","last_active":"2024-04-26T20:56:52.713041+0000","last_peered":"2024-04-26T20:56:52.713041+0000","last_clean":"2024-04-26T20:56:52.713041+0000","last_became_active":"2024-04-26T20:56:51.703007+0000","last_became_peered":"2024-04-26T20:56:51.703007+0000","last_unstale":"2024-04-26T20:56:52.713041+0000","last_undegraded":"2024-04-26T20:56:52.713041+0000","last_fullsized":"2024-04-26T20:56:52.713041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:36.354765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716554+0000","last_change":"2024-04-26T20:56:51.724622+0000","last_active":"2024-04-26T20:56:52.716554+0000","last_peered":"2024-04-26T20:56:52.716554+0000","last_clean":"2024-04-26T20:56:52.716554+0000","last_became_active":"2024-04-26T20:56:51.723198+0000","last_became_peered":"2024-04-26T20:56:51.723198+0000","last_unstale":"2024-04-26T20:56:52.716554+0000","last_undegraded":"2024-04-26T20:56:52.716554+0000","last_fullsized":"2024-04-26T20:56:52.716554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:52.912874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.718883+0000","last_change":"2024-04-26T20:56:51.736813+0000","last_active":"2024-04-26T20:56:52.718883+0000","last_peered":"2024-04-26T20:56:52.718883+0000","last_clean":"2024-04-26T20:56:52.718883+0000","last_became_active":"2024-04-26T20:56:51.736532+0000","last_became_peered":"2024-04-26T20:56:51.736532+0000","last_unstale":"2024-04-26T20:56:52.718883+0000","last_undegraded":"2024-04-26T20:56:52.718883+0000","last_fullsized":"2024-04-26T20:56:52.718883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:08.679353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225324+0000","last_change":"2024-04-26T20:56:51.726234+0000","last_active":"2024-04-26T20:56:53.225324+0000","last_peered":"2024-04-26T20:56:53.225324+0000","last_clean":"2024-04-26T20:56:53.225324+0000","last_became_active":"2024-04-26T20:56:51.723289+0000","last_became_peered":"2024-04-26T20:56:51.723289+0000","last_unstale":"2024-04-26T20:56:53.225324+0000","last_undegraded":"2024-04-26T20:56:53.225324+0000","last_fullsized":"2024-04-26T20:56:53.225324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:12.708963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153878+0000","last_change":"2024-04-26T20:56:51.735058+0000","last_active":"2024-04-26T20:56:53.153878+0000","last_peered":"2024-04-26T20:56:53.153878+0000","last_clean":"2024-04-26T20:56:53.153878+0000","last_became_active":"2024-04-26T20:56:51.734516+0000","last_became_peered":"2024-04-26T20:56:51.734516+0000","last_unstale":"2024-04-26T20:56:53.153878+0000","last_undegraded":"2024-04-26T20:56:53.153878+0000","last_fullsized":"2024-04-26T20:56:53.153878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:33.950624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730309+0000","last_change":"2024-04-26T20:56:51.715023+0000","last_active":"2024-04-26T20:56:52.730309+0000","last_peered":"2024-04-26T20:56:52.730309+0000","last_clean":"2024-04-26T20:56:52.730309+0000","last_became_active":"2024-04-26T20:56:51.714250+0000","last_became_peered":"2024-04-26T20:56:51.714250+0000","last_unstale":"2024-04-26T20:56:52.730309+0000","last_undegraded":"2024-04-26T20:56:52.730309+0000","last_fullsized":"2024-04-26T20:56:52.730309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:24.679612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144998+0000","last_change":"2024-04-26T20:56:51.737492+0000","last_active":"2024-04-26T20:56:53.144998+0000","last_peered":"2024-04-26T20:56:53.144998+0000","last_clean":"2024-04-26T20:56:53.144998+0000","last_became_active":"2024-04-26T20:56:51.736774+0000","last_became_peered":"2024-04-26T20:56:51.736774+0000","last_unstale":"2024-04-26T20:56:53.144998+0000","last_undegraded":"2024-04-26T20:56:53.144998+0000","last_fullsized":"2024-04-26T20:56:53.144998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:36.354765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716674+0000","last_change":"2024-04-26T20:56:51.731749+0000","last_active":"2024-04-26T20:56:52.716674+0000","last_peered":"2024-04-26T20:56:52.716674+0000","last_clean":"2024-04-26T20:56:52.716674+0000","last_became_active":"2024-04-26T20:56:51.731460+0000","last_became_peered":"2024-04-26T20:56:51.731460+0000","last_unstale":"2024-04-26T20:56:52.716674+0000","last_undegraded":"2024-04-26T20:56:52.716674+0000","last_fullsized":"2024-04-26T20:56:52.716674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:47.662176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226704+0000","last_change":"2024-04-26T20:56:51.721856+0000","last_active":"2024-04-26T20:56:53.226704+0000","last_peered":"2024-04-26T20:56:53.226704+0000","last_clean":"2024-04-26T20:56:53.226704+0000","last_became_active":"2024-04-26T20:56:51.721531+0000","last_became_peered":"2024-04-26T20:56:51.721531+0000","last_unstale":"2024-04-26T20:56:53.226704+0000","last_undegraded":"2024-04-26T20:56:53.226704+0000","last_fullsized":"2024-04-26T20:56:53.226704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:53.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716438+0000","last_change":"2024-04-26T20:56:51.718492+0000","last_active":"2024-04-26T20:56:52.716438+0000","last_peered":"2024-04-26T20:56:52.716438+0000","last_clean":"2024-04-26T20:56:52.716438+0000","last_became_active":"2024-04-26T20:56:51.718127+0000","last_became_peered":"2024-04-26T20:56:51.718127+0000","last_unstale":"2024-04-26T20:56:52.716438+0000","last_undegraded":"2024-04-26T20:56:52.716438+0000","last_fullsized":"2024-04-26T20:56:52.716438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.428432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.227580+0000","last_change":"2024-04-26T20:56:51.722240+0000","last_active":"2024-04-26T20:56:53.227580+0000","last_peered":"2024-04-26T20:56:53.227580+0000","last_clean":"2024-04-26T20:56:53.227580+0000","last_became_active":"2024-04-26T20:56:51.721601+0000","last_became_peered":"2024-04-26T20:56:51.721601+0000","last_unstale":"2024-04-26T20:56:53.227580+0000","last_undegraded":"2024-04-26T20:56:53.227580+0000","last_fullsized":"2024-04-26T20:56:53.227580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:52.912874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730730+0000","last_change":"2024-04-26T20:56:51.722608+0000","last_active":"2024-04-26T20:56:52.730730+0000","last_peered":"2024-04-26T20:56:52.730730+0000","last_clean":"2024-04-26T20:56:52.730730+0000","last_became_active":"2024-04-26T20:56:51.722321+0000","last_became_peered":"2024-04-26T20:56:51.722321+0000","last_unstale":"2024-04-26T20:56:52.730730+0000","last_undegraded":"2024-04-26T20:56:52.730730+0000","last_fullsized":"2024-04-26T20:56:52.730730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:23.268496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713738+0000","last_change":"2024-04-26T20:56:51.715230+0000","last_active":"2024-04-26T20:56:52.713738+0000","last_peered":"2024-04-26T20:56:52.713738+0000","last_clean":"2024-04-26T20:56:52.713738+0000","last_became_active":"2024-04-26T20:56:51.714911+0000","last_became_peered":"2024-04-26T20:56:51.714911+0000","last_unstale":"2024-04-26T20:56:52.713738+0000","last_undegraded":"2024-04-26T20:56:52.713738+0000","last_fullsized":"2024-04-26T20:56:52.713738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:37.778325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.227039+0000","last_change":"2024-04-26T20:56:51.712010+0000","last_active":"2024-04-26T20:56:53.227039+0000","last_peered":"2024-04-26T20:56:53.227039+0000","last_clean":"2024-04-26T20:56:53.227039+0000","last_became_active":"2024-04-26T20:56:51.711583+0000","last_became_peered":"2024-04-26T20:56:51.711583+0000","last_unstale":"2024-04-26T20:56:53.227039+0000","last_undegraded":"2024-04-26T20:56:53.227039+0000","last_fullsized":"2024-04-26T20:56:53.227039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.428432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.145333+0000","last_change":"2024-04-26T20:56:51.722575+0000","last_active":"2024-04-26T20:56:53.145333+0000","last_peered":"2024-04-26T20:56:53.145333+0000","last_clean":"2024-04-26T20:56:53.145333+0000","last_became_active":"2024-04-26T20:56:51.722274+0000","last_became_peered":"2024-04-26T20:56:51.722274+0000","last_unstale":"2024-04-26T20:56:53.145333+0000","last_undegraded":"2024-04-26T20:56:53.145333+0000","last_fullsized":"2024-04-26T20:56:53.145333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.428432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716895+0000","last_change":"2024-04-26T20:56:51.712244+0000","last_active":"2024-04-26T20:56:52.716895+0000","last_peered":"2024-04-26T20:56:52.716895+0000","last_clean":"2024-04-26T20:56:52.716895+0000","last_became_active":"2024-04-26T20:56:51.711957+0000","last_became_peered":"2024-04-26T20:56:51.711957+0000","last_unstale":"2024-04-26T20:56:52.716895+0000","last_undegraded":"2024-04-26T20:56:52.716895+0000","last_fullsized":"2024-04-26T20:56:52.716895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:22.643252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225198+0000","last_change":"2024-04-26T20:56:51.721797+0000","last_active":"2024-04-26T20:56:53.225198+0000","last_peered":"2024-04-26T20:56:53.225198+0000","last_clean":"2024-04-26T20:56:53.225198+0000","last_became_active":"2024-04-26T20:56:51.721465+0000","last_became_peered":"2024-04-26T20:56:51.721465+0000","last_unstale":"2024-04-26T20:56:53.225198+0000","last_undegraded":"2024-04-26T20:56:53.225198+0000","last_fullsized":"2024-04-26T20:56:53.225198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:26.791119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.729925+0000","last_change":"2024-04-26T20:56:51.716527+0000","last_active":"2024-04-26T20:56:52.729925+0000","last_peered":"2024-04-26T20:56:52.729925+0000","last_clean":"2024-04-26T20:56:52.729925+0000","last_became_active":"2024-04-26T20:56:51.716103+0000","last_became_peered":"2024-04-26T20:56:51.716103+0000","last_unstale":"2024-04-26T20:56:52.729925+0000","last_undegraded":"2024-04-26T20:56:52.729925+0000","last_fullsized":"2024-04-26T20:56:52.729925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:53.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.714884+0000","last_change":"2024-04-26T20:56:51.713515+0000","last_active":"2024-04-26T20:56:52.714884+0000","last_peered":"2024-04-26T20:56:52.714884+0000","last_clean":"2024-04-26T20:56:52.714884+0000","last_became_active":"2024-04-26T20:56:51.710208+0000","last_became_peered":"2024-04-26T20:56:51.710208+0000","last_unstale":"2024-04-26T20:56:52.714884+0000","last_undegraded":"2024-04-26T20:56:52.714884+0000","last_fullsized":"2024-04-26T20:56:52.714884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:51.540923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226969+0000","last_change":"2024-04-26T20:56:50.692216+0000","last_active":"2024-04-26T20:56:53.226969+0000","last_peered":"2024-04-26T20:56:53.226969+0000","last_clean":"2024-04-26T20:56:53.226969+0000","last_became_active":"2024-04-26T20:56:50.691457+0000","last_became_peered":"2024-04-26T20:56:50.691457+0000","last_unstale":"2024-04-26T20:56:53.226969+0000","last_undegraded":"2024-04-26T20:56:53.226969+0000","last_fullsized":"2024-04-26T20:56:53.226969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:50.477216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.145297+0000","last_change":"2024-04-26T20:56:51.730925+0000","last_active":"2024-04-26T20:56:53.145297+0000","last_peered":"2024-04-26T20:56:53.145297+0000","last_clean":"2024-04-26T20:56:53.145297+0000","last_became_active":"2024-04-26T20:56:51.730497+0000","last_became_peered":"2024-04-26T20:56:51.730497+0000","last_unstale":"2024-04-26T20:56:53.145297+0000","last_undegraded":"2024-04-26T20:56:53.145297+0000","last_fullsized":"2024-04-26T20:56:53.145297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:37.778325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.729866+0000","last_change":"2024-04-26T20:56:50.684303+0000","last_active":"2024-04-26T20:56:52.729866+0000","last_peered":"2024-04-26T20:56:52.729866+0000","last_clean":"2024-04-26T20:56:52.729866+0000","last_became_active":"2024-04-26T20:56:50.684006+0000","last_became_peered":"2024-04-26T20:56:50.684006+0000","last_unstale":"2024-04-26T20:56:52.729866+0000","last_undegraded":"2024-04-26T20:56:52.729866+0000","last_fullsized":"2024-04-26T20:56:52.729866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:50.526509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226111+0000","last_change":"2024-04-26T20:56:50.688365+0000","last_active":"2024-04-26T20:56:53.226111+0000","last_peered":"2024-04-26T20:56:53.226111+0000","last_clean":"2024-04-26T20:56:53.226111+0000","last_became_active":"2024-04-26T20:56:50.688034+0000","last_became_peered":"2024-04-26T20:56:50.688034+0000","last_unstale":"2024-04-26T20:56:53.226111+0000","last_undegraded":"2024-04-26T20:56:53.226111+0000","last_fullsized":"2024-04-26T20:56:53.226111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:14.594594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226190+0000","last_change":"2024-04-26T20:56:51.721954+0000","last_active":"2024-04-26T20:56:53.226190+0000","last_peered":"2024-04-26T20:56:53.226190+0000","last_clean":"2024-04-26T20:56:53.226190+0000","last_became_active":"2024-04-26T20:56:51.721650+0000","last_became_peered":"2024-04-26T20:56:51.721650+0000","last_unstale":"2024-04-26T20:56:53.226190+0000","last_undegraded":"2024-04-26T20:56:53.226190+0000","last_fullsized":"2024-04-26T20:56:53.226190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:01.617412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716559+0000","last_change":"2024-04-26T20:56:50.680262+0000","last_active":"2024-04-26T20:56:52.716559+0000","last_peered":"2024-04-26T20:56:52.716559+0000","last_clean":"2024-04-26T20:56:52.716559+0000","last_became_active":"2024-04-26T20:56:50.675501+0000","last_became_peered":"2024-04-26T20:56:50.675501+0000","last_unstale":"2024-04-26T20:56:52.716559+0000","last_undegraded":"2024-04-26T20:56:52.716559+0000","last_fullsized":"2024-04-26T20:56:52.716559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:31.999414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713009+0000","last_change":"2024-04-26T20:56:51.714659+0000","last_active":"2024-04-26T20:56:52.713009+0000","last_peered":"2024-04-26T20:56:52.713009+0000","last_clean":"2024-04-26T20:56:52.713009+0000","last_became_active":"2024-04-26T20:56:51.714340+0000","last_became_peered":"2024-04-26T20:56:51.714340+0000","last_unstale":"2024-04-26T20:56:52.713009+0000","last_undegraded":"2024-04-26T20:56:52.713009+0000","last_fullsized":"2024-04-26T20:56:52.713009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:53.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730521+0000","last_change":"2024-04-26T20:56:50.689246+0000","last_active":"2024-04-26T20:56:52.730521+0000","last_peered":"2024-04-26T20:56:52.730521+0000","last_clean":"2024-04-26T20:56:52.730521+0000","last_became_active":"2024-04-26T20:56:50.688882+0000","last_became_peered":"2024-04-26T20:56:50.688882+0000","last_unstale":"2024-04-26T20:56:52.730521+0000","last_undegraded":"2024-04-26T20:56:52.730521+0000","last_fullsized":"2024-04-26T20:56:52.730521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:39.328210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.718004+0000","last_change":"2024-04-26T20:56:51.737626+0000","last_active":"2024-04-26T20:56:52.718004+0000","last_peered":"2024-04-26T20:56:52.718004+0000","last_clean":"2024-04-26T20:56:52.718004+0000","last_became_active":"2024-04-26T20:56:51.737273+0000","last_became_peered":"2024-04-26T20:56:51.737273+0000","last_unstale":"2024-04-26T20:56:52.718004+0000","last_undegraded":"2024-04-26T20:56:52.718004+0000","last_fullsized":"2024-04-26T20:56:52.718004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:33.950624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.718371+0000","last_change":"2024-04-26T20:56:50.690573+0000","last_active":"2024-04-26T20:56:52.718371+0000","last_peered":"2024-04-26T20:56:52.718371+0000","last_clean":"2024-04-26T20:56:52.718371+0000","last_became_active":"2024-04-26T20:56:50.690198+0000","last_became_peered":"2024-04-26T20:56:50.690198+0000","last_unstale":"2024-04-26T20:56:52.718371+0000","last_undegraded":"2024-04-26T20:56:52.718371+0000","last_fullsized":"2024-04-26T20:56:52.718371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:25.776704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144843+0000","last_change":"2024-04-26T20:56:51.733136+0000","last_active":"2024-04-26T20:56:53.144843+0000","last_peered":"2024-04-26T20:56:53.144843+0000","last_clean":"2024-04-26T20:56:53.144843+0000","last_became_active":"2024-04-26T20:56:51.732782+0000","last_became_peered":"2024-04-26T20:56:51.732782+0000","last_unstale":"2024-04-26T20:56:53.144843+0000","last_undegraded":"2024-04-26T20:56:53.144843+0000","last_fullsized":"2024-04-26T20:56:53.144843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:16.275518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713411+0000","last_change":"2024-04-26T20:56:50.689210+0000","last_active":"2024-04-26T20:56:52.713411+0000","last_peered":"2024-04-26T20:56:52.713411+0000","last_clean":"2024-04-26T20:56:52.713411+0000","last_became_active":"2024-04-26T20:56:50.688655+0000","last_became_peered":"2024-04-26T20:56:50.688655+0000","last_unstale":"2024-04-26T20:56:52.713411+0000","last_undegraded":"2024-04-26T20:56:52.713411+0000","last_fullsized":"2024-04-26T20:56:52.713411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:25.776704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.715628+0000","last_change":"2024-04-26T20:56:51.737157+0000","last_active":"2024-04-26T20:56:52.715628+0000","last_peered":"2024-04-26T20:56:52.715628+0000","last_clean":"2024-04-26T20:56:52.715628+0000","last_became_active":"2024-04-26T20:56:51.736082+0000","last_became_peered":"2024-04-26T20:56:51.736082+0000","last_unstale":"2024-04-26T20:56:52.715628+0000","last_undegraded":"2024-04-26T20:56:52.715628+0000","last_fullsized":"2024-04-26T20:56:52.715628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:40.342624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730295+0000","last_change":"2024-04-26T20:56:50.695225+0000","last_active":"2024-04-26T20:56:52.730295+0000","last_peered":"2024-04-26T20:56:52.730295+0000","last_clean":"2024-04-26T20:56:52.730295+0000","last_became_active":"2024-04-26T20:56:50.694892+0000","last_became_peered":"2024-04-26T20:56:50.694892+0000","last_unstale":"2024-04-26T20:56:52.730295+0000","last_undegraded":"2024-04-26T20:56:52.730295+0000","last_fullsized":"2024-04-26T20:56:52.730295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:41.662331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153191+0000","last_change":"2024-04-26T20:56:51.722030+0000","last_active":"2024-04-26T20:56:53.153191+0000","last_peered":"2024-04-26T20:56:53.153191+0000","last_clean":"2024-04-26T20:56:53.153191+0000","last_became_active":"2024-04-26T20:56:51.721724+0000","last_became_peered":"2024-04-26T20:56:51.721724+0000","last_unstale":"2024-04-26T20:56:53.153191+0000","last_undegraded":"2024-04-26T20:56:53.153191+0000","last_fullsized":"2024-04-26T20:56:53.153191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:52.330678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.145449+0000","last_change":"2024-04-26T20:56:50.682167+0000","last_active":"2024-04-26T20:56:53.145449+0000","last_peered":"2024-04-26T20:56:53.145449+0000","last_clean":"2024-04-26T20:56:53.145449+0000","last_became_active":"2024-04-26T20:56:50.675090+0000","last_became_peered":"2024-04-26T20:56:50.675090+0000","last_unstale":"2024-04-26T20:56:53.145449+0000","last_undegraded":"2024-04-26T20:56:53.145449+0000","last_fullsized":"2024-04-26T20:56:53.145449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:03.566375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.145134+0000","last_change":"2024-04-26T20:56:51.736362+0000","last_active":"2024-04-26T20:56:53.145134+0000","last_peered":"2024-04-26T20:56:53.145134+0000","last_clean":"2024-04-26T20:56:53.145134+0000","last_became_active":"2024-04-26T20:56:51.736050+0000","last_became_peered":"2024-04-26T20:56:51.736050+0000","last_unstale":"2024-04-26T20:56:53.145134+0000","last_undegraded":"2024-04-26T20:56:53.145134+0000","last_fullsized":"2024-04-26T20:56:53.145134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:24.679612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717056+0000","last_change":"2024-04-26T20:56:50.683775+0000","last_active":"2024-04-26T20:56:52.717056+0000","last_peered":"2024-04-26T20:56:52.717056+0000","last_clean":"2024-04-26T20:56:52.717056+0000","last_became_active":"2024-04-26T20:56:50.683470+0000","last_became_peered":"2024-04-26T20:56:50.683470+0000","last_unstale":"2024-04-26T20:56:52.717056+0000","last_undegraded":"2024-04-26T20:56:52.717056+0000","last_fullsized":"2024-04-26T20:56:52.717056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:50.526509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153154+0000","last_change":"2024-04-26T20:56:51.730365+0000","last_active":"2024-04-26T20:56:53.153154+0000","last_peered":"2024-04-26T20:56:53.153154+0000","last_clean":"2024-04-26T20:56:53.153154+0000","last_became_active":"2024-04-26T20:56:51.730015+0000","last_became_peered":"2024-04-26T20:56:51.730015+0000","last_unstale":"2024-04-26T20:56:53.153154+0000","last_undegraded":"2024-04-26T20:56:53.153154+0000","last_fullsized":"2024-04-26T20:56:53.153154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:22.643252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.729839+0000","last_change":"2024-04-26T20:56:50.686055+0000","last_active":"2024-04-26T20:56:52.729839+0000","last_peered":"2024-04-26T20:56:52.729839+0000","last_clean":"2024-04-26T20:56:52.729839+0000","last_became_active":"2024-04-26T20:56:50.685771+0000","last_became_peered":"2024-04-26T20:56:50.685771+0000","last_unstale":"2024-04-26T20:56:52.729839+0000","last_undegraded":"2024-04-26T20:56:52.729839+0000","last_fullsized":"2024-04-26T20:56:52.729839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:31.999414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.729780+0000","last_change":"2024-04-26T20:56:51.719273+0000","last_active":"2024-04-26T20:56:52.729780+0000","last_peered":"2024-04-26T20:56:52.729780+0000","last_clean":"2024-04-26T20:56:52.729780+0000","last_became_active":"2024-04-26T20:56:51.718659+0000","last_became_peered":"2024-04-26T20:56:51.718659+0000","last_unstale":"2024-04-26T20:56:52.729780+0000","last_undegraded":"2024-04-26T20:56:52.729780+0000","last_fullsized":"2024-04-26T20:56:52.729780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:16.275518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225397+0000","last_change":"2024-04-26T20:56:50.686714+0000","last_active":"2024-04-26T20:56:53.225397+0000","last_peered":"2024-04-26T20:56:53.225397+0000","last_clean":"2024-04-26T20:56:53.225397+0000","last_became_active":"2024-04-26T20:56:50.686348+0000","last_became_peered":"2024-04-26T20:56:50.686348+0000","last_unstale":"2024-04-26T20:56:53.225397+0000","last_undegraded":"2024-04-26T20:56:53.225397+0000","last_fullsized":"2024-04-26T20:56:53.225397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:14.594594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.152873+0000","last_change":"2024-04-26T20:56:51.730639+0000","last_active":"2024-04-26T20:56:53.152873+0000","last_peered":"2024-04-26T20:56:53.152873+0000","last_clean":"2024-04-26T20:56:53.152873+0000","last_became_active":"2024-04-26T20:56:51.730196+0000","last_became_peered":"2024-04-26T20:56:51.730196+0000","last_unstale":"2024-04-26T20:56:53.152873+0000","last_undegraded":"2024-04-26T20:56:53.152873+0000","last_fullsized":"2024-04-26T20:56:53.152873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:47.662176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.715683+0000","last_change":"2024-04-26T20:56:50.685016+0000","last_active":"2024-04-26T20:56:52.715683+0000","last_peered":"2024-04-26T20:56:52.715683+0000","last_clean":"2024-04-26T20:56:52.715683+0000","last_became_active":"2024-04-26T20:56:50.684679+0000","last_became_peered":"2024-04-26T20:56:50.684679+0000","last_unstale":"2024-04-26T20:56:52.715683+0000","last_undegraded":"2024-04-26T20:56:52.715683+0000","last_fullsized":"2024-04-26T20:56:52.715683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:34.811589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153972+0000","last_change":"2024-04-26T20:56:51.728717+0000","last_active":"2024-04-26T20:56:53.153972+0000","last_peered":"2024-04-26T20:56:53.153972+0000","last_clean":"2024-04-26T20:56:53.153972+0000","last_became_active":"2024-04-26T20:56:51.728261+0000","last_became_peered":"2024-04-26T20:56:51.728261+0000","last_unstale":"2024-04-26T20:56:53.153972+0000","last_undegraded":"2024-04-26T20:56:53.153972+0000","last_fullsized":"2024-04-26T20:56:53.153972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:02:31.238922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225598+0000","last_change":"2024-04-26T20:56:50.695251+0000","last_active":"2024-04-26T20:56:53.225598+0000","last_peered":"2024-04-26T20:56:53.225598+0000","last_clean":"2024-04-26T20:56:53.225598+0000","last_became_active":"2024-04-26T20:56:50.694956+0000","last_became_peered":"2024-04-26T20:56:50.694956+0000","last_unstale":"2024-04-26T20:56:53.225598+0000","last_undegraded":"2024-04-26T20:56:53.225598+0000","last_fullsized":"2024-04-26T20:56:53.225598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:22.304242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153932+0000","last_change":"2024-04-26T20:56:51.736963+0000","last_active":"2024-04-26T20:56:53.153932+0000","last_peered":"2024-04-26T20:56:53.153932+0000","last_clean":"2024-04-26T20:56:53.153932+0000","last_became_active":"2024-04-26T20:56:51.734437+0000","last_became_peered":"2024-04-26T20:56:51.734437+0000","last_unstale":"2024-04-26T20:56:53.153932+0000","last_undegraded":"2024-04-26T20:56:53.153932+0000","last_fullsized":"2024-04-26T20:56:53.153932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:08.679353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713517+0000","last_change":"2024-04-26T20:56:50.684034+0000","last_active":"2024-04-26T20:56:52.713517+0000","last_peered":"2024-04-26T20:56:52.713517+0000","last_clean":"2024-04-26T20:56:52.713517+0000","last_became_active":"2024-04-26T20:56:50.683720+0000","last_became_peered":"2024-04-26T20:56:50.683720+0000","last_unstale":"2024-04-26T20:56:52.713517+0000","last_undegraded":"2024-04-26T20:56:52.713517+0000","last_fullsized":"2024-04-26T20:56:52.713517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:46.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717068+0000","last_change":"2024-04-26T20:56:51.725602+0000","last_active":"2024-04-26T20:56:52.717068+0000","last_peered":"2024-04-26T20:56:52.717068+0000","last_clean":"2024-04-26T20:56:52.717068+0000","last_became_active":"2024-04-26T20:56:51.725308+0000","last_became_peered":"2024-04-26T20:56:51.725308+0000","last_unstale":"2024-04-26T20:56:52.717068+0000","last_undegraded":"2024-04-26T20:56:52.717068+0000","last_fullsized":"2024-04-26T20:56:52.717068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:52.330678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225216+0000","last_change":"2024-04-26T20:56:50.688464+0000","last_active":"2024-04-26T20:56:53.225216+0000","last_peered":"2024-04-26T20:56:53.225216+0000","last_clean":"2024-04-26T20:56:53.225216+0000","last_became_active":"2024-04-26T20:56:50.688101+0000","last_became_peered":"2024-04-26T20:56:50.688101+0000","last_unstale":"2024-04-26T20:56:53.225216+0000","last_undegraded":"2024-04-26T20:56:53.225216+0000","last_fullsized":"2024-04-26T20:56:53.225216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:50.526509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153386+0000","last_change":"2024-04-26T20:56:51.734773+0000","last_active":"2024-04-26T20:56:53.153386+0000","last_peered":"2024-04-26T20:56:53.153386+0000","last_clean":"2024-04-26T20:56:53.153386+0000","last_became_active":"2024-04-26T20:56:51.733903+0000","last_became_peered":"2024-04-26T20:56:51.733903+0000","last_unstale":"2024-04-26T20:56:53.153386+0000","last_undegraded":"2024-04-26T20:56:53.153386+0000","last_fullsized":"2024-04-26T20:56:53.153386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:23.450962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.152300+0000","last_change":"2024-04-26T20:56:46.848603+0000","last_active":"2024-04-26T20:56:53.152300+0000","last_peered":"2024-04-26T20:56:53.152300+0000","last_clean":"2024-04-26T20:56:53.152300+0000","last_became_active":"2024-04-26T20:56:46.846091+0000","last_became_peered":"2024-04-26T20:56:46.846091+0000","last_unstale":"2024-04-26T20:56:53.152300+0000","last_undegraded":"2024-04-26T20:56:53.152300+0000","last_fullsized":"2024-04-26T20:56:53.152300+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_clean_scrub_stamp":"2024-04-26T20:56:45.594041+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:00.232387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716618+0000","last_change":"2024-04-26T20:56:50.681942+0000","last_active":"2024-04-26T20:56:52.716618+0000","last_peered":"2024-04-26T20:56:52.716618+0000","last_clean":"2024-04-26T20:56:52.716618+0000","last_became_active":"2024-04-26T20:56:50.681645+0000","last_became_peered":"2024-04-26T20:56:50.681645+0000","last_unstale":"2024-04-26T20:56:52.716618+0000","last_undegraded":"2024-04-26T20:56:52.716618+0000","last_fullsized":"2024-04-26T20:56:52.716618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:39.328210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226904+0000","last_change":"2024-04-26T20:56:51.730048+0000","last_active":"2024-04-26T20:56:53.226904+0000","last_peered":"2024-04-26T20:56:53.226904+0000","last_clean":"2024-04-26T20:56:53.226904+0000","last_became_active":"2024-04-26T20:56:51.729733+0000","last_became_peered":"2024-04-26T20:56:51.729733+0000","last_unstale":"2024-04-26T20:56:53.226904+0000","last_undegraded":"2024-04-26T20:56:53.226904+0000","last_fullsized":"2024-04-26T20:56:53.226904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:24.679612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.152261+0000","last_change":"2024-04-26T20:56:50.699113+0000","last_active":"2024-04-26T20:56:53.152261+0000","last_peered":"2024-04-26T20:56:53.152261+0000","last_clean":"2024-04-26T20:56:53.152261+0000","last_became_active":"2024-04-26T20:56:50.698744+0000","last_became_peered":"2024-04-26T20:56:50.698744+0000","last_unstale":"2024-04-26T20:56:53.152261+0000","last_undegraded":"2024-04-26T20:56:53.152261+0000","last_fullsized":"2024-04-26T20:56:53.152261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:11.694549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144333+0000","last_change":"2024-04-26T20:56:51.732556+0000","last_active":"2024-04-26T20:56:53.144333+0000","last_peered":"2024-04-26T20:56:53.144333+0000","last_clean":"2024-04-26T20:56:53.144333+0000","last_became_active":"2024-04-26T20:56:51.732197+0000","last_became_peered":"2024-04-26T20:56:51.732197+0000","last_unstale":"2024-04-26T20:56:53.144333+0000","last_undegraded":"2024-04-26T20:56:53.144333+0000","last_fullsized":"2024-04-26T20:56:53.144333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:27:15.978359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225098+0000","last_change":"2024-04-26T20:56:50.694069+0000","last_active":"2024-04-26T20:56:53.225098+0000","last_peered":"2024-04-26T20:56:53.225098+0000","last_clean":"2024-04-26T20:56:53.225098+0000","last_became_active":"2024-04-26T20:56:50.693779+0000","last_became_peered":"2024-04-26T20:56:50.693779+0000","last_unstale":"2024-04-26T20:56:53.225098+0000","last_undegraded":"2024-04-26T20:56:53.225098+0000","last_fullsized":"2024-04-26T20:56:53.225098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:31.999414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.714833+0000","last_change":"2024-04-26T20:56:51.710866+0000","last_active":"2024-04-26T20:56:52.714833+0000","last_peered":"2024-04-26T20:56:52.714833+0000","last_clean":"2024-04-26T20:56:52.714833+0000","last_became_active":"2024-04-26T20:56:51.709325+0000","last_became_peered":"2024-04-26T20:56:51.709325+0000","last_unstale":"2024-04-26T20:56:52.714833+0000","last_undegraded":"2024-04-26T20:56:52.714833+0000","last_fullsized":"2024-04-26T20:56:52.714833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:23.268496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717176+0000","last_change":"2024-04-26T20:56:50.698231+0000","last_active":"2024-04-26T20:56:52.717176+0000","last_peered":"2024-04-26T20:56:52.717176+0000","last_clean":"2024-04-26T20:56:52.717176+0000","last_became_active":"2024-04-26T20:56:50.697872+0000","last_became_peered":"2024-04-26T20:56:50.697872+0000","last_unstale":"2024-04-26T20:56:52.717176+0000","last_undegraded":"2024-04-26T20:56:52.717176+0000","last_fullsized":"2024-04-26T20:56:52.717176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:41.662331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.715920+0000","last_change":"2024-04-26T20:56:51.720748+0000","last_active":"2024-04-26T20:56:52.715920+0000","last_peered":"2024-04-26T20:56:52.715920+0000","last_clean":"2024-04-26T20:56:52.715920+0000","last_became_active":"2024-04-26T20:56:51.720478+0000","last_became_peered":"2024-04-26T20:56:51.720478+0000","last_unstale":"2024-04-26T20:56:52.715920+0000","last_undegraded":"2024-04-26T20:56:52.715920+0000","last_fullsized":"2024-04-26T20:56:52.715920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:26.791119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713396+0000","last_change":"2024-04-26T20:56:50.685444+0000","last_active":"2024-04-26T20:56:52.713396+0000","last_peered":"2024-04-26T20:56:52.713396+0000","last_clean":"2024-04-26T20:56:52.713396+0000","last_became_active":"2024-04-26T20:56:50.684937+0000","last_became_peered":"2024-04-26T20:56:50.684937+0000","last_unstale":"2024-04-26T20:56:52.713396+0000","last_undegraded":"2024-04-26T20:56:52.713396+0000","last_fullsized":"2024-04-26T20:56:52.713396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:39.328210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717440+0000","last_change":"2024-04-26T20:56:51.727132+0000","last_active":"2024-04-26T20:56:52.717440+0000","last_peered":"2024-04-26T20:56:52.717440+0000","last_clean":"2024-04-26T20:56:52.717440+0000","last_became_active":"2024-04-26T20:56:51.720623+0000","last_became_peered":"2024-04-26T20:56:51.720623+0000","last_unstale":"2024-04-26T20:56:52.717440+0000","last_undegraded":"2024-04-26T20:56:52.717440+0000","last_fullsized":"2024-04-26T20:56:52.717440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:06.270681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153818+0000","last_change":"2024-04-26T20:56:50.693929+0000","last_active":"2024-04-26T20:56:53.153818+0000","last_peered":"2024-04-26T20:56:53.153818+0000","last_clean":"2024-04-26T20:56:53.153818+0000","last_became_active":"2024-04-26T20:56:50.693576+0000","last_became_peered":"2024-04-26T20:56:50.693576+0000","last_unstale":"2024-04-26T20:56:53.153818+0000","last_undegraded":"2024-04-26T20:56:53.153818+0000","last_fullsized":"2024-04-26T20:56:53.153818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:46.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713243+0000","last_change":"2024-04-26T20:56:51.714289+0000","last_active":"2024-04-26T20:56:52.713243+0000","last_peered":"2024-04-26T20:56:52.713243+0000","last_clean":"2024-04-26T20:56:52.713243+0000","last_became_active":"2024-04-26T20:56:51.713970+0000","last_became_peered":"2024-04-26T20:56:51.713970+0000","last_unstale":"2024-04-26T20:56:52.713243+0000","last_undegraded":"2024-04-26T20:56:52.713243+0000","last_fullsized":"2024-04-26T20:56:52.713243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:24.679612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.728669+0000","last_change":"2024-04-26T20:56:50.690572+0000","last_active":"2024-04-26T20:56:52.728669+0000","last_peered":"2024-04-26T20:56:52.728669+0000","last_clean":"2024-04-26T20:56:52.728669+0000","last_became_active":"2024-04-26T20:56:50.690221+0000","last_became_peered":"2024-04-26T20:56:50.690221+0000","last_unstale":"2024-04-26T20:56:52.728669+0000","last_undegraded":"2024-04-26T20:56:52.728669+0000","last_fullsized":"2024-04-26T20:56:52.728669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:11.694549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.227813+0000","last_change":"2024-04-26T20:56:51.740978+0000","last_active":"2024-04-26T20:56:53.227813+0000","last_peered":"2024-04-26T20:56:53.227813+0000","last_clean":"2024-04-26T20:56:53.227813+0000","last_became_active":"2024-04-26T20:56:51.739561+0000","last_became_peered":"2024-04-26T20:56:51.739561+0000","last_unstale":"2024-04-26T20:56:53.227813+0000","last_undegraded":"2024-04-26T20:56:53.227813+0000","last_fullsized":"2024-04-26T20:56:53.227813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:47.662176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730672+0000","last_change":"2024-04-26T20:56:50.694545+0000","last_active":"2024-04-26T20:56:52.730672+0000","last_peered":"2024-04-26T20:56:52.730672+0000","last_clean":"2024-04-26T20:56:52.730672+0000","last_became_active":"2024-04-26T20:56:50.694189+0000","last_became_peered":"2024-04-26T20:56:50.694189+0000","last_unstale":"2024-04-26T20:56:52.730672+0000","last_undegraded":"2024-04-26T20:56:52.730672+0000","last_fullsized":"2024-04-26T20:56:52.730672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:25.776704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144945+0000","last_change":"2024-04-26T20:56:51.724210+0000","last_active":"2024-04-26T20:56:53.144945+0000","last_peered":"2024-04-26T20:56:53.144945+0000","last_clean":"2024-04-26T20:56:53.144945+0000","last_became_active":"2024-04-26T20:56:51.723882+0000","last_became_peered":"2024-04-26T20:56:51.723882+0000","last_unstale":"2024-04-26T20:56:53.144945+0000","last_undegraded":"2024-04-26T20:56:53.144945+0000","last_fullsized":"2024-04-26T20:56:53.144945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:53.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.152108+0000","last_change":"2024-04-26T20:56:50.694039+0000","last_active":"2024-04-26T20:56:53.152108+0000","last_peered":"2024-04-26T20:56:53.152108+0000","last_clean":"2024-04-26T20:56:53.152108+0000","last_became_active":"2024-04-26T20:56:50.693569+0000","last_became_peered":"2024-04-26T20:56:50.693569+0000","last_unstale":"2024-04-26T20:56:53.152108+0000","last_undegraded":"2024-04-26T20:56:53.152108+0000","last_fullsized":"2024-04-26T20:56:53.152108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:41.662331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.714862+0000","last_change":"2024-04-26T20:56:51.719531+0000","last_active":"2024-04-26T20:56:52.714862+0000","last_peered":"2024-04-26T20:56:52.714862+0000","last_clean":"2024-04-26T20:56:52.714862+0000","last_became_active":"2024-04-26T20:56:51.714881+0000","last_became_peered":"2024-04-26T20:56:51.714881+0000","last_unstale":"2024-04-26T20:56:52.714862+0000","last_undegraded":"2024-04-26T20:56:52.714862+0000","last_fullsized":"2024-04-26T20:56:52.714862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:33.013829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713174+0000","last_change":"2024-04-26T20:56:50.689750+0000","last_active":"2024-04-26T20:56:52.713174+0000","last_peered":"2024-04-26T20:56:52.713174+0000","last_clean":"2024-04-26T20:56:52.713174+0000","last_became_active":"2024-04-26T20:56:50.689337+0000","last_became_peered":"2024-04-26T20:56:50.689337+0000","last_unstale":"2024-04-26T20:56:52.713174+0000","last_undegraded":"2024-04-26T20:56:52.713174+0000","last_fullsized":"2024-04-26T20:56:52.713174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:41.662331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226387+0000","last_change":"2024-04-26T20:56:51.712295+0000","last_active":"2024-04-26T20:56:53.226387+0000","last_peered":"2024-04-26T20:56:53.226387+0000","last_clean":"2024-04-26T20:56:53.226387+0000","last_became_active":"2024-04-26T20:56:51.711999+0000","last_became_peered":"2024-04-26T20:56:51.711999+0000","last_unstale":"2024-04-26T20:56:53.226387+0000","last_undegraded":"2024-04-26T20:56:53.226387+0000","last_fullsized":"2024-04-26T20:56:53.226387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:16.275518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713352+0000","last_change":"2024-04-26T20:56:50.697160+0000","last_active":"2024-04-26T20:56:52.713352+0000","last_peered":"2024-04-26T20:56:52.713352+0000","last_clean":"2024-04-26T20:56:52.713352+0000","last_became_active":"2024-04-26T20:56:50.695595+0000","last_became_peered":"2024-04-26T20:56:50.695595+0000","last_unstale":"2024-04-26T20:56:52.713352+0000","last_undegraded":"2024-04-26T20:56:52.713352+0000","last_fullsized":"2024-04-26T20:56:52.713352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:50.526509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226926+0000","last_change":"2024-04-26T20:56:51.717706+0000","last_active":"2024-04-26T20:56:53.226926+0000","last_peered":"2024-04-26T20:56:53.226926+0000","last_clean":"2024-04-26T20:56:53.226926+0000","last_became_active":"2024-04-26T20:56:51.717422+0000","last_became_peered":"2024-04-26T20:56:51.717422+0000","last_unstale":"2024-04-26T20:56:53.226926+0000","last_undegraded":"2024-04-26T20:56:53.226926+0000","last_fullsized":"2024-04-26T20:56:53.226926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:37.778325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153433+0000","last_change":"2024-04-26T20:56:50.695179+0000","last_active":"2024-04-26T20:56:53.153433+0000","last_peered":"2024-04-26T20:56:53.153433+0000","last_clean":"2024-04-26T20:56:53.153433+0000","last_became_active":"2024-04-26T20:56:50.694889+0000","last_became_peered":"2024-04-26T20:56:50.694889+0000","last_unstale":"2024-04-26T20:56:53.153433+0000","last_undegraded":"2024-04-26T20:56:53.153433+0000","last_fullsized":"2024-04-26T20:56:53.153433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:50.526509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730113+0000","last_change":"2024-04-26T20:56:51.713029+0000","last_active":"2024-04-26T20:56:52.730113+0000","last_peered":"2024-04-26T20:56:52.730113+0000","last_clean":"2024-04-26T20:56:52.730113+0000","last_became_active":"2024-04-26T20:56:51.712311+0000","last_became_peered":"2024-04-26T20:56:51.712311+0000","last_unstale":"2024-04-26T20:56:52.730113+0000","last_undegraded":"2024-04-26T20:56:52.730113+0000","last_fullsized":"2024-04-26T20:56:52.730113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:36.354765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717262+0000","last_change":"2024-04-26T20:56:51.730361+0000","last_active":"2024-04-26T20:56:52.717262+0000","last_peered":"2024-04-26T20:56:52.717262+0000","last_clean":"2024-04-26T20:56:52.717262+0000","last_became_active":"2024-04-26T20:56:51.730068+0000","last_became_peered":"2024-04-26T20:56:51.730068+0000","last_unstale":"2024-04-26T20:56:52.717262+0000","last_undegraded":"2024-04-26T20:56:52.717262+0000","last_fullsized":"2024-04-26T20:56:52.717262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:07.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153083+0000","last_change":"2024-04-26T20:56:50.696973+0000","last_active":"2024-04-26T20:56:53.153083+0000","last_peered":"2024-04-26T20:56:53.153083+0000","last_clean":"2024-04-26T20:56:53.153083+0000","last_became_active":"2024-04-26T20:56:50.696603+0000","last_became_peered":"2024-04-26T20:56:50.696603+0000","last_unstale":"2024-04-26T20:56:53.153083+0000","last_undegraded":"2024-04-26T20:56:53.153083+0000","last_fullsized":"2024-04-26T20:56:53.153083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:25.776704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717108+0000","last_change":"2024-04-26T20:56:51.728709+0000","last_active":"2024-04-26T20:56:52.717108+0000","last_peered":"2024-04-26T20:56:52.717108+0000","last_clean":"2024-04-26T20:56:52.717108+0000","last_became_active":"2024-04-26T20:56:51.727858+0000","last_became_peered":"2024-04-26T20:56:51.727858+0000","last_unstale":"2024-04-26T20:56:52.717108+0000","last_undegraded":"2024-04-26T20:56:52.717108+0000","last_fullsized":"2024-04-26T20:56:52.717108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:23.450962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153531+0000","last_change":"2024-04-26T20:56:50.696966+0000","last_active":"2024-04-26T20:56:53.153531+0000","last_peered":"2024-04-26T20:56:53.153531+0000","last_clean":"2024-04-26T20:56:53.153531+0000","last_became_active":"2024-04-26T20:56:50.696448+0000","last_became_peered":"2024-04-26T20:56:50.696448+0000","last_unstale":"2024-04-26T20:56:53.153531+0000","last_undegraded":"2024-04-26T20:56:53.153531+0000","last_fullsized":"2024-04-26T20:56:53.153531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:39.328210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144675+0000","last_change":"2024-04-26T20:56:51.731938+0000","last_active":"2024-04-26T20:56:53.144675+0000","last_peered":"2024-04-26T20:56:53.144675+0000","last_clean":"2024-04-26T20:56:53.144675+0000","last_became_active":"2024-04-26T20:56:51.731655+0000","last_became_peered":"2024-04-26T20:56:51.731655+0000","last_unstale":"2024-04-26T20:56:53.144675+0000","last_undegraded":"2024-04-26T20:56:53.144675+0000","last_fullsized":"2024-04-26T20:56:53.144675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:18.631055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.227678+0000","last_change":"2024-04-26T20:56:50.692059+0000","last_active":"2024-04-26T20:56:53.227678+0000","last_peered":"2024-04-26T20:56:53.227678+0000","last_clean":"2024-04-26T20:56:53.227678+0000","last_became_active":"2024-04-26T20:56:50.690628+0000","last_became_peered":"2024-04-26T20:56:50.690628+0000","last_unstale":"2024-04-26T20:56:53.227678+0000","last_undegraded":"2024-04-26T20:56:53.227678+0000","last_fullsized":"2024-04-26T20:56:53.227678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:00.125994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.227526+0000","last_change":"2024-04-26T20:56:51.719241+0000","last_active":"2024-04-26T20:56:53.227526+0000","last_peered":"2024-04-26T20:56:53.227526+0000","last_clean":"2024-04-26T20:56:53.227526+0000","last_became_active":"2024-04-26T20:56:51.718893+0000","last_became_peered":"2024-04-26T20:56:51.718893+0000","last_unstale":"2024-04-26T20:56:53.227526+0000","last_undegraded":"2024-04-26T20:56:53.227526+0000","last_fullsized":"2024-04-26T20:56:53.227526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:23.268496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144835+0000","last_change":"2024-04-26T20:56:50.691078+0000","last_active":"2024-04-26T20:56:53.144835+0000","last_peered":"2024-04-26T20:56:53.144835+0000","last_clean":"2024-04-26T20:56:53.144835+0000","last_became_active":"2024-04-26T20:56:50.689362+0000","last_became_peered":"2024-04-26T20:56:50.689362+0000","last_unstale":"2024-04-26T20:56:53.144835+0000","last_undegraded":"2024-04-26T20:56:53.144835+0000","last_fullsized":"2024-04-26T20:56:53.144835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:00.125994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:56:59.869 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:57:00.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.116+0000 7f8e23200640 1 Processor -- start 2024-04-26T20:57:00.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.116+0000 7f8e23200640 1 -- start start 2024-04-26T20:57:00.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.118+0000 7f8e23200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f8e24146670 0x7f8e24071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:00.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.118+0000 7f8e23200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8e241610c0 0x7f8e240cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:00.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.118+0000 7f8e23200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e24146440 con 0x7f8e24146060 2024-04-26T20:57:00.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.118+0000 7f8e23200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f8e2406f510 con 0x7f8e241610c0 2024-04-26T20:57:00.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.118+0000 7f8e23200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f8e2406f010 con 0x7f8e24146670 2024-04-26T20:57:00.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.118+0000 7f8e22800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8e241610c0 0x7f8e240cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:00.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.118+0000 7f8e22800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8e241610c0 0x7f8e240cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35100/0 (socket says 172.21.15.28:35100) 2024-04-26T20:57:00.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.118+0000 7f8e21e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8e24146060 0x7f8e241609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47736/0 (socket says 172.21.15.28:47736) 2024-04-26T20:57:00.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.118+0000 7f8e22800640 1 -- 172.21.15.28:0/2762280619 learned_addr learned my addr 172.21.15.28:0/2762280619 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:00.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.118+0000 7f8e19400640 1 --2- 172.21.15.28:0/2762280619 >> v2:172.21.15.103:3300/0 conn(0x7f8e24146670 0x7f8e24071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:00.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.119+0000 7f8e22800640 1 -- 172.21.15.28:0/2762280619 >> v2:172.21.15.103:3300/0 conn(0x7f8e24146670 msgr2=0x7f8e24071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:00.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.119+0000 7f8e22800640 1 --2- 172.21.15.28:0/2762280619 >> v2:172.21.15.103:3300/0 conn(0x7f8e24146670 0x7f8e24071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:00.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.119+0000 7f8e22800640 1 -- 172.21.15.28:0/2762280619 >> v1:172.21.15.28:6789/0 conn(0x7f8e24146060 legacy=0x7f8e241609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:00.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.119+0000 7f8e22800640 1 -- 172.21.15.28:0/2762280619 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e241614d0 con 0x7f8e241610c0 2024-04-26T20:57:00.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.119+0000 7f8e21400640 1 -- 172.21.15.28:0/2762280619 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2410219097 0 0) 0x7f8e24146440 con 0x7f8e24146060 2024-04-26T20:57:00.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.120+0000 7f8e22800640 1 --2- 172.21.15.28:0/2762280619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e241610c0 0x7f8e240cb0a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8e2406f8c0 tx=0x7f8e1c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89d52165e7e0890 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:00.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.120+0000 7f8e21400640 1 -- 172.21.15.28:0/2762280619 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c036070 con 0x7f8e241610c0 2024-04-26T20:57:00.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.120+0000 7f8e21400640 1 -- 172.21.15.28:0/2762280619 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e1c031600 con 0x7f8e241610c0 2024-04-26T20:57:00.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.120+0000 7f8e21400640 1 -- 172.21.15.28:0/2762280619 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c0319d0 con 0x7f8e241610c0 2024-04-26T20:57:00.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.120+0000 7f8e23200640 1 -- 172.21.15.28:0/2762280619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e241610c0 msgr2=0x7f8e240cb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:00.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.120+0000 7f8e23200640 1 --2- 172.21.15.28:0/2762280619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e241610c0 0x7f8e240cb0a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8e2406f8c0 tx=0x7f8e1c02a6a0 comp rx=0 tx=0).stop 2024-04-26T20:57:00.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.121+0000 7f8e23200640 1 -- 172.21.15.28:0/2762280619 shutdown_connections 2024-04-26T20:57:00.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.121+0000 7f8e23200640 1 --2- 172.21.15.28:0/2762280619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e241610c0 0x7f8e240cb0a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:00.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.121+0000 7f8e23200640 1 --2- 172.21.15.28:0/2762280619 >> v2:172.21.15.103:3300/0 conn(0x7f8e24146670 0x7f8e24071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:00.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.121+0000 7f8e23200640 1 -- 172.21.15.28:0/2762280619 >> 172.21.15.28:0/2762280619 conn(0x7f8e240d0a50 msgr2=0x7f8e240d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:00.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.121+0000 7f8e23200640 1 -- 172.21.15.28:0/2762280619 shutdown_connections 2024-04-26T20:57:00.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.121+0000 7f8e23200640 1 -- 172.21.15.28:0/2762280619 wait complete. 2024-04-26T20:57:00.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.122+0000 7f8e23200640 1 Processor -- start 2024-04-26T20:57:00.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.123+0000 7f8e23200640 1 -- start start 2024-04-26T20:57:00.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.123+0000 7f8e23200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8e24146060 0x7f8e2415e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:00.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.123+0000 7f8e23200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e24146670 0x7f8e240b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:00.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.123+0000 7f8e23200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e241610c0 0x7f8e240ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:00.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.124+0000 7f8e23200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2406ff60 con 0x7f8e241610c0 2024-04-26T20:57:00.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.124+0000 7f8e23200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8e2406fc80 con 0x7f8e24146670 2024-04-26T20:57:00.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.124+0000 7f8e23200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f8e240cb5e0 con 0x7f8e24146060 2024-04-26T20:57:00.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.124+0000 7f8e22800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e241610c0 0x7f8e240ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:00.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.124+0000 7f8e22800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e241610c0 0x7f8e240ad8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35104/0 (socket says 172.21.15.28:35104) 2024-04-26T20:57:00.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.124+0000 7f8e22800640 1 -- 172.21.15.28:0/49189557 learned_addr learned my addr 172.21.15.28:0/49189557 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:00.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.124+0000 7f8e21e00640 1 --2- 172.21.15.28:0/49189557 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8e24146060 0x7f8e2415e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:00.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.124+0000 7f8e19400640 1 --2- 172.21.15.28:0/49189557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e24146670 0x7f8e240b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:00.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.124+0000 7f8e22800640 1 -- 172.21.15.28:0/49189557 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8e24146060 msgr2=0x7f8e2415e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:00.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.124+0000 7f8e22800640 1 --2- 172.21.15.28:0/49189557 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8e24146060 0x7f8e2415e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:00.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.124+0000 7f8e22800640 1 -- 172.21.15.28:0/49189557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e24146670 msgr2=0x7f8e240b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:00.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.124+0000 7f8e22800640 1 --2- 172.21.15.28:0/49189557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e24146670 0x7f8e240b5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:00.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.124+0000 7f8e22800640 1 -- 172.21.15.28:0/49189557 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e240b4a00 con 0x7f8e241610c0 2024-04-26T20:57:00.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.124+0000 7f8e19400640 1 --2- 172.21.15.28:0/49189557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e24146670 0x7f8e240b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:00.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.125+0000 7f8e22800640 1 --2- 172.21.15.28:0/49189557 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e241610c0 0x7f8e240ad8f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c034040 tx=0x7f8e1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:00.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.125+0000 7f8e1aa00640 1 -- 172.21.15.28:0/49189557 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c036050 con 0x7f8e241610c0 2024-04-26T20:57:00.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.125+0000 7f8e1aa00640 1 -- 172.21.15.28:0/49189557 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e1c042070 con 0x7f8e241610c0 2024-04-26T20:57:00.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.125+0000 7f8e1aa00640 1 -- 172.21.15.28:0/49189557 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c009510 con 0x7f8e241610c0 2024-04-26T20:57:00.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.125+0000 7f8e23200640 1 -- 172.21.15.28:0/49189557 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e240b4ce0 con 0x7f8e241610c0 2024-04-26T20:57:00.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.125+0000 7f8e23200640 1 -- 172.21.15.28:0/49189557 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e240b52a0 con 0x7f8e241610c0 2024-04-26T20:57:00.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.126+0000 7f8e23200640 1 -- 172.21.15.28:0/49189557 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e2406a360 con 0x7f8e241610c0 2024-04-26T20:57:00.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.127+0000 7f8e1aa00640 1 -- 172.21.15.28:0/49189557 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e1c007bd0 con 0x7f8e241610c0 2024-04-26T20:57:00.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.128+0000 7f8e1aa00640 1 --2- 172.21.15.28:0/49189557 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8e0006c8f0 0x7f8e0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:00.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.128+0000 7f8e1aa00640 1 -- 172.21.15.28:0/49189557 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8e1c0aca90 con 0x7f8e241610c0 2024-04-26T20:57:00.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.128+0000 7f8e21e00640 1 --2- 172.21.15.28:0/49189557 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8e0006c8f0 0x7f8e0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:00.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.129+0000 7f8e21e00640 1 --2- 172.21.15.28:0/49189557 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8e0006c8f0 0x7f8e0006edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8e14000f60 tx=0x7f8e14007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:00.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.132+0000 7f8e1aa00640 1 -- 172.21.15.28:0/49189557 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8e1c07ae20 con 0x7f8e241610c0 2024-04-26T20:57:00.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.324+0000 7f8e23200640 1 -- 172.21.15.28:0/49189557 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8e2415fbb0 con 0x7f8e0006c8f0 2024-04-26T20:57:00.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.334+0000 7f8e1aa00640 1 -- 172.21.15.28:0/49189557 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251836 (secure 0 0 0) 0x7f8e2415fbb0 con 0x7f8e0006c8f0 2024-04-26T20:57:00.339 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:00.340 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T20:57:00.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.340+0000 7f8e23200640 1 -- 172.21.15.28:0/49189557 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8e0006c8f0 msgr2=0x7f8e0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:00.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.340+0000 7f8e23200640 1 --2- 172.21.15.28:0/49189557 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8e0006c8f0 0x7f8e0006edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8e14000f60 tx=0x7f8e14007b50 comp rx=0 tx=0).stop 2024-04-26T20:57:00.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.340+0000 7f8e23200640 1 -- 172.21.15.28:0/49189557 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e241610c0 msgr2=0x7f8e240ad8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:00.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.340+0000 7f8e23200640 1 --2- 172.21.15.28:0/49189557 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e241610c0 0x7f8e240ad8f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c034040 tx=0x7f8e1c004a40 comp rx=0 tx=0).stop 2024-04-26T20:57:00.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.341+0000 7f8e23200640 1 -- 172.21.15.28:0/49189557 shutdown_connections 2024-04-26T20:57:00.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.341+0000 7f8e23200640 1 --2- 172.21.15.28:0/49189557 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8e0006c8f0 0x7f8e0006edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:00.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.341+0000 7f8e23200640 1 --2- 172.21.15.28:0/49189557 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8e241610c0 0x7f8e240ad8f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:00.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.341+0000 7f8e23200640 1 --2- 172.21.15.28:0/49189557 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8e24146670 0x7f8e240b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:00.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.341+0000 7f8e23200640 1 --2- 172.21.15.28:0/49189557 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8e24146060 0x7f8e2415e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:00.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.341+0000 7f8e23200640 1 -- 172.21.15.28:0/49189557 >> 172.21.15.28:0/49189557 conn(0x7f8e240d0a50 msgr2=0x7f8e2406e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:00.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.341+0000 7f8e23200640 1 -- 172.21.15.28:0/49189557 shutdown_connections 2024-04-26T20:57:00.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.341+0000 7f8e23200640 1 -- 172.21.15.28:0/49189557 wait complete. 2024-04-26T20:57:00.358 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-26T20:56:58.706460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235752,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738968,"statfs":{"total":773094113280,"available":772852703232,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.005812"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153718+0000","last_change":"2024-04-26T20:56:51.731207+0000","last_active":"2024-04-26T20:56:53.153718+0000","last_peered":"2024-04-26T20:56:53.153718+0000","last_clean":"2024-04-26T20:56:53.153718+0000","last_became_active":"2024-04-26T20:56:51.730930+0000","last_became_peered":"2024-04-26T20:56:51.730930+0000","last_unstale":"2024-04-26T20:56:53.153718+0000","last_undegraded":"2024-04-26T20:56:53.153718+0000","last_fullsized":"2024-04-26T20:56:53.153718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:06.270681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.714391+0000","last_change":"2024-04-26T20:56:51.719125+0000","last_active":"2024-04-26T20:56:52.714391+0000","last_peered":"2024-04-26T20:56:52.714391+0000","last_clean":"2024-04-26T20:56:52.714391+0000","last_became_active":"2024-04-26T20:56:51.717685+0000","last_became_peered":"2024-04-26T20:56:51.717685+0000","last_unstale":"2024-04-26T20:56:52.714391+0000","last_undegraded":"2024-04-26T20:56:52.714391+0000","last_fullsized":"2024-04-26T20:56:52.714391+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.428432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144729+0000","last_change":"2024-04-26T20:56:51.717024+0000","last_active":"2024-04-26T20:56:53.144729+0000","last_peered":"2024-04-26T20:56:53.144729+0000","last_clean":"2024-04-26T20:56:53.144729+0000","last_became_active":"2024-04-26T20:56:51.716486+0000","last_became_peered":"2024-04-26T20:56:51.716486+0000","last_unstale":"2024-04-26T20:56:53.144729+0000","last_undegraded":"2024-04-26T20:56:53.144729+0000","last_fullsized":"2024-04-26T20:56:53.144729+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:01.617412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716816+0000","last_change":"2024-04-26T20:56:51.716770+0000","last_active":"2024-04-26T20:56:52.716816+0000","last_peered":"2024-04-26T20:56:52.716816+0000","last_clean":"2024-04-26T20:56:52.716816+0000","last_became_active":"2024-04-26T20:56:51.716443+0000","last_became_peered":"2024-04-26T20:56:51.716443+0000","last_unstale":"2024-04-26T20:56:52.716816+0000","last_undegraded":"2024-04-26T20:56:52.716816+0000","last_fullsized":"2024-04-26T20:56:52.716816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:08.812637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153115+0000","last_change":"2024-04-26T20:56:51.735130+0000","last_active":"2024-04-26T20:56:53.153115+0000","last_peered":"2024-04-26T20:56:53.153115+0000","last_clean":"2024-04-26T20:56:53.153115+0000","last_became_active":"2024-04-26T20:56:51.734775+0000","last_became_peered":"2024-04-26T20:56:51.734775+0000","last_unstale":"2024-04-26T20:56:53.153115+0000","last_undegraded":"2024-04-26T20:56:53.153115+0000","last_fullsized":"2024-04-26T20:56:53.153115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:08.812637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716491+0000","last_change":"2024-04-26T20:56:51.719621+0000","last_active":"2024-04-26T20:56:52.716491+0000","last_peered":"2024-04-26T20:56:52.716491+0000","last_clean":"2024-04-26T20:56:52.716491+0000","last_became_active":"2024-04-26T20:56:51.719312+0000","last_became_peered":"2024-04-26T20:56:51.719312+0000","last_unstale":"2024-04-26T20:56:52.716491+0000","last_undegraded":"2024-04-26T20:56:52.716491+0000","last_fullsized":"2024-04-26T20:56:52.716491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:23.268496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.152560+0000","last_change":"2024-04-26T20:56:51.730495+0000","last_active":"2024-04-26T20:56:53.152560+0000","last_peered":"2024-04-26T20:56:53.152560+0000","last_clean":"2024-04-26T20:56:53.152560+0000","last_became_active":"2024-04-26T20:56:51.729796+0000","last_became_peered":"2024-04-26T20:56:51.729796+0000","last_unstale":"2024-04-26T20:56:53.152560+0000","last_undegraded":"2024-04-26T20:56:53.152560+0000","last_fullsized":"2024-04-26T20:56:53.152560+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:52.912874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225874+0000","last_change":"2024-04-26T20:56:51.729824+0000","last_active":"2024-04-26T20:56:53.225874+0000","last_peered":"2024-04-26T20:56:53.225874+0000","last_clean":"2024-04-26T20:56:53.225874+0000","last_became_active":"2024-04-26T20:56:51.729546+0000","last_became_peered":"2024-04-26T20:56:51.729546+0000","last_unstale":"2024-04-26T20:56:53.225874+0000","last_undegraded":"2024-04-26T20:56:53.225874+0000","last_fullsized":"2024-04-26T20:56:53.225874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:18.631055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153639+0000","last_change":"2024-04-26T20:56:51.732556+0000","last_active":"2024-04-26T20:56:53.153639+0000","last_peered":"2024-04-26T20:56:53.153639+0000","last_clean":"2024-04-26T20:56:53.153639+0000","last_became_active":"2024-04-26T20:56:51.732298+0000","last_became_peered":"2024-04-26T20:56:51.732298+0000","last_unstale":"2024-04-26T20:56:53.153639+0000","last_undegraded":"2024-04-26T20:56:53.153639+0000","last_fullsized":"2024-04-26T20:56:53.153639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:07.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730718+0000","last_change":"2024-04-26T20:56:51.724747+0000","last_active":"2024-04-26T20:56:52.730718+0000","last_peered":"2024-04-26T20:56:52.730718+0000","last_clean":"2024-04-26T20:56:52.730718+0000","last_became_active":"2024-04-26T20:56:51.724452+0000","last_became_peered":"2024-04-26T20:56:51.724452+0000","last_unstale":"2024-04-26T20:56:52.730718+0000","last_undegraded":"2024-04-26T20:56:52.730718+0000","last_fullsized":"2024-04-26T20:56:52.730718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.428432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730341+0000","last_change":"2024-04-26T20:56:51.729732+0000","last_active":"2024-04-26T20:56:52.730341+0000","last_peered":"2024-04-26T20:56:52.730341+0000","last_clean":"2024-04-26T20:56:52.730341+0000","last_became_active":"2024-04-26T20:56:51.729464+0000","last_became_peered":"2024-04-26T20:56:51.729464+0000","last_unstale":"2024-04-26T20:56:52.730341+0000","last_undegraded":"2024-04-26T20:56:52.730341+0000","last_fullsized":"2024-04-26T20:56:52.730341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:37.778325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.715425+0000","last_change":"2024-04-26T20:56:51.723870+0000","last_active":"2024-04-26T20:56:52.715425+0000","last_peered":"2024-04-26T20:56:52.715425+0000","last_clean":"2024-04-26T20:56:52.715425+0000","last_became_active":"2024-04-26T20:56:51.722037+0000","last_became_peered":"2024-04-26T20:56:51.722037+0000","last_unstale":"2024-04-26T20:56:52.715425+0000","last_undegraded":"2024-04-26T20:56:52.715425+0000","last_fullsized":"2024-04-26T20:56:52.715425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:42.676745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226737+0000","last_change":"2024-04-26T20:56:51.719947+0000","last_active":"2024-04-26T20:56:53.226737+0000","last_peered":"2024-04-26T20:56:53.226737+0000","last_clean":"2024-04-26T20:56:53.226737+0000","last_became_active":"2024-04-26T20:56:51.719565+0000","last_became_peered":"2024-04-26T20:56:51.719565+0000","last_unstale":"2024-04-26T20:56:53.226737+0000","last_undegraded":"2024-04-26T20:56:53.226737+0000","last_fullsized":"2024-04-26T20:56:53.226737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:36.354765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144673+0000","last_change":"2024-04-26T20:56:51.725826+0000","last_active":"2024-04-26T20:56:53.144673+0000","last_peered":"2024-04-26T20:56:53.144673+0000","last_clean":"2024-04-26T20:56:53.144673+0000","last_became_active":"2024-04-26T20:56:51.723315+0000","last_became_peered":"2024-04-26T20:56:51.723315+0000","last_unstale":"2024-04-26T20:56:53.144673+0000","last_undegraded":"2024-04-26T20:56:53.144673+0000","last_fullsized":"2024-04-26T20:56:53.144673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:38:55.851891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713041+0000","last_change":"2024-04-26T20:56:51.704000+0000","last_active":"2024-04-26T20:56:52.713041+0000","last_peered":"2024-04-26T20:56:52.713041+0000","last_clean":"2024-04-26T20:56:52.713041+0000","last_became_active":"2024-04-26T20:56:51.703007+0000","last_became_peered":"2024-04-26T20:56:51.703007+0000","last_unstale":"2024-04-26T20:56:52.713041+0000","last_undegraded":"2024-04-26T20:56:52.713041+0000","last_fullsized":"2024-04-26T20:56:52.713041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:36.354765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716554+0000","last_change":"2024-04-26T20:56:51.724622+0000","last_active":"2024-04-26T20:56:52.716554+0000","last_peered":"2024-04-26T20:56:52.716554+0000","last_clean":"2024-04-26T20:56:52.716554+0000","last_became_active":"2024-04-26T20:56:51.723198+0000","last_became_peered":"2024-04-26T20:56:51.723198+0000","last_unstale":"2024-04-26T20:56:52.716554+0000","last_undegraded":"2024-04-26T20:56:52.716554+0000","last_fullsized":"2024-04-26T20:56:52.716554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:52.912874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.718883+0000","last_change":"2024-04-26T20:56:51.736813+0000","last_active":"2024-04-26T20:56:52.718883+0000","last_peered":"2024-04-26T20:56:52.718883+0000","last_clean":"2024-04-26T20:56:52.718883+0000","last_became_active":"2024-04-26T20:56:51.736532+0000","last_became_peered":"2024-04-26T20:56:51.736532+0000","last_unstale":"2024-04-26T20:56:52.718883+0000","last_undegraded":"2024-04-26T20:56:52.718883+0000","last_fullsized":"2024-04-26T20:56:52.718883+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:08.679353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225324+0000","last_change":"2024-04-26T20:56:51.726234+0000","last_active":"2024-04-26T20:56:53.225324+0000","last_peered":"2024-04-26T20:56:53.225324+0000","last_clean":"2024-04-26T20:56:53.225324+0000","last_became_active":"2024-04-26T20:56:51.723289+0000","last_became_peered":"2024-04-26T20:56:51.723289+0000","last_unstale":"2024-04-26T20:56:53.225324+0000","last_undegraded":"2024-04-26T20:56:53.225324+0000","last_fullsized":"2024-04-26T20:56:53.225324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:12.708963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153878+0000","last_change":"2024-04-26T20:56:51.735058+0000","last_active":"2024-04-26T20:56:53.153878+0000","last_peered":"2024-04-26T20:56:53.153878+0000","last_clean":"2024-04-26T20:56:53.153878+0000","last_became_active":"2024-04-26T20:56:51.734516+0000","last_became_peered":"2024-04-26T20:56:51.734516+0000","last_unstale":"2024-04-26T20:56:53.153878+0000","last_undegraded":"2024-04-26T20:56:53.153878+0000","last_fullsized":"2024-04-26T20:56:53.153878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:33.950624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730309+0000","last_change":"2024-04-26T20:56:51.715023+0000","last_active":"2024-04-26T20:56:52.730309+0000","last_peered":"2024-04-26T20:56:52.730309+0000","last_clean":"2024-04-26T20:56:52.730309+0000","last_became_active":"2024-04-26T20:56:51.714250+0000","last_became_peered":"2024-04-26T20:56:51.714250+0000","last_unstale":"2024-04-26T20:56:52.730309+0000","last_undegraded":"2024-04-26T20:56:52.730309+0000","last_fullsized":"2024-04-26T20:56:52.730309+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:24.679612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144998+0000","last_change":"2024-04-26T20:56:51.737492+0000","last_active":"2024-04-26T20:56:53.144998+0000","last_peered":"2024-04-26T20:56:53.144998+0000","last_clean":"2024-04-26T20:56:53.144998+0000","last_became_active":"2024-04-26T20:56:51.736774+0000","last_became_peered":"2024-04-26T20:56:51.736774+0000","last_unstale":"2024-04-26T20:56:53.144998+0000","last_undegraded":"2024-04-26T20:56:53.144998+0000","last_fullsized":"2024-04-26T20:56:53.144998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:36.354765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716674+0000","last_change":"2024-04-26T20:56:51.731749+0000","last_active":"2024-04-26T20:56:52.716674+0000","last_peered":"2024-04-26T20:56:52.716674+0000","last_clean":"2024-04-26T20:56:52.716674+0000","last_became_active":"2024-04-26T20:56:51.731460+0000","last_became_peered":"2024-04-26T20:56:51.731460+0000","last_unstale":"2024-04-26T20:56:52.716674+0000","last_undegraded":"2024-04-26T20:56:52.716674+0000","last_fullsized":"2024-04-26T20:56:52.716674+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:47.662176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226704+0000","last_change":"2024-04-26T20:56:51.721856+0000","last_active":"2024-04-26T20:56:53.226704+0000","last_peered":"2024-04-26T20:56:53.226704+0000","last_clean":"2024-04-26T20:56:53.226704+0000","last_became_active":"2024-04-26T20:56:51.721531+0000","last_became_peered":"2024-04-26T20:56:51.721531+0000","last_unstale":"2024-04-26T20:56:53.226704+0000","last_undegraded":"2024-04-26T20:56:53.226704+0000","last_fullsized":"2024-04-26T20:56:53.226704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:53.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716438+0000","last_change":"2024-04-26T20:56:51.718492+0000","last_active":"2024-04-26T20:56:52.716438+0000","last_peered":"2024-04-26T20:56:52.716438+0000","last_clean":"2024-04-26T20:56:52.716438+0000","last_became_active":"2024-04-26T20:56:51.718127+0000","last_became_peered":"2024-04-26T20:56:51.718127+0000","last_unstale":"2024-04-26T20:56:52.716438+0000","last_undegraded":"2024-04-26T20:56:52.716438+0000","last_fullsized":"2024-04-26T20:56:52.716438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.428432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.227580+0000","last_change":"2024-04-26T20:56:51.722240+0000","last_active":"2024-04-26T20:56:53.227580+0000","last_peered":"2024-04-26T20:56:53.227580+0000","last_clean":"2024-04-26T20:56:53.227580+0000","last_became_active":"2024-04-26T20:56:51.721601+0000","last_became_peered":"2024-04-26T20:56:51.721601+0000","last_unstale":"2024-04-26T20:56:53.227580+0000","last_undegraded":"2024-04-26T20:56:53.227580+0000","last_fullsized":"2024-04-26T20:56:53.227580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:21:52.912874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730730+0000","last_change":"2024-04-26T20:56:51.722608+0000","last_active":"2024-04-26T20:56:52.730730+0000","last_peered":"2024-04-26T20:56:52.730730+0000","last_clean":"2024-04-26T20:56:52.730730+0000","last_became_active":"2024-04-26T20:56:51.722321+0000","last_became_peered":"2024-04-26T20:56:51.722321+0000","last_unstale":"2024-04-26T20:56:52.730730+0000","last_undegraded":"2024-04-26T20:56:52.730730+0000","last_fullsized":"2024-04-26T20:56:52.730730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:23.268496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713738+0000","last_change":"2024-04-26T20:56:51.715230+0000","last_active":"2024-04-26T20:56:52.713738+0000","last_peered":"2024-04-26T20:56:52.713738+0000","last_clean":"2024-04-26T20:56:52.713738+0000","last_became_active":"2024-04-26T20:56:51.714911+0000","last_became_peered":"2024-04-26T20:56:51.714911+0000","last_unstale":"2024-04-26T20:56:52.713738+0000","last_undegraded":"2024-04-26T20:56:52.713738+0000","last_fullsized":"2024-04-26T20:56:52.713738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:37.778325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.227039+0000","last_change":"2024-04-26T20:56:51.712010+0000","last_active":"2024-04-26T20:56:53.227039+0000","last_peered":"2024-04-26T20:56:53.227039+0000","last_clean":"2024-04-26T20:56:53.227039+0000","last_became_active":"2024-04-26T20:56:51.711583+0000","last_became_peered":"2024-04-26T20:56:51.711583+0000","last_unstale":"2024-04-26T20:56:53.227039+0000","last_undegraded":"2024-04-26T20:56:53.227039+0000","last_fullsized":"2024-04-26T20:56:53.227039+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.428432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.145333+0000","last_change":"2024-04-26T20:56:51.722575+0000","last_active":"2024-04-26T20:56:53.145333+0000","last_peered":"2024-04-26T20:56:53.145333+0000","last_clean":"2024-04-26T20:56:53.145333+0000","last_became_active":"2024-04-26T20:56:51.722274+0000","last_became_peered":"2024-04-26T20:56:51.722274+0000","last_unstale":"2024-04-26T20:56:53.145333+0000","last_undegraded":"2024-04-26T20:56:53.145333+0000","last_fullsized":"2024-04-26T20:56:53.145333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.428432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716895+0000","last_change":"2024-04-26T20:56:51.712244+0000","last_active":"2024-04-26T20:56:52.716895+0000","last_peered":"2024-04-26T20:56:52.716895+0000","last_clean":"2024-04-26T20:56:52.716895+0000","last_became_active":"2024-04-26T20:56:51.711957+0000","last_became_peered":"2024-04-26T20:56:51.711957+0000","last_unstale":"2024-04-26T20:56:52.716895+0000","last_undegraded":"2024-04-26T20:56:52.716895+0000","last_fullsized":"2024-04-26T20:56:52.716895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:22.643252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225198+0000","last_change":"2024-04-26T20:56:51.721797+0000","last_active":"2024-04-26T20:56:53.225198+0000","last_peered":"2024-04-26T20:56:53.225198+0000","last_clean":"2024-04-26T20:56:53.225198+0000","last_became_active":"2024-04-26T20:56:51.721465+0000","last_became_peered":"2024-04-26T20:56:51.721465+0000","last_unstale":"2024-04-26T20:56:53.225198+0000","last_undegraded":"2024-04-26T20:56:53.225198+0000","last_fullsized":"2024-04-26T20:56:53.225198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:26.791119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.729925+0000","last_change":"2024-04-26T20:56:51.716527+0000","last_active":"2024-04-26T20:56:52.729925+0000","last_peered":"2024-04-26T20:56:52.729925+0000","last_clean":"2024-04-26T20:56:52.729925+0000","last_became_active":"2024-04-26T20:56:51.716103+0000","last_became_peered":"2024-04-26T20:56:51.716103+0000","last_unstale":"2024-04-26T20:56:52.729925+0000","last_undegraded":"2024-04-26T20:56:52.729925+0000","last_fullsized":"2024-04-26T20:56:52.729925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:53.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.714884+0000","last_change":"2024-04-26T20:56:51.713515+0000","last_active":"2024-04-26T20:56:52.714884+0000","last_peered":"2024-04-26T20:56:52.714884+0000","last_clean":"2024-04-26T20:56:52.714884+0000","last_became_active":"2024-04-26T20:56:51.710208+0000","last_became_peered":"2024-04-26T20:56:51.710208+0000","last_unstale":"2024-04-26T20:56:52.714884+0000","last_undegraded":"2024-04-26T20:56:52.714884+0000","last_fullsized":"2024-04-26T20:56:52.714884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:51.540923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226969+0000","last_change":"2024-04-26T20:56:50.692216+0000","last_active":"2024-04-26T20:56:53.226969+0000","last_peered":"2024-04-26T20:56:53.226969+0000","last_clean":"2024-04-26T20:56:53.226969+0000","last_became_active":"2024-04-26T20:56:50.691457+0000","last_became_peered":"2024-04-26T20:56:50.691457+0000","last_unstale":"2024-04-26T20:56:53.226969+0000","last_undegraded":"2024-04-26T20:56:53.226969+0000","last_fullsized":"2024-04-26T20:56:53.226969+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:56:50.477216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.145297+0000","last_change":"2024-04-26T20:56:51.730925+0000","last_active":"2024-04-26T20:56:53.145297+0000","last_peered":"2024-04-26T20:56:53.145297+0000","last_clean":"2024-04-26T20:56:53.145297+0000","last_became_active":"2024-04-26T20:56:51.730497+0000","last_became_peered":"2024-04-26T20:56:51.730497+0000","last_unstale":"2024-04-26T20:56:53.145297+0000","last_undegraded":"2024-04-26T20:56:53.145297+0000","last_fullsized":"2024-04-26T20:56:53.145297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:37.778325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.729866+0000","last_change":"2024-04-26T20:56:50.684303+0000","last_active":"2024-04-26T20:56:52.729866+0000","last_peered":"2024-04-26T20:56:52.729866+0000","last_clean":"2024-04-26T20:56:52.729866+0000","last_became_active":"2024-04-26T20:56:50.684006+0000","last_became_peered":"2024-04-26T20:56:50.684006+0000","last_unstale":"2024-04-26T20:56:52.729866+0000","last_undegraded":"2024-04-26T20:56:52.729866+0000","last_fullsized":"2024-04-26T20:56:52.729866+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:50.526509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226111+0000","last_change":"2024-04-26T20:56:50.688365+0000","last_active":"2024-04-26T20:56:53.226111+0000","last_peered":"2024-04-26T20:56:53.226111+0000","last_clean":"2024-04-26T20:56:53.226111+0000","last_became_active":"2024-04-26T20:56:50.688034+0000","last_became_peered":"2024-04-26T20:56:50.688034+0000","last_unstale":"2024-04-26T20:56:53.226111+0000","last_undegraded":"2024-04-26T20:56:53.226111+0000","last_fullsized":"2024-04-26T20:56:53.226111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:14.594594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226190+0000","last_change":"2024-04-26T20:56:51.721954+0000","last_active":"2024-04-26T20:56:53.226190+0000","last_peered":"2024-04-26T20:56:53.226190+0000","last_clean":"2024-04-26T20:56:53.226190+0000","last_became_active":"2024-04-26T20:56:51.721650+0000","last_became_peered":"2024-04-26T20:56:51.721650+0000","last_unstale":"2024-04-26T20:56:53.226190+0000","last_undegraded":"2024-04-26T20:56:53.226190+0000","last_fullsized":"2024-04-26T20:56:53.226190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:01.617412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716559+0000","last_change":"2024-04-26T20:56:50.680262+0000","last_active":"2024-04-26T20:56:52.716559+0000","last_peered":"2024-04-26T20:56:52.716559+0000","last_clean":"2024-04-26T20:56:52.716559+0000","last_became_active":"2024-04-26T20:56:50.675501+0000","last_became_peered":"2024-04-26T20:56:50.675501+0000","last_unstale":"2024-04-26T20:56:52.716559+0000","last_undegraded":"2024-04-26T20:56:52.716559+0000","last_fullsized":"2024-04-26T20:56:52.716559+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:31.999414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713009+0000","last_change":"2024-04-26T20:56:51.714659+0000","last_active":"2024-04-26T20:56:52.713009+0000","last_peered":"2024-04-26T20:56:52.713009+0000","last_clean":"2024-04-26T20:56:52.713009+0000","last_became_active":"2024-04-26T20:56:51.714340+0000","last_became_peered":"2024-04-26T20:56:51.714340+0000","last_unstale":"2024-04-26T20:56:52.713009+0000","last_undegraded":"2024-04-26T20:56:52.713009+0000","last_fullsized":"2024-04-26T20:56:52.713009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:53.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730521+0000","last_change":"2024-04-26T20:56:50.689246+0000","last_active":"2024-04-26T20:56:52.730521+0000","last_peered":"2024-04-26T20:56:52.730521+0000","last_clean":"2024-04-26T20:56:52.730521+0000","last_became_active":"2024-04-26T20:56:50.688882+0000","last_became_peered":"2024-04-26T20:56:50.688882+0000","last_unstale":"2024-04-26T20:56:52.730521+0000","last_undegraded":"2024-04-26T20:56:52.730521+0000","last_fullsized":"2024-04-26T20:56:52.730521+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:39.328210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.718004+0000","last_change":"2024-04-26T20:56:51.737626+0000","last_active":"2024-04-26T20:56:52.718004+0000","last_peered":"2024-04-26T20:56:52.718004+0000","last_clean":"2024-04-26T20:56:52.718004+0000","last_became_active":"2024-04-26T20:56:51.737273+0000","last_became_peered":"2024-04-26T20:56:51.737273+0000","last_unstale":"2024-04-26T20:56:52.718004+0000","last_undegraded":"2024-04-26T20:56:52.718004+0000","last_fullsized":"2024-04-26T20:56:52.718004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:33.950624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.718371+0000","last_change":"2024-04-26T20:56:50.690573+0000","last_active":"2024-04-26T20:56:52.718371+0000","last_peered":"2024-04-26T20:56:52.718371+0000","last_clean":"2024-04-26T20:56:52.718371+0000","last_became_active":"2024-04-26T20:56:50.690198+0000","last_became_peered":"2024-04-26T20:56:50.690198+0000","last_unstale":"2024-04-26T20:56:52.718371+0000","last_undegraded":"2024-04-26T20:56:52.718371+0000","last_fullsized":"2024-04-26T20:56:52.718371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:25.776704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144843+0000","last_change":"2024-04-26T20:56:51.733136+0000","last_active":"2024-04-26T20:56:53.144843+0000","last_peered":"2024-04-26T20:56:53.144843+0000","last_clean":"2024-04-26T20:56:53.144843+0000","last_became_active":"2024-04-26T20:56:51.732782+0000","last_became_peered":"2024-04-26T20:56:51.732782+0000","last_unstale":"2024-04-26T20:56:53.144843+0000","last_undegraded":"2024-04-26T20:56:53.144843+0000","last_fullsized":"2024-04-26T20:56:53.144843+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:16.275518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713411+0000","last_change":"2024-04-26T20:56:50.689210+0000","last_active":"2024-04-26T20:56:52.713411+0000","last_peered":"2024-04-26T20:56:52.713411+0000","last_clean":"2024-04-26T20:56:52.713411+0000","last_became_active":"2024-04-26T20:56:50.688655+0000","last_became_peered":"2024-04-26T20:56:50.688655+0000","last_unstale":"2024-04-26T20:56:52.713411+0000","last_undegraded":"2024-04-26T20:56:52.713411+0000","last_fullsized":"2024-04-26T20:56:52.713411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:25.776704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.715628+0000","last_change":"2024-04-26T20:56:51.737157+0000","last_active":"2024-04-26T20:56:52.715628+0000","last_peered":"2024-04-26T20:56:52.715628+0000","last_clean":"2024-04-26T20:56:52.715628+0000","last_became_active":"2024-04-26T20:56:51.736082+0000","last_became_peered":"2024-04-26T20:56:51.736082+0000","last_unstale":"2024-04-26T20:56:52.715628+0000","last_undegraded":"2024-04-26T20:56:52.715628+0000","last_fullsized":"2024-04-26T20:56:52.715628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:40.342624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730295+0000","last_change":"2024-04-26T20:56:50.695225+0000","last_active":"2024-04-26T20:56:52.730295+0000","last_peered":"2024-04-26T20:56:52.730295+0000","last_clean":"2024-04-26T20:56:52.730295+0000","last_became_active":"2024-04-26T20:56:50.694892+0000","last_became_peered":"2024-04-26T20:56:50.694892+0000","last_unstale":"2024-04-26T20:56:52.730295+0000","last_undegraded":"2024-04-26T20:56:52.730295+0000","last_fullsized":"2024-04-26T20:56:52.730295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:41.662331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153191+0000","last_change":"2024-04-26T20:56:51.722030+0000","last_active":"2024-04-26T20:56:53.153191+0000","last_peered":"2024-04-26T20:56:53.153191+0000","last_clean":"2024-04-26T20:56:53.153191+0000","last_became_active":"2024-04-26T20:56:51.721724+0000","last_became_peered":"2024-04-26T20:56:51.721724+0000","last_unstale":"2024-04-26T20:56:53.153191+0000","last_undegraded":"2024-04-26T20:56:53.153191+0000","last_fullsized":"2024-04-26T20:56:53.153191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:52.330678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.145449+0000","last_change":"2024-04-26T20:56:50.682167+0000","last_active":"2024-04-26T20:56:53.145449+0000","last_peered":"2024-04-26T20:56:53.145449+0000","last_clean":"2024-04-26T20:56:53.145449+0000","last_became_active":"2024-04-26T20:56:50.675090+0000","last_became_peered":"2024-04-26T20:56:50.675090+0000","last_unstale":"2024-04-26T20:56:53.145449+0000","last_undegraded":"2024-04-26T20:56:53.145449+0000","last_fullsized":"2024-04-26T20:56:53.145449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:03.566375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.145134+0000","last_change":"2024-04-26T20:56:51.736362+0000","last_active":"2024-04-26T20:56:53.145134+0000","last_peered":"2024-04-26T20:56:53.145134+0000","last_clean":"2024-04-26T20:56:53.145134+0000","last_became_active":"2024-04-26T20:56:51.736050+0000","last_became_peered":"2024-04-26T20:56:51.736050+0000","last_unstale":"2024-04-26T20:56:53.145134+0000","last_undegraded":"2024-04-26T20:56:53.145134+0000","last_fullsized":"2024-04-26T20:56:53.145134+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:24.679612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717056+0000","last_change":"2024-04-26T20:56:50.683775+0000","last_active":"2024-04-26T20:56:52.717056+0000","last_peered":"2024-04-26T20:56:52.717056+0000","last_clean":"2024-04-26T20:56:52.717056+0000","last_became_active":"2024-04-26T20:56:50.683470+0000","last_became_peered":"2024-04-26T20:56:50.683470+0000","last_unstale":"2024-04-26T20:56:52.717056+0000","last_undegraded":"2024-04-26T20:56:52.717056+0000","last_fullsized":"2024-04-26T20:56:52.717056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:50.526509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153154+0000","last_change":"2024-04-26T20:56:51.730365+0000","last_active":"2024-04-26T20:56:53.153154+0000","last_peered":"2024-04-26T20:56:53.153154+0000","last_clean":"2024-04-26T20:56:53.153154+0000","last_became_active":"2024-04-26T20:56:51.730015+0000","last_became_peered":"2024-04-26T20:56:51.730015+0000","last_unstale":"2024-04-26T20:56:53.153154+0000","last_undegraded":"2024-04-26T20:56:53.153154+0000","last_fullsized":"2024-04-26T20:56:53.153154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:22.643252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.729839+0000","last_change":"2024-04-26T20:56:50.686055+0000","last_active":"2024-04-26T20:56:52.729839+0000","last_peered":"2024-04-26T20:56:52.729839+0000","last_clean":"2024-04-26T20:56:52.729839+0000","last_became_active":"2024-04-26T20:56:50.685771+0000","last_became_peered":"2024-04-26T20:56:50.685771+0000","last_unstale":"2024-04-26T20:56:52.729839+0000","last_undegraded":"2024-04-26T20:56:52.729839+0000","last_fullsized":"2024-04-26T20:56:52.729839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:31.999414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.729780+0000","last_change":"2024-04-26T20:56:51.719273+0000","last_active":"2024-04-26T20:56:52.729780+0000","last_peered":"2024-04-26T20:56:52.729780+0000","last_clean":"2024-04-26T20:56:52.729780+0000","last_became_active":"2024-04-26T20:56:51.718659+0000","last_became_peered":"2024-04-26T20:56:51.718659+0000","last_unstale":"2024-04-26T20:56:52.729780+0000","last_undegraded":"2024-04-26T20:56:52.729780+0000","last_fullsized":"2024-04-26T20:56:52.729780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:16.275518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225397+0000","last_change":"2024-04-26T20:56:50.686714+0000","last_active":"2024-04-26T20:56:53.225397+0000","last_peered":"2024-04-26T20:56:53.225397+0000","last_clean":"2024-04-26T20:56:53.225397+0000","last_became_active":"2024-04-26T20:56:50.686348+0000","last_became_peered":"2024-04-26T20:56:50.686348+0000","last_unstale":"2024-04-26T20:56:53.225397+0000","last_undegraded":"2024-04-26T20:56:53.225397+0000","last_fullsized":"2024-04-26T20:56:53.225397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:12:14.594594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.152873+0000","last_change":"2024-04-26T20:56:51.730639+0000","last_active":"2024-04-26T20:56:53.152873+0000","last_peered":"2024-04-26T20:56:53.152873+0000","last_clean":"2024-04-26T20:56:53.152873+0000","last_became_active":"2024-04-26T20:56:51.730196+0000","last_became_peered":"2024-04-26T20:56:51.730196+0000","last_unstale":"2024-04-26T20:56:53.152873+0000","last_undegraded":"2024-04-26T20:56:53.152873+0000","last_fullsized":"2024-04-26T20:56:53.152873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:47.662176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.715683+0000","last_change":"2024-04-26T20:56:50.685016+0000","last_active":"2024-04-26T20:56:52.715683+0000","last_peered":"2024-04-26T20:56:52.715683+0000","last_clean":"2024-04-26T20:56:52.715683+0000","last_became_active":"2024-04-26T20:56:50.684679+0000","last_became_peered":"2024-04-26T20:56:50.684679+0000","last_unstale":"2024-04-26T20:56:52.715683+0000","last_undegraded":"2024-04-26T20:56:52.715683+0000","last_fullsized":"2024-04-26T20:56:52.715683+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:07:34.811589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153972+0000","last_change":"2024-04-26T20:56:51.728717+0000","last_active":"2024-04-26T20:56:53.153972+0000","last_peered":"2024-04-26T20:56:53.153972+0000","last_clean":"2024-04-26T20:56:53.153972+0000","last_became_active":"2024-04-26T20:56:51.728261+0000","last_became_peered":"2024-04-26T20:56:51.728261+0000","last_unstale":"2024-04-26T20:56:53.153972+0000","last_undegraded":"2024-04-26T20:56:53.153972+0000","last_fullsized":"2024-04-26T20:56:53.153972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:02:31.238922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225598+0000","last_change":"2024-04-26T20:56:50.695251+0000","last_active":"2024-04-26T20:56:53.225598+0000","last_peered":"2024-04-26T20:56:53.225598+0000","last_clean":"2024-04-26T20:56:53.225598+0000","last_became_active":"2024-04-26T20:56:50.694956+0000","last_became_peered":"2024-04-26T20:56:50.694956+0000","last_unstale":"2024-04-26T20:56:53.225598+0000","last_undegraded":"2024-04-26T20:56:53.225598+0000","last_fullsized":"2024-04-26T20:56:53.225598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:22.304242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153932+0000","last_change":"2024-04-26T20:56:51.736963+0000","last_active":"2024-04-26T20:56:53.153932+0000","last_peered":"2024-04-26T20:56:53.153932+0000","last_clean":"2024-04-26T20:56:53.153932+0000","last_became_active":"2024-04-26T20:56:51.734437+0000","last_became_peered":"2024-04-26T20:56:51.734437+0000","last_unstale":"2024-04-26T20:56:53.153932+0000","last_undegraded":"2024-04-26T20:56:53.153932+0000","last_fullsized":"2024-04-26T20:56:53.153932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:08.679353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713517+0000","last_change":"2024-04-26T20:56:50.684034+0000","last_active":"2024-04-26T20:56:52.713517+0000","last_peered":"2024-04-26T20:56:52.713517+0000","last_clean":"2024-04-26T20:56:52.713517+0000","last_became_active":"2024-04-26T20:56:50.683720+0000","last_became_peered":"2024-04-26T20:56:50.683720+0000","last_unstale":"2024-04-26T20:56:52.713517+0000","last_undegraded":"2024-04-26T20:56:52.713517+0000","last_fullsized":"2024-04-26T20:56:52.713517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:46.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717068+0000","last_change":"2024-04-26T20:56:51.725602+0000","last_active":"2024-04-26T20:56:52.717068+0000","last_peered":"2024-04-26T20:56:52.717068+0000","last_clean":"2024-04-26T20:56:52.717068+0000","last_became_active":"2024-04-26T20:56:51.725308+0000","last_became_peered":"2024-04-26T20:56:51.725308+0000","last_unstale":"2024-04-26T20:56:52.717068+0000","last_undegraded":"2024-04-26T20:56:52.717068+0000","last_fullsized":"2024-04-26T20:56:52.717068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:52.330678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225216+0000","last_change":"2024-04-26T20:56:50.688464+0000","last_active":"2024-04-26T20:56:53.225216+0000","last_peered":"2024-04-26T20:56:53.225216+0000","last_clean":"2024-04-26T20:56:53.225216+0000","last_became_active":"2024-04-26T20:56:50.688101+0000","last_became_peered":"2024-04-26T20:56:50.688101+0000","last_unstale":"2024-04-26T20:56:53.225216+0000","last_undegraded":"2024-04-26T20:56:53.225216+0000","last_fullsized":"2024-04-26T20:56:53.225216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:50.526509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153386+0000","last_change":"2024-04-26T20:56:51.734773+0000","last_active":"2024-04-26T20:56:53.153386+0000","last_peered":"2024-04-26T20:56:53.153386+0000","last_clean":"2024-04-26T20:56:53.153386+0000","last_became_active":"2024-04-26T20:56:51.733903+0000","last_became_peered":"2024-04-26T20:56:51.733903+0000","last_unstale":"2024-04-26T20:56:53.153386+0000","last_undegraded":"2024-04-26T20:56:53.153386+0000","last_fullsized":"2024-04-26T20:56:53.153386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:23.450962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.152300+0000","last_change":"2024-04-26T20:56:46.848603+0000","last_active":"2024-04-26T20:56:53.152300+0000","last_peered":"2024-04-26T20:56:53.152300+0000","last_clean":"2024-04-26T20:56:53.152300+0000","last_became_active":"2024-04-26T20:56:46.846091+0000","last_became_peered":"2024-04-26T20:56:46.846091+0000","last_unstale":"2024-04-26T20:56:53.152300+0000","last_undegraded":"2024-04-26T20:56:53.152300+0000","last_fullsized":"2024-04-26T20:56:53.152300+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_clean_scrub_stamp":"2024-04-26T20:56:45.594041+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:00.232387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.716618+0000","last_change":"2024-04-26T20:56:50.681942+0000","last_active":"2024-04-26T20:56:52.716618+0000","last_peered":"2024-04-26T20:56:52.716618+0000","last_clean":"2024-04-26T20:56:52.716618+0000","last_became_active":"2024-04-26T20:56:50.681645+0000","last_became_peered":"2024-04-26T20:56:50.681645+0000","last_unstale":"2024-04-26T20:56:52.716618+0000","last_undegraded":"2024-04-26T20:56:52.716618+0000","last_fullsized":"2024-04-26T20:56:52.716618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:39.328210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226904+0000","last_change":"2024-04-26T20:56:51.730048+0000","last_active":"2024-04-26T20:56:53.226904+0000","last_peered":"2024-04-26T20:56:53.226904+0000","last_clean":"2024-04-26T20:56:53.226904+0000","last_became_active":"2024-04-26T20:56:51.729733+0000","last_became_peered":"2024-04-26T20:56:51.729733+0000","last_unstale":"2024-04-26T20:56:53.226904+0000","last_undegraded":"2024-04-26T20:56:53.226904+0000","last_fullsized":"2024-04-26T20:56:53.226904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:24.679612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.152261+0000","last_change":"2024-04-26T20:56:50.699113+0000","last_active":"2024-04-26T20:56:53.152261+0000","last_peered":"2024-04-26T20:56:53.152261+0000","last_clean":"2024-04-26T20:56:53.152261+0000","last_became_active":"2024-04-26T20:56:50.698744+0000","last_became_peered":"2024-04-26T20:56:50.698744+0000","last_unstale":"2024-04-26T20:56:53.152261+0000","last_undegraded":"2024-04-26T20:56:53.152261+0000","last_fullsized":"2024-04-26T20:56:53.152261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:11.694549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144333+0000","last_change":"2024-04-26T20:56:51.732556+0000","last_active":"2024-04-26T20:56:53.144333+0000","last_peered":"2024-04-26T20:56:53.144333+0000","last_clean":"2024-04-26T20:56:53.144333+0000","last_became_active":"2024-04-26T20:56:51.732197+0000","last_became_peered":"2024-04-26T20:56:51.732197+0000","last_unstale":"2024-04-26T20:56:53.144333+0000","last_undegraded":"2024-04-26T20:56:53.144333+0000","last_fullsized":"2024-04-26T20:56:53.144333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:27:15.978359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.225098+0000","last_change":"2024-04-26T20:56:50.694069+0000","last_active":"2024-04-26T20:56:53.225098+0000","last_peered":"2024-04-26T20:56:53.225098+0000","last_clean":"2024-04-26T20:56:53.225098+0000","last_became_active":"2024-04-26T20:56:50.693779+0000","last_became_peered":"2024-04-26T20:56:50.693779+0000","last_unstale":"2024-04-26T20:56:53.225098+0000","last_undegraded":"2024-04-26T20:56:53.225098+0000","last_fullsized":"2024-04-26T20:56:53.225098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:31.999414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.714833+0000","last_change":"2024-04-26T20:56:51.710866+0000","last_active":"2024-04-26T20:56:52.714833+0000","last_peered":"2024-04-26T20:56:52.714833+0000","last_clean":"2024-04-26T20:56:52.714833+0000","last_became_active":"2024-04-26T20:56:51.709325+0000","last_became_peered":"2024-04-26T20:56:51.709325+0000","last_unstale":"2024-04-26T20:56:52.714833+0000","last_undegraded":"2024-04-26T20:56:52.714833+0000","last_fullsized":"2024-04-26T20:56:52.714833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:23.268496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717176+0000","last_change":"2024-04-26T20:56:50.698231+0000","last_active":"2024-04-26T20:56:52.717176+0000","last_peered":"2024-04-26T20:56:52.717176+0000","last_clean":"2024-04-26T20:56:52.717176+0000","last_became_active":"2024-04-26T20:56:50.697872+0000","last_became_peered":"2024-04-26T20:56:50.697872+0000","last_unstale":"2024-04-26T20:56:52.717176+0000","last_undegraded":"2024-04-26T20:56:52.717176+0000","last_fullsized":"2024-04-26T20:56:52.717176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:41.662331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.715920+0000","last_change":"2024-04-26T20:56:51.720748+0000","last_active":"2024-04-26T20:56:52.715920+0000","last_peered":"2024-04-26T20:56:52.715920+0000","last_clean":"2024-04-26T20:56:52.715920+0000","last_became_active":"2024-04-26T20:56:51.720478+0000","last_became_peered":"2024-04-26T20:56:51.720478+0000","last_unstale":"2024-04-26T20:56:52.715920+0000","last_undegraded":"2024-04-26T20:56:52.715920+0000","last_fullsized":"2024-04-26T20:56:52.715920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:26.791119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713396+0000","last_change":"2024-04-26T20:56:50.685444+0000","last_active":"2024-04-26T20:56:52.713396+0000","last_peered":"2024-04-26T20:56:52.713396+0000","last_clean":"2024-04-26T20:56:52.713396+0000","last_became_active":"2024-04-26T20:56:50.684937+0000","last_became_peered":"2024-04-26T20:56:50.684937+0000","last_unstale":"2024-04-26T20:56:52.713396+0000","last_undegraded":"2024-04-26T20:56:52.713396+0000","last_fullsized":"2024-04-26T20:56:52.713396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:39.328210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717440+0000","last_change":"2024-04-26T20:56:51.727132+0000","last_active":"2024-04-26T20:56:52.717440+0000","last_peered":"2024-04-26T20:56:52.717440+0000","last_clean":"2024-04-26T20:56:52.717440+0000","last_became_active":"2024-04-26T20:56:51.720623+0000","last_became_peered":"2024-04-26T20:56:51.720623+0000","last_unstale":"2024-04-26T20:56:52.717440+0000","last_undegraded":"2024-04-26T20:56:52.717440+0000","last_fullsized":"2024-04-26T20:56:52.717440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:47:06.270681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153818+0000","last_change":"2024-04-26T20:56:50.693929+0000","last_active":"2024-04-26T20:56:53.153818+0000","last_peered":"2024-04-26T20:56:53.153818+0000","last_clean":"2024-04-26T20:56:53.153818+0000","last_became_active":"2024-04-26T20:56:50.693576+0000","last_became_peered":"2024-04-26T20:56:50.693576+0000","last_unstale":"2024-04-26T20:56:53.153818+0000","last_undegraded":"2024-04-26T20:56:53.153818+0000","last_fullsized":"2024-04-26T20:56:53.153818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:28:46.302482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713243+0000","last_change":"2024-04-26T20:56:51.714289+0000","last_active":"2024-04-26T20:56:52.713243+0000","last_peered":"2024-04-26T20:56:52.713243+0000","last_clean":"2024-04-26T20:56:52.713243+0000","last_became_active":"2024-04-26T20:56:51.713970+0000","last_became_peered":"2024-04-26T20:56:51.713970+0000","last_unstale":"2024-04-26T20:56:52.713243+0000","last_undegraded":"2024-04-26T20:56:52.713243+0000","last_fullsized":"2024-04-26T20:56:52.713243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:09:24.679612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.728669+0000","last_change":"2024-04-26T20:56:50.690572+0000","last_active":"2024-04-26T20:56:52.728669+0000","last_peered":"2024-04-26T20:56:52.728669+0000","last_clean":"2024-04-26T20:56:52.728669+0000","last_became_active":"2024-04-26T20:56:50.690221+0000","last_became_peered":"2024-04-26T20:56:50.690221+0000","last_unstale":"2024-04-26T20:56:52.728669+0000","last_undegraded":"2024-04-26T20:56:52.728669+0000","last_fullsized":"2024-04-26T20:56:52.728669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:11.694549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.227813+0000","last_change":"2024-04-26T20:56:51.740978+0000","last_active":"2024-04-26T20:56:53.227813+0000","last_peered":"2024-04-26T20:56:53.227813+0000","last_clean":"2024-04-26T20:56:53.227813+0000","last_became_active":"2024-04-26T20:56:51.739561+0000","last_became_peered":"2024-04-26T20:56:51.739561+0000","last_unstale":"2024-04-26T20:56:53.227813+0000","last_undegraded":"2024-04-26T20:56:53.227813+0000","last_fullsized":"2024-04-26T20:56:53.227813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:01:47.662176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730672+0000","last_change":"2024-04-26T20:56:50.694545+0000","last_active":"2024-04-26T20:56:52.730672+0000","last_peered":"2024-04-26T20:56:52.730672+0000","last_clean":"2024-04-26T20:56:52.730672+0000","last_became_active":"2024-04-26T20:56:50.694189+0000","last_became_peered":"2024-04-26T20:56:50.694189+0000","last_unstale":"2024-04-26T20:56:52.730672+0000","last_undegraded":"2024-04-26T20:56:52.730672+0000","last_fullsized":"2024-04-26T20:56:52.730672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:25.776704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144945+0000","last_change":"2024-04-26T20:56:51.724210+0000","last_active":"2024-04-26T20:56:53.144945+0000","last_peered":"2024-04-26T20:56:53.144945+0000","last_clean":"2024-04-26T20:56:53.144945+0000","last_became_active":"2024-04-26T20:56:51.723882+0000","last_became_peered":"2024-04-26T20:56:51.723882+0000","last_unstale":"2024-04-26T20:56:53.144945+0000","last_undegraded":"2024-04-26T20:56:53.144945+0000","last_fullsized":"2024-04-26T20:56:53.144945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:35:53.133820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.152108+0000","last_change":"2024-04-26T20:56:50.694039+0000","last_active":"2024-04-26T20:56:53.152108+0000","last_peered":"2024-04-26T20:56:53.152108+0000","last_clean":"2024-04-26T20:56:53.152108+0000","last_became_active":"2024-04-26T20:56:50.693569+0000","last_became_peered":"2024-04-26T20:56:50.693569+0000","last_unstale":"2024-04-26T20:56:53.152108+0000","last_undegraded":"2024-04-26T20:56:53.152108+0000","last_fullsized":"2024-04-26T20:56:53.152108+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:41.662331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.714862+0000","last_change":"2024-04-26T20:56:51.719531+0000","last_active":"2024-04-26T20:56:52.714862+0000","last_peered":"2024-04-26T20:56:52.714862+0000","last_clean":"2024-04-26T20:56:52.714862+0000","last_became_active":"2024-04-26T20:56:51.714881+0000","last_became_peered":"2024-04-26T20:56:51.714881+0000","last_unstale":"2024-04-26T20:56:52.714862+0000","last_undegraded":"2024-04-26T20:56:52.714862+0000","last_fullsized":"2024-04-26T20:56:52.714862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:33.013829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713174+0000","last_change":"2024-04-26T20:56:50.689750+0000","last_active":"2024-04-26T20:56:52.713174+0000","last_peered":"2024-04-26T20:56:52.713174+0000","last_clean":"2024-04-26T20:56:52.713174+0000","last_became_active":"2024-04-26T20:56:50.689337+0000","last_became_peered":"2024-04-26T20:56:50.689337+0000","last_unstale":"2024-04-26T20:56:52.713174+0000","last_undegraded":"2024-04-26T20:56:52.713174+0000","last_fullsized":"2024-04-26T20:56:52.713174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:02:41.662331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226387+0000","last_change":"2024-04-26T20:56:51.712295+0000","last_active":"2024-04-26T20:56:53.226387+0000","last_peered":"2024-04-26T20:56:53.226387+0000","last_clean":"2024-04-26T20:56:53.226387+0000","last_became_active":"2024-04-26T20:56:51.711999+0000","last_became_peered":"2024-04-26T20:56:51.711999+0000","last_unstale":"2024-04-26T20:56:53.226387+0000","last_undegraded":"2024-04-26T20:56:53.226387+0000","last_fullsized":"2024-04-26T20:56:53.226387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:47:16.275518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.713352+0000","last_change":"2024-04-26T20:56:50.697160+0000","last_active":"2024-04-26T20:56:52.713352+0000","last_peered":"2024-04-26T20:56:52.713352+0000","last_clean":"2024-04-26T20:56:52.713352+0000","last_became_active":"2024-04-26T20:56:50.695595+0000","last_became_peered":"2024-04-26T20:56:50.695595+0000","last_unstale":"2024-04-26T20:56:52.713352+0000","last_undegraded":"2024-04-26T20:56:52.713352+0000","last_fullsized":"2024-04-26T20:56:52.713352+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:50.526509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.226926+0000","last_change":"2024-04-26T20:56:51.717706+0000","last_active":"2024-04-26T20:56:53.226926+0000","last_peered":"2024-04-26T20:56:53.226926+0000","last_clean":"2024-04-26T20:56:53.226926+0000","last_became_active":"2024-04-26T20:56:51.717422+0000","last_became_peered":"2024-04-26T20:56:51.717422+0000","last_unstale":"2024-04-26T20:56:53.226926+0000","last_undegraded":"2024-04-26T20:56:53.226926+0000","last_fullsized":"2024-04-26T20:56:53.226926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:37.778325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153433+0000","last_change":"2024-04-26T20:56:50.695179+0000","last_active":"2024-04-26T20:56:53.153433+0000","last_peered":"2024-04-26T20:56:53.153433+0000","last_clean":"2024-04-26T20:56:53.153433+0000","last_became_active":"2024-04-26T20:56:50.694889+0000","last_became_peered":"2024-04-26T20:56:50.694889+0000","last_unstale":"2024-04-26T20:56:53.153433+0000","last_undegraded":"2024-04-26T20:56:53.153433+0000","last_fullsized":"2024-04-26T20:56:53.153433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:50.526509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.730113+0000","last_change":"2024-04-26T20:56:51.713029+0000","last_active":"2024-04-26T20:56:52.730113+0000","last_peered":"2024-04-26T20:56:52.730113+0000","last_clean":"2024-04-26T20:56:52.730113+0000","last_became_active":"2024-04-26T20:56:51.712311+0000","last_became_peered":"2024-04-26T20:56:51.712311+0000","last_unstale":"2024-04-26T20:56:52.730113+0000","last_undegraded":"2024-04-26T20:56:52.730113+0000","last_fullsized":"2024-04-26T20:56:52.730113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:10:36.354765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717262+0000","last_change":"2024-04-26T20:56:51.730361+0000","last_active":"2024-04-26T20:56:52.717262+0000","last_peered":"2024-04-26T20:56:52.717262+0000","last_clean":"2024-04-26T20:56:52.717262+0000","last_became_active":"2024-04-26T20:56:51.730068+0000","last_became_peered":"2024-04-26T20:56:51.730068+0000","last_unstale":"2024-04-26T20:56:52.717262+0000","last_undegraded":"2024-04-26T20:56:52.717262+0000","last_fullsized":"2024-04-26T20:56:52.717262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:07.828489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153083+0000","last_change":"2024-04-26T20:56:50.696973+0000","last_active":"2024-04-26T20:56:53.153083+0000","last_peered":"2024-04-26T20:56:53.153083+0000","last_clean":"2024-04-26T20:56:53.153083+0000","last_became_active":"2024-04-26T20:56:50.696603+0000","last_became_peered":"2024-04-26T20:56:50.696603+0000","last_unstale":"2024-04-26T20:56:53.153083+0000","last_undegraded":"2024-04-26T20:56:53.153083+0000","last_fullsized":"2024-04-26T20:56:53.153083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:31:25.776704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:52.717108+0000","last_change":"2024-04-26T20:56:51.728709+0000","last_active":"2024-04-26T20:56:52.717108+0000","last_peered":"2024-04-26T20:56:52.717108+0000","last_clean":"2024-04-26T20:56:52.717108+0000","last_became_active":"2024-04-26T20:56:51.727858+0000","last_became_peered":"2024-04-26T20:56:51.727858+0000","last_unstale":"2024-04-26T20:56:52.717108+0000","last_undegraded":"2024-04-26T20:56:52.717108+0000","last_fullsized":"2024-04-26T20:56:52.717108+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:39:23.450962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.153531+0000","last_change":"2024-04-26T20:56:50.696966+0000","last_active":"2024-04-26T20:56:53.153531+0000","last_peered":"2024-04-26T20:56:53.153531+0000","last_clean":"2024-04-26T20:56:53.153531+0000","last_became_active":"2024-04-26T20:56:50.696448+0000","last_became_peered":"2024-04-26T20:56:50.696448+0000","last_unstale":"2024-04-26T20:56:53.153531+0000","last_undegraded":"2024-04-26T20:56:53.153531+0000","last_fullsized":"2024-04-26T20:56:53.153531+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:39.328210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144675+0000","last_change":"2024-04-26T20:56:51.731938+0000","last_active":"2024-04-26T20:56:53.144675+0000","last_peered":"2024-04-26T20:56:53.144675+0000","last_clean":"2024-04-26T20:56:53.144675+0000","last_became_active":"2024-04-26T20:56:51.731655+0000","last_became_peered":"2024-04-26T20:56:51.731655+0000","last_unstale":"2024-04-26T20:56:53.144675+0000","last_undegraded":"2024-04-26T20:56:53.144675+0000","last_fullsized":"2024-04-26T20:56:53.144675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:08:18.631055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.227678+0000","last_change":"2024-04-26T20:56:50.692059+0000","last_active":"2024-04-26T20:56:53.227678+0000","last_peered":"2024-04-26T20:56:53.227678+0000","last_clean":"2024-04-26T20:56:53.227678+0000","last_became_active":"2024-04-26T20:56:50.690628+0000","last_became_peered":"2024-04-26T20:56:50.690628+0000","last_unstale":"2024-04-26T20:56:53.227678+0000","last_undegraded":"2024-04-26T20:56:53.227678+0000","last_fullsized":"2024-04-26T20:56:53.227678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:00.125994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.227526+0000","last_change":"2024-04-26T20:56:51.719241+0000","last_active":"2024-04-26T20:56:53.227526+0000","last_peered":"2024-04-26T20:56:53.227526+0000","last_clean":"2024-04-26T20:56:53.227526+0000","last_became_active":"2024-04-26T20:56:51.718893+0000","last_became_peered":"2024-04-26T20:56:51.718893+0000","last_unstale":"2024-04-26T20:56:53.227526+0000","last_undegraded":"2024-04-26T20:56:53.227526+0000","last_fullsized":"2024-04-26T20:56:53.227526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:50.640766+0000","last_clean_scrub_stamp":"2024-04-26T20:56:50.640766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:51:23.268496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-04-26T20:56:53.144835+0000","last_change":"2024-04-26T20:56:50.691078+0000","last_active":"2024-04-26T20:56:53.144835+0000","last_peered":"2024-04-26T20:56:53.144835+0000","last_clean":"2024-04-26T20:56:53.144835+0000","last_became_active":"2024-04-26T20:56:50.689362+0000","last_became_peered":"2024-04-26T20:56:50.689362+0000","last_unstale":"2024-04-26T20:56:53.144835+0000","last_undegraded":"2024-04-26T20:56:53.144835+0000","last_fullsized":"2024-04-26T20:56:53.144835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:49.626352+0000","last_clean_scrub_stamp":"2024-04-26T20:56:49.626352+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:13:00.125994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:57:00.362 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T20:57:00.362 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T20:57:00.362 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-26T20:57:00.362 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T20:57:00.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.606+0000 7ff27a800640 1 Processor -- start 2024-04-26T20:57:00.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.606+0000 7ff27a800640 1 -- start start 2024-04-26T20:57:00.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.607+0000 7ff27a800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff2740d1110 0x7ff274073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:00.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.608+0000 7ff27a800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff27406ef40 0x7ff274163590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:00.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.608+0000 7ff27a800640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff27406dee0 con 0x7ff2740d14f0 2024-04-26T20:57:00.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.608+0000 7ff27a800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff27406e1a0 con 0x7ff27406ef40 2024-04-26T20:57:00.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.608+0000 7ff27a800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff27406e480 con 0x7ff2740d1110 2024-04-26T20:57:00.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.608+0000 7ff278a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff27406ef40 0x7ff274163590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:00.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.608+0000 7ff278a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff27406ef40 0x7ff274163590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35108/0 (socket says 172.21.15.28:35108) 2024-04-26T20:57:00.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.608+0000 7ff278a00640 1 -- 172.21.15.28:0/3078683587 learned_addr learned my addr 172.21.15.28:0/3078683587 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:00.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.608+0000 7ff278a00640 1 -- 172.21.15.28:0/3078683587 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff2740d1110 msgr2=0x7ff274073cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:00.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.608+0000 7ff279400640 1 --2- 172.21.15.28:0/3078683587 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff2740d1110 0x7ff274073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:00.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.608+0000 7ff278a00640 1 --2- 172.21.15.28:0/3078683587 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff2740d1110 0x7ff274073cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:00.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.608+0000 7ff278a00640 1 -- 172.21.15.28:0/3078683587 >> v1:172.21.15.103:6789/0 conn(0x7ff2740d14f0 legacy=0x7ff27415e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:00.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.608+0000 7ff278a00640 1 -- 172.21.15.28:0/3078683587 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27406e760 con 0x7ff27406ef40 2024-04-26T20:57:00.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.608+0000 7ff279400640 1 --2- 172.21.15.28:0/3078683587 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff2740d1110 0x7ff274073cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:00.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.609+0000 7ff278a00640 1 --2- 172.21.15.28:0/3078683587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff27406ef40 0x7ff274163590 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff260009870 tx=0x7ff26002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c4abd930cd7278 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:00.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.610+0000 7ff273e00640 1 -- 172.21.15.28:0/3078683587 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260037070 con 0x7ff27406ef40 2024-04-26T20:57:00.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.610+0000 7ff273e00640 1 -- 172.21.15.28:0/3078683587 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff260032440 con 0x7ff27406ef40 2024-04-26T20:57:00.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.610+0000 7ff273e00640 1 -- 172.21.15.28:0/3078683587 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260032810 con 0x7ff27406ef40 2024-04-26T20:57:00.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.610+0000 7ff27a800640 1 -- 172.21.15.28:0/3078683587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff27406ef40 msgr2=0x7ff274163590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:00.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.610+0000 7ff27a800640 1 --2- 172.21.15.28:0/3078683587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff27406ef40 0x7ff274163590 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff260009870 tx=0x7ff26002a210 comp rx=0 tx=0).stop 2024-04-26T20:57:00.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.610+0000 7ff27a800640 1 -- 172.21.15.28:0/3078683587 shutdown_connections 2024-04-26T20:57:00.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.611+0000 7ff27a800640 1 --2- 172.21.15.28:0/3078683587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff27406ef40 0x7ff274163590 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:00.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.611+0000 7ff27a800640 1 --2- 172.21.15.28:0/3078683587 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff2740d1110 0x7ff274073cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:00.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.611+0000 7ff27a800640 1 -- 172.21.15.28:0/3078683587 >> 172.21.15.28:0/3078683587 conn(0x7ff27406c770 msgr2=0x7ff27406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:00.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.611+0000 7ff27a800640 1 -- 172.21.15.28:0/3078683587 shutdown_connections 2024-04-26T20:57:00.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.611+0000 7ff27a800640 1 -- 172.21.15.28:0/3078683587 wait complete. 2024-04-26T20:57:00.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.612+0000 7ff27a800640 1 Processor -- start 2024-04-26T20:57:00.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.612+0000 7ff27a800640 1 -- start start 2024-04-26T20:57:00.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.613+0000 7ff27a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff27406ef40 0x7ff2740ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:00.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.613+0000 7ff27a800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff2740d1110 0x7ff274161d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:00.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.613+0000 7ff27a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2740d14f0 0x7ff27415ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:00.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.613+0000 7ff27a800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff27415f210 con 0x7ff27406ef40 2024-04-26T20:57:00.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.613+0000 7ff27a800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff27415f090 con 0x7ff2740d14f0 2024-04-26T20:57:00.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.613+0000 7ff27a800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff27415f390 con 0x7ff2740d1110 2024-04-26T20:57:00.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.614+0000 7ff279400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff27406ef40 0x7ff2740ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:00.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.614+0000 7ff279400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff27406ef40 0x7ff2740ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35112/0 (socket says 172.21.15.28:35112) 2024-04-26T20:57:00.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.614+0000 7ff279400640 1 -- 172.21.15.28:0/2390624116 learned_addr learned my addr 172.21.15.28:0/2390624116 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:00.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.614+0000 7ff278a00640 1 --2- 172.21.15.28:0/2390624116 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff2740d1110 0x7ff274161d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:00.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.614+0000 7ff279e00640 1 --2- 172.21.15.28:0/2390624116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2740d14f0 0x7ff27415ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:00.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.614+0000 7ff279400640 1 -- 172.21.15.28:0/2390624116 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff2740d1110 msgr2=0x7ff274161d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:00.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.614+0000 7ff279400640 1 --2- 172.21.15.28:0/2390624116 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff2740d1110 0x7ff274161d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:00.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.614+0000 7ff279400640 1 -- 172.21.15.28:0/2390624116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2740d14f0 msgr2=0x7ff27415ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:00.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.614+0000 7ff279400640 1 --2- 172.21.15.28:0/2390624116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2740d14f0 0x7ff27415ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:00.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.614+0000 7ff279400640 1 -- 172.21.15.28:0/2390624116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff274160330 con 0x7ff27406ef40 2024-04-26T20:57:00.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.614+0000 7ff279e00640 1 --2- 172.21.15.28:0/2390624116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2740d14f0 0x7ff27415ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:00.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.614+0000 7ff279400640 1 --2- 172.21.15.28:0/2390624116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff27406ef40 0x7ff2740ccca0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff26800c820 tx=0x7ff26800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:00.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.615+0000 7ff271600640 1 -- 172.21.15.28:0/2390624116 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268015070 con 0x7ff27406ef40 2024-04-26T20:57:00.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.615+0000 7ff271600640 1 -- 172.21.15.28:0/2390624116 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff268007a30 con 0x7ff27406ef40 2024-04-26T20:57:00.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.615+0000 7ff271600640 1 -- 172.21.15.28:0/2390624116 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268007e00 con 0x7ff27406ef40 2024-04-26T20:57:00.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.615+0000 7ff27a800640 1 -- 172.21.15.28:0/2390624116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff274160610 con 0x7ff27406ef40 2024-04-26T20:57:00.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.615+0000 7ff27a800640 1 -- 172.21.15.28:0/2390624116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff274156620 con 0x7ff27406ef40 2024-04-26T20:57:00.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.616+0000 7ff27a800640 1 -- 172.21.15.28:0/2390624116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff27406a360 con 0x7ff27406ef40 2024-04-26T20:57:00.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.617+0000 7ff271600640 1 -- 172.21.15.28:0/2390624116 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff268018070 con 0x7ff27406ef40 2024-04-26T20:57:00.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.618+0000 7ff271600640 1 --2- 172.21.15.28:0/2390624116 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff25406c8f0 0x7ff25406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:00.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.618+0000 7ff271600640 1 -- 172.21.15.28:0/2390624116 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff26808bb30 con 0x7ff27406ef40 2024-04-26T20:57:00.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.618+0000 7ff278a00640 1 --2- 172.21.15.28:0/2390624116 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff25406c8f0 0x7ff25406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:00.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.618+0000 7ff278a00640 1 --2- 172.21.15.28:0/2390624116 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff25406c8f0 0x7ff25406edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff260034000 tx=0x7ff26002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:00.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.622+0000 7ff271600640 1 -- 172.21.15.28:0/2390624116 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff268059f50 con 0x7ff27406ef40 2024-04-26T20:57:00.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.860+0000 7ff27a800640 1 -- 172.21.15.28:0/2390624116 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff2740d0530 con 0x7ff27406ef40 2024-04-26T20:57:00.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.860+0000 7ff271600640 1 -- 172.21.15.28:0/2390624116 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff268059850 con 0x7ff27406ef40 2024-04-26T20:57:00.862 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:00.862 INFO:teuthology.orchestra.run.smithi028.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T20:57:00.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.863+0000 7ff27a800640 1 -- 172.21.15.28:0/2390624116 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff25406c8f0 msgr2=0x7ff25406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:00.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.863+0000 7ff27a800640 1 --2- 172.21.15.28:0/2390624116 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff25406c8f0 0x7ff25406edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff260034000 tx=0x7ff26002e000 comp rx=0 tx=0).stop 2024-04-26T20:57:00.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.863+0000 7ff27a800640 1 -- 172.21.15.28:0/2390624116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff27406ef40 msgr2=0x7ff2740ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:00.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.863+0000 7ff27a800640 1 --2- 172.21.15.28:0/2390624116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff27406ef40 0x7ff2740ccca0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff26800c820 tx=0x7ff26800cc20 comp rx=0 tx=0).stop 2024-04-26T20:57:00.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.863+0000 7ff27a800640 1 -- 172.21.15.28:0/2390624116 shutdown_connections 2024-04-26T20:57:00.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.863+0000 7ff27a800640 1 --2- 172.21.15.28:0/2390624116 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff25406c8f0 0x7ff25406edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:00.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.863+0000 7ff27a800640 1 --2- 172.21.15.28:0/2390624116 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2740d14f0 0x7ff27415ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:00.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.863+0000 7ff27a800640 1 --2- 172.21.15.28:0/2390624116 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff2740d1110 0x7ff274161d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:00.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.863+0000 7ff27a800640 1 --2- 172.21.15.28:0/2390624116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff27406ef40 0x7ff2740ccca0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:00.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.863+0000 7ff27a800640 1 -- 172.21.15.28:0/2390624116 >> 172.21.15.28:0/2390624116 conn(0x7ff27406c770 msgr2=0x7ff2740b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:00.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.863+0000 7ff27a800640 1 -- 172.21.15.28:0/2390624116 shutdown_connections 2024-04-26T20:57:00.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:00.863+0000 7ff27a800640 1 -- 172.21.15.28:0/2390624116 wait complete. 2024-04-26T20:57:00.883 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-26T20:57:00.883 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:57:01.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.083+0000 7efe5ea00640 1 Processor -- start 2024-04-26T20:57:01.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.083+0000 7efe5ea00640 1 -- start start 2024-04-26T20:57:01.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.085+0000 7efe5ea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7efe58074050 0x7efe5816a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:01.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.085+0000 7efe5ea00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe580726d0 con 0x7efe58073770 2024-04-26T20:57:01.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.085+0000 7efe5ea00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe580729b0 con 0x7efe58145800 2024-04-26T20:57:01.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.085+0000 7efe5ea00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7efe58145be0 con 0x7efe58074050 2024-04-26T20:57:01.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.085+0000 7efe5ca00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7efe58074050 0x7efe5816a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:01.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.085+0000 7efe4f400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7efe58073770 0x7efe580aefd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47750/0 (socket says 172.21.15.28:47750) 2024-04-26T20:57:01.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.085+0000 7efe4f400640 1 -- 172.21.15.28:0/3079967302 learned_addr learned my addr 172.21.15.28:0/3079967302 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:01.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.086+0000 7efe57400640 1 -- 172.21.15.28:0/3079967302 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2466362060 0 0) 0x7efe580726d0 con 0x7efe58073770 2024-04-26T20:57:01.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.086+0000 7efe57400640 1 -- 172.21.15.28:0/3079967302 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe3c003520 con 0x7efe58073770 2024-04-26T20:57:01.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.086+0000 7efe5ca00640 1 -- 172.21.15.28:0/3079967302 >> v1:172.21.15.103:6789/0 conn(0x7efe58145800 legacy=0x7efe580ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:01.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.086+0000 7efe5ca00640 1 -- 172.21.15.28:0/3079967302 >> v1:172.21.15.28:6789/0 conn(0x7efe58073770 legacy=0x7efe580aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:01.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.086+0000 7efe5ca00640 1 -- 172.21.15.28:0/3079967302 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe58072c90 con 0x7efe58074050 2024-04-26T20:57:01.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.087+0000 7efe5ca00640 1 --2- 172.21.15.28:0/3079967302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe58074050 0x7efe5816a0a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efe58145d60 tx=0x7efe5002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b6ac854918463f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:01.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.088+0000 7efe57400640 1 -- 172.21.15.28:0/3079967302 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50007740 con 0x7efe58074050 2024-04-26T20:57:01.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.088+0000 7efe57400640 1 -- 172.21.15.28:0/3079967302 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe500078e0 con 0x7efe58074050 2024-04-26T20:57:01.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.088+0000 7efe57400640 1 -- 172.21.15.28:0/3079967302 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe50007c90 con 0x7efe58074050 2024-04-26T20:57:01.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.088+0000 7efe5ea00640 1 -- 172.21.15.28:0/3079967302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe58074050 msgr2=0x7efe5816a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:01.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.088+0000 7efe5ea00640 1 --2- 172.21.15.28:0/3079967302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe58074050 0x7efe5816a0a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efe58145d60 tx=0x7efe5002a6f0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.088+0000 7efe5ea00640 1 -- 172.21.15.28:0/3079967302 shutdown_connections 2024-04-26T20:57:01.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.088+0000 7efe5ea00640 1 --2- 172.21.15.28:0/3079967302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe58074050 0x7efe5816a0a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.088+0000 7efe5ea00640 1 -- 172.21.15.28:0/3079967302 >> 172.21.15.28:0/3079967302 conn(0x7efe5806e220 msgr2=0x7efe5806e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:01.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.088+0000 7efe5ea00640 1 -- 172.21.15.28:0/3079967302 shutdown_connections 2024-04-26T20:57:01.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.088+0000 7efe5ea00640 1 -- 172.21.15.28:0/3079967302 wait complete. 2024-04-26T20:57:01.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.090+0000 7efe5ea00640 1 Processor -- start 2024-04-26T20:57:01.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.090+0000 7efe5ea00640 1 -- start start 2024-04-26T20:57:01.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.091+0000 7efe5ea00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efe58073770 0x7efe5813efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:01.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.091+0000 7efe5ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe58074050 0x7efe5813d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:01.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.091+0000 7efe5ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe58145800 0x7efe5815ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:01.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.091+0000 7efe5ea00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efe580af6e0 con 0x7efe58074050 2024-04-26T20:57:01.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.091+0000 7efe5ea00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efe58073250 con 0x7efe58145800 2024-04-26T20:57:01.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.091+0000 7efe5ea00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7efe5816a5e0 con 0x7efe58073770 2024-04-26T20:57:01.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.091+0000 7efe4f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe58074050 0x7efe5813d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:01.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.091+0000 7efe4f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe58074050 0x7efe5813d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35128/0 (socket says 172.21.15.28:35128) 2024-04-26T20:57:01.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.092+0000 7efe4f400640 1 -- 172.21.15.28:0/4230729712 learned_addr learned my addr 172.21.15.28:0/4230729712 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:01.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.092+0000 7efe57e00640 1 --2- 172.21.15.28:0/4230729712 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efe58073770 0x7efe5813efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:01.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.092+0000 7efe5ca00640 1 --2- 172.21.15.28:0/4230729712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe58145800 0x7efe5815ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:01.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.092+0000 7efe4f400640 1 -- 172.21.15.28:0/4230729712 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efe58073770 msgr2=0x7efe5813efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:01.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.092+0000 7efe4f400640 1 --2- 172.21.15.28:0/4230729712 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efe58073770 0x7efe5813efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.092+0000 7efe4f400640 1 -- 172.21.15.28:0/4230729712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe58145800 msgr2=0x7efe5815ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:01.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.092+0000 7efe4f400640 1 --2- 172.21.15.28:0/4230729712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe58145800 0x7efe5815ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.092+0000 7efe4f400640 1 -- 172.21.15.28:0/4230729712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5815dee0 con 0x7efe58074050 2024-04-26T20:57:01.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.092+0000 7efe4f400640 1 --2- 172.21.15.28:0/4230729712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe58074050 0x7efe5813d7f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efe4000c4d0 tx=0x7efe4000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:01.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.093+0000 7efe54c00640 1 -- 172.21.15.28:0/4230729712 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe40017070 con 0x7efe58074050 2024-04-26T20:57:01.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.093+0000 7efe54c00640 1 -- 172.21.15.28:0/4230729712 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe40005ce0 con 0x7efe58074050 2024-04-26T20:57:01.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.093+0000 7efe54c00640 1 -- 172.21.15.28:0/4230729712 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe40007270 con 0x7efe58074050 2024-04-26T20:57:01.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.093+0000 7efe5ea00640 1 -- 172.21.15.28:0/4230729712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5815e1c0 con 0x7efe58074050 2024-04-26T20:57:01.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.093+0000 7efe5ea00640 1 -- 172.21.15.28:0/4230729712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe5813dd30 con 0x7efe58074050 2024-04-26T20:57:01.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.094+0000 7efe5ea00640 1 -- 172.21.15.28:0/4230729712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe58069bc0 con 0x7efe58074050 2024-04-26T20:57:01.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.095+0000 7efe54c00640 1 -- 172.21.15.28:0/4230729712 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe4001a070 con 0x7efe58074050 2024-04-26T20:57:01.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.096+0000 7efe54c00640 1 --2- 172.21.15.28:0/4230729712 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efe3406c8f0 0x7efe3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:01.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.096+0000 7efe54c00640 1 -- 172.21.15.28:0/4230729712 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efe4008aea0 con 0x7efe58074050 2024-04-26T20:57:01.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.096+0000 7efe57e00640 1 --2- 172.21.15.28:0/4230729712 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efe3406c8f0 0x7efe3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:01.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.097+0000 7efe57e00640 1 --2- 172.21.15.28:0/4230729712 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efe3406c8f0 0x7efe3406edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efe48007fd0 tx=0x7efe48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:01.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.100+0000 7efe54c00640 1 -- 172.21.15.28:0/4230729712 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efe58069bc0 con 0x7efe58074050 2024-04-26T20:57:01.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.305+0000 7efe5ea00640 1 -- 172.21.15.28:0/4230729712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe5806a400 con 0x7efe58074050 2024-04-26T20:57:01.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.307+0000 7efe54c00640 1 -- 172.21.15.28:0/4230729712 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7efe40058be0 con 0x7efe58074050 2024-04-26T20:57:01.308 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:01.308 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":21,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:56:52.699663+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:57:01.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.309+0000 7efe5ea00640 1 -- 172.21.15.28:0/4230729712 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efe3406c8f0 msgr2=0x7efe3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:01.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.309+0000 7efe5ea00640 1 --2- 172.21.15.28:0/4230729712 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efe3406c8f0 0x7efe3406edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efe48007fd0 tx=0x7efe48009040 comp rx=0 tx=0).stop 2024-04-26T20:57:01.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.309+0000 7efe5ea00640 1 -- 172.21.15.28:0/4230729712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe58074050 msgr2=0x7efe5813d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:01.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.309+0000 7efe5ea00640 1 --2- 172.21.15.28:0/4230729712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe58074050 0x7efe5813d7f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7efe4000c4d0 tx=0x7efe4000c990 comp rx=0 tx=0).stop 2024-04-26T20:57:01.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.310+0000 7efe5ea00640 1 -- 172.21.15.28:0/4230729712 shutdown_connections 2024-04-26T20:57:01.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.310+0000 7efe5ea00640 1 --2- 172.21.15.28:0/4230729712 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efe3406c8f0 0x7efe3406edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.310+0000 7efe5ea00640 1 --2- 172.21.15.28:0/4230729712 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe58145800 0x7efe5815ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.310+0000 7efe5ea00640 1 --2- 172.21.15.28:0/4230729712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe58074050 0x7efe5813d7f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.310+0000 7efe5ea00640 1 --2- 172.21.15.28:0/4230729712 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efe58073770 0x7efe5813efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.310+0000 7efe5ea00640 1 -- 172.21.15.28:0/4230729712 >> 172.21.15.28:0/4230729712 conn(0x7efe5806e220 msgr2=0x7efe580b2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:01.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.310+0000 7efe5ea00640 1 -- 172.21.15.28:0/4230729712 shutdown_connections 2024-04-26T20:57:01.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.310+0000 7efe5ea00640 1 -- 172.21.15.28:0/4230729712 wait complete. 2024-04-26T20:57:01.330 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:56:48.688498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:56:50.117541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:57:01.331 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:57:01.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.547+0000 7f39ada00640 1 Processor -- start 2024-04-26T20:57:01.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.547+0000 7f39ada00640 1 -- start start 2024-04-26T20:57:01.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.549+0000 7f39ada00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f39a815bb80 0x7f39a8073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:01.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.549+0000 7f39ada00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f39a8160650 0x7f39a80d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:01.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.549+0000 7f39ada00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f39a815bf60 0x7f39a80d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:01.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.549+0000 7f39ada00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f39a815f920 con 0x7f39a815bf60 2024-04-26T20:57:01.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.549+0000 7f39ada00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f39a80b3d90 con 0x7f39a815bb80 2024-04-26T20:57:01.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.549+0000 7f39ada00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f39a815fb70 con 0x7f39a8160650 2024-04-26T20:57:01.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.550+0000 7f39ad000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f39a815bf60 0x7f39a80d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:01.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.550+0000 7f39ad000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f39a815bf60 0x7f39a80d1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35140/0 (socket says 172.21.15.28:35140) 2024-04-26T20:57:01.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.550+0000 7f39ad000640 1 -- 172.21.15.28:0/1317127330 learned_addr learned my addr 172.21.15.28:0/1317127330 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:01.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.550+0000 7f39a7e00640 1 --2- 172.21.15.28:0/1317127330 >> v2:172.21.15.103:3300/0 conn(0x7f39a815bb80 0x7f39a8073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:01.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.550+0000 7f399fe00640 1 --2- 172.21.15.28:0/1317127330 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f39a8160650 0x7f39a80d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:01.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.550+0000 7f39ad000640 1 -- 172.21.15.28:0/1317127330 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f39a8160650 msgr2=0x7f39a80d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:01.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.550+0000 7f39ad000640 1 --2- 172.21.15.28:0/1317127330 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f39a8160650 0x7f39a80d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.550+0000 7f39ad000640 1 -- 172.21.15.28:0/1317127330 >> v2:172.21.15.103:3300/0 conn(0x7f39a815bb80 msgr2=0x7f39a8073bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:01.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.550+0000 7f39ad000640 1 --2- 172.21.15.28:0/1317127330 >> v2:172.21.15.103:3300/0 conn(0x7f39a815bb80 0x7f39a8073bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.550+0000 7f39ad000640 1 -- 172.21.15.28:0/1317127330 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a815fe50 con 0x7f39a815bf60 2024-04-26T20:57:01.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.550+0000 7f399fe00640 1 --2- 172.21.15.28:0/1317127330 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f39a8160650 0x7f39a80d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:01.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.550+0000 7f39a7e00640 1 --2- 172.21.15.28:0/1317127330 >> v2:172.21.15.103:3300/0 conn(0x7f39a815bb80 0x7f39a8073bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:01.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.551+0000 7f39ad000640 1 --2- 172.21.15.28:0/1317127330 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39a815bf60 0x7f39a80d1250 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f39a80b6110 tx=0x7f39a002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5394464d0908d57 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:01.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.552+0000 7f39a7400640 1 -- 172.21.15.28:0/1317127330 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0036070 con 0x7f39a815bf60 2024-04-26T20:57:01.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.552+0000 7f39a7400640 1 -- 172.21.15.28:0/1317127330 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a0031600 con 0x7f39a815bf60 2024-04-26T20:57:01.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.552+0000 7f39a7400640 1 -- 172.21.15.28:0/1317127330 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a00319d0 con 0x7f39a815bf60 2024-04-26T20:57:01.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.552+0000 7f39ada00640 1 -- 172.21.15.28:0/1317127330 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39a815bf60 msgr2=0x7f39a80d1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:01.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.552+0000 7f39ada00640 1 --2- 172.21.15.28:0/1317127330 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39a815bf60 0x7f39a80d1250 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f39a80b6110 tx=0x7f39a002a6a0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.553+0000 7f39ada00640 1 -- 172.21.15.28:0/1317127330 shutdown_connections 2024-04-26T20:57:01.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.553+0000 7f39ada00640 1 --2- 172.21.15.28:0/1317127330 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39a815bf60 0x7f39a80d1250 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.553+0000 7f39ada00640 1 --2- 172.21.15.28:0/1317127330 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f39a8160650 0x7f39a80d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.553+0000 7f39ada00640 1 --2- 172.21.15.28:0/1317127330 >> v2:172.21.15.103:3300/0 conn(0x7f39a815bb80 0x7f39a8073bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.553+0000 7f39ada00640 1 -- 172.21.15.28:0/1317127330 >> 172.21.15.28:0/1317127330 conn(0x7f39a806c530 msgr2=0x7f39a806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:01.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.553+0000 7f39ada00640 1 -- 172.21.15.28:0/1317127330 shutdown_connections 2024-04-26T20:57:01.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.553+0000 7f39ada00640 1 -- 172.21.15.28:0/1317127330 wait complete. 2024-04-26T20:57:01.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.554+0000 7f39ada00640 1 Processor -- start 2024-04-26T20:57:01.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.555+0000 7f39ada00640 1 -- start start 2024-04-26T20:57:01.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.555+0000 7f39ada00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39a815bb80 0x7f39a806e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:01.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.555+0000 7f39ada00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f39a815bf60 0x7f39a815cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:01.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.555+0000 7f39ada00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39a8160650 0x7f39a814c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:01.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.556+0000 7f39ada00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f39a80736d0 con 0x7f39a815bb80 2024-04-26T20:57:01.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.556+0000 7f39ada00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f39a80b3a20 con 0x7f39a8160650 2024-04-26T20:57:01.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.556+0000 7f39ada00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f39a8073850 con 0x7f39a815bf60 2024-04-26T20:57:01.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.556+0000 7f39a7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39a815bb80 0x7f39a806e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:01.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.556+0000 7f39a7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39a815bb80 0x7f39a806e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35148/0 (socket says 172.21.15.28:35148) 2024-04-26T20:57:01.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.556+0000 7f39a7e00640 1 -- 172.21.15.28:0/1671831579 learned_addr learned my addr 172.21.15.28:0/1671831579 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:01.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.556+0000 7f39ad000640 1 --2- 172.21.15.28:0/1671831579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39a8160650 0x7f39a814c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:01.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.556+0000 7f399fe00640 1 --2- 172.21.15.28:0/1671831579 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f39a815bf60 0x7f39a815cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:01.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.556+0000 7f39a7e00640 1 -- 172.21.15.28:0/1671831579 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f39a815bf60 msgr2=0x7f39a815cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:01.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.556+0000 7f39a7e00640 1 --2- 172.21.15.28:0/1671831579 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f39a815bf60 0x7f39a815cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.556+0000 7f39a7e00640 1 -- 172.21.15.28:0/1671831579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39a8160650 msgr2=0x7f39a814c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:01.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.556+0000 7f39a7e00640 1 --2- 172.21.15.28:0/1671831579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39a8160650 0x7f39a814c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.556+0000 7f39a7e00640 1 -- 172.21.15.28:0/1671831579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a815d730 con 0x7f39a815bb80 2024-04-26T20:57:01.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.556+0000 7f39ad000640 1 --2- 172.21.15.28:0/1671831579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39a8160650 0x7f39a814c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:01.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.557+0000 7f39a7e00640 1 --2- 172.21.15.28:0/1671831579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39a815bb80 0x7f39a806e190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f399800c7f0 tx=0x7f399800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:01.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.557+0000 7f39a4c00640 1 -- 172.21.15.28:0/1671831579 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998015070 con 0x7f39a815bb80 2024-04-26T20:57:01.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.557+0000 7f39a4c00640 1 -- 172.21.15.28:0/1671831579 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3998007820 con 0x7f39a815bb80 2024-04-26T20:57:01.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.557+0000 7f39ada00640 1 -- 172.21.15.28:0/1671831579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a815da10 con 0x7f39a815bb80 2024-04-26T20:57:01.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.557+0000 7f39a4c00640 1 -- 172.21.15.28:0/1671831579 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3998007c10 con 0x7f39a815bb80 2024-04-26T20:57:01.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.557+0000 7f39ada00640 1 -- 172.21.15.28:0/1671831579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a81f20f0 con 0x7f39a815bb80 2024-04-26T20:57:01.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.558+0000 7f39ada00640 1 -- 172.21.15.28:0/1671831579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a806a250 con 0x7f39a815bb80 2024-04-26T20:57:01.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.559+0000 7f39a4c00640 1 -- 172.21.15.28:0/1671831579 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3998018070 con 0x7f39a815bb80 2024-04-26T20:57:01.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.560+0000 7f39a4c00640 1 --2- 172.21.15.28:0/1671831579 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f398406c8f0 0x7f398406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:01.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.561+0000 7f39a4c00640 1 -- 172.21.15.28:0/1671831579 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f399808bbe0 con 0x7f39a815bb80 2024-04-26T20:57:01.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.561+0000 7f399fe00640 1 --2- 172.21.15.28:0/1671831579 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f398406c8f0 0x7f398406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:01.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.562+0000 7f399fe00640 1 --2- 172.21.15.28:0/1671831579 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f398406c8f0 0x7f398406edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f39a814de90 tx=0x7f3990008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:01.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.566+0000 7f39a4c00640 1 -- 172.21.15.28:0/1671831579 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f399805a020 con 0x7f39a815bb80 2024-04-26T20:57:01.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.766+0000 7f39ada00640 1 -- 172.21.15.28:0/1671831579 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f39a806f9a0 con 0x7f39a815bb80 2024-04-26T20:57:01.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.768+0000 7f39a4c00640 1 -- 172.21.15.28:0/1671831579 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f3998059920 con 0x7f39a815bb80 2024-04-26T20:57:01.769 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:57:01.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.770+0000 7f39ada00640 1 -- 172.21.15.28:0/1671831579 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f398406c8f0 msgr2=0x7f398406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:01.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.770+0000 7f39ada00640 1 --2- 172.21.15.28:0/1671831579 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f398406c8f0 0x7f398406edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f39a814de90 tx=0x7f3990008040 comp rx=0 tx=0).stop 2024-04-26T20:57:01.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.770+0000 7f39ada00640 1 -- 172.21.15.28:0/1671831579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39a815bb80 msgr2=0x7f39a806e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:01.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.770+0000 7f39ada00640 1 --2- 172.21.15.28:0/1671831579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39a815bb80 0x7f39a806e190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f399800c7f0 tx=0x7f399800ccb0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.770+0000 7f39ada00640 1 -- 172.21.15.28:0/1671831579 shutdown_connections 2024-04-26T20:57:01.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.770+0000 7f39ada00640 1 --2- 172.21.15.28:0/1671831579 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f398406c8f0 0x7f398406edb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.770+0000 7f39ada00640 1 --2- 172.21.15.28:0/1671831579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f39a8160650 0x7f39a814c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.770+0000 7f39ada00640 1 --2- 172.21.15.28:0/1671831579 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f39a815bf60 0x7f39a815cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.770+0000 7f39ada00640 1 --2- 172.21.15.28:0/1671831579 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f39a815bb80 0x7f39a806e190 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:01.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.770+0000 7f39ada00640 1 -- 172.21.15.28:0/1671831579 >> 172.21.15.28:0/1671831579 conn(0x7f39a806c530 msgr2=0x7f39a81587f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:01.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.770+0000 7f39ada00640 1 -- 172.21.15.28:0/1671831579 shutdown_connections 2024-04-26T20:57:01.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:01.770+0000 7f39ada00640 1 -- 172.21.15.28:0/1671831579 wait complete. 2024-04-26T20:57:01.792 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:57:02.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.010+0000 7f241a800640 1 Processor -- start 2024-04-26T20:57:02.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.011+0000 7f241a800640 1 -- start start 2024-04-26T20:57:02.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.012+0000 7f241a800640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f241415f360 0x7f24140b7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.012+0000 7f241a800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f24140bac00 0x7f24140b8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.013+0000 7f241a800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24140ba350 con 0x7f24140d1de0 2024-04-26T20:57:02.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.013+0000 7f241a800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f24140b9ab0 con 0x7f24140bac00 2024-04-26T20:57:02.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.013+0000 7f241a800640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f24140b9d90 con 0x7f241415f360 2024-04-26T20:57:02.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.013+0000 7f2419e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f24140bac00 0x7f24140b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.013+0000 7f2419400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f24140d1de0 0x7f24140d21c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47758/0 (socket says 172.21.15.28:47758) 2024-04-26T20:57:02.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.013+0000 7f2419400640 1 -- 172.21.15.28:0/1717129694 learned_addr learned my addr 172.21.15.28:0/1717129694 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:02.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.013+0000 7f2418a00640 1 --2- 172.21.15.28:0/1717129694 >> v2:172.21.15.103:3300/0 conn(0x7f241415f360 0x7f24140b7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.013+0000 7f2413e00640 1 -- 172.21.15.28:0/1717129694 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1772182892 0 0) 0x7f24140ba350 con 0x7f24140d1de0 2024-04-26T20:57:02.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.013+0000 7f2413e00640 1 -- 172.21.15.28:0/1717129694 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23fc003520 con 0x7f24140d1de0 2024-04-26T20:57:02.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.013+0000 7f2419e00640 1 -- 172.21.15.28:0/1717129694 >> v2:172.21.15.103:3300/0 conn(0x7f241415f360 msgr2=0x7f24140b7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:02.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.013+0000 7f2419e00640 1 --2- 172.21.15.28:0/1717129694 >> v2:172.21.15.103:3300/0 conn(0x7f241415f360 0x7f24140b7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.013+0000 7f2419e00640 1 -- 172.21.15.28:0/1717129694 >> v1:172.21.15.28:6789/0 conn(0x7f24140d1de0 legacy=0x7f24140d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:02.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.013+0000 7f2419e00640 1 -- 172.21.15.28:0/1717129694 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24140ba070 con 0x7f24140bac00 2024-04-26T20:57:02.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.015+0000 7f2419e00640 1 --2- 172.21.15.28:0/1717129694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24140bac00 0x7f24140b8500 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f24140d0c20 tx=0x7f240c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500f37b66de9d6f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:02.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.015+0000 7f2413e00640 1 -- 172.21.15.28:0/1717129694 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c007740 con 0x7f24140bac00 2024-04-26T20:57:02.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.015+0000 7f2413e00640 1 -- 172.21.15.28:0/1717129694 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f240c0078e0 con 0x7f24140bac00 2024-04-26T20:57:02.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.015+0000 7f2413e00640 1 -- 172.21.15.28:0/1717129694 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c007cb0 con 0x7f24140bac00 2024-04-26T20:57:02.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.016+0000 7f241a800640 1 -- 172.21.15.28:0/1717129694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24140bac00 msgr2=0x7f24140b8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:02.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.016+0000 7f241a800640 1 --2- 172.21.15.28:0/1717129694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24140bac00 0x7f24140b8500 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f24140d0c20 tx=0x7f240c02a680 comp rx=0 tx=0).stop 2024-04-26T20:57:02.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.016+0000 7f241a800640 1 -- 172.21.15.28:0/1717129694 shutdown_connections 2024-04-26T20:57:02.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.016+0000 7f241a800640 1 --2- 172.21.15.28:0/1717129694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24140bac00 0x7f24140b8500 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.016+0000 7f241a800640 1 --2- 172.21.15.28:0/1717129694 >> v2:172.21.15.103:3300/0 conn(0x7f241415f360 0x7f24140b7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.016+0000 7f241a800640 1 -- 172.21.15.28:0/1717129694 >> 172.21.15.28:0/1717129694 conn(0x7f241406c770 msgr2=0x7f241406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:02.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.016+0000 7f241a800640 1 -- 172.21.15.28:0/1717129694 shutdown_connections 2024-04-26T20:57:02.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.016+0000 7f241a800640 1 -- 172.21.15.28:0/1717129694 wait complete. 2024-04-26T20:57:02.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.018+0000 7f241a800640 1 Processor -- start 2024-04-26T20:57:02.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.018+0000 7f241a800640 1 -- start start 2024-04-26T20:57:02.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.019+0000 7f241a800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24140bac00 0x7f24140ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.019+0000 7f241a800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24140d1de0 0x7f24140cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.019+0000 7f241a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f241415f360 0x7f24140ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.019+0000 7f241a800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f24140d4620 con 0x7f241415f360 2024-04-26T20:57:02.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.019+0000 7f241a800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f24140d44a0 con 0x7f24140bac00 2024-04-26T20:57:02.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.019+0000 7f241a800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f24140d47a0 con 0x7f24140d1de0 2024-04-26T20:57:02.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.019+0000 7f2419e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f241415f360 0x7f24140ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.019+0000 7f2419e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f241415f360 0x7f24140ce4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35152/0 (socket says 172.21.15.28:35152) 2024-04-26T20:57:02.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.019+0000 7f2419e00640 1 -- 172.21.15.28:0/1996995505 learned_addr learned my addr 172.21.15.28:0/1996995505 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:02.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.019+0000 7f2419400640 1 --2- 172.21.15.28:0/1996995505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24140bac00 0x7f24140ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.019+0000 7f2418a00640 1 --2- 172.21.15.28:0/1996995505 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24140d1de0 0x7f24140cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.019+0000 7f2419e00640 1 -- 172.21.15.28:0/1996995505 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24140d1de0 msgr2=0x7f24140cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:02.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.019+0000 7f2419e00640 1 --2- 172.21.15.28:0/1996995505 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24140d1de0 0x7f24140cdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.019+0000 7f2419e00640 1 -- 172.21.15.28:0/1996995505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24140bac00 msgr2=0x7f24140ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:02.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.019+0000 7f2419e00640 1 --2- 172.21.15.28:0/1996995505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24140bac00 0x7f24140ca950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.019+0000 7f2419e00640 1 -- 172.21.15.28:0/1996995505 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2414160860 con 0x7f241415f360 2024-04-26T20:57:02.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.020+0000 7f2419e00640 1 --2- 172.21.15.28:0/1996995505 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f241415f360 0x7f24140ce4b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f240c02ac70 tx=0x7f240c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:02.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.020+0000 7f2411600640 1 -- 172.21.15.28:0/1996995505 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c040070 con 0x7f241415f360 2024-04-26T20:57:02.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.020+0000 7f2411600640 1 -- 172.21.15.28:0/1996995505 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f240c031bc0 con 0x7f241415f360 2024-04-26T20:57:02.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.020+0000 7f2411600640 1 -- 172.21.15.28:0/1996995505 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c009450 con 0x7f241415f360 2024-04-26T20:57:02.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.020+0000 7f241a800640 1 -- 172.21.15.28:0/1996995505 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2414160b40 con 0x7f241415f360 2024-04-26T20:57:02.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.020+0000 7f241a800640 1 -- 172.21.15.28:0/1996995505 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2414161100 con 0x7f241415f360 2024-04-26T20:57:02.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.021+0000 7f241a800640 1 -- 172.21.15.28:0/1996995505 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f241406a250 con 0x7f241415f360 2024-04-26T20:57:02.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.022+0000 7f2411600640 1 -- 172.21.15.28:0/1996995505 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f240c039040 con 0x7f241415f360 2024-04-26T20:57:02.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.023+0000 7f2411600640 1 --2- 172.21.15.28:0/1996995505 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f23f406c8f0 0x7f23f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.023+0000 7f2411600640 1 -- 172.21.15.28:0/1996995505 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f240c0ad440 con 0x7f241415f360 2024-04-26T20:57:02.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.024+0000 7f2419400640 1 --2- 172.21.15.28:0/1996995505 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f23f406c8f0 0x7f23f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.024+0000 7f2419400640 1 --2- 172.21.15.28:0/1996995505 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f23f406c8f0 0x7f23f406edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2408000f60 tx=0x7f2408009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:02.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.028+0000 7f2411600640 1 -- 172.21.15.28:0/1996995505 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f240c07b7d0 con 0x7f241415f360 2024-04-26T20:57:02.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.234+0000 7f241a800640 1 -- 172.21.15.28:0/1996995505 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f24140b6810 con 0x7f241415f360 2024-04-26T20:57:02.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.235+0000 7f2411600640 1 -- 172.21.15.28:0/1996995505 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f240c07b0d0 con 0x7f241415f360 2024-04-26T20:57:02.236 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-04-26T20:57:02.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.237+0000 7f241a800640 1 -- 172.21.15.28:0/1996995505 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f23f406c8f0 msgr2=0x7f23f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:02.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.237+0000 7f241a800640 1 --2- 172.21.15.28:0/1996995505 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f23f406c8f0 0x7f23f406edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2408000f60 tx=0x7f2408009040 comp rx=0 tx=0).stop 2024-04-26T20:57:02.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.237+0000 7f241a800640 1 -- 172.21.15.28:0/1996995505 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f241415f360 msgr2=0x7f24140ce4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:02.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.237+0000 7f241a800640 1 --2- 172.21.15.28:0/1996995505 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f241415f360 0x7f24140ce4b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f240c02ac70 tx=0x7f240c004a40 comp rx=0 tx=0).stop 2024-04-26T20:57:02.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.238+0000 7f241a800640 1 -- 172.21.15.28:0/1996995505 shutdown_connections 2024-04-26T20:57:02.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.238+0000 7f241a800640 1 --2- 172.21.15.28:0/1996995505 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f23f406c8f0 0x7f23f406edb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.238+0000 7f241a800640 1 --2- 172.21.15.28:0/1996995505 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f241415f360 0x7f24140ce4b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.238+0000 7f241a800640 1 --2- 172.21.15.28:0/1996995505 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f24140d1de0 0x7f24140cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.238+0000 7f241a800640 1 --2- 172.21.15.28:0/1996995505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f24140bac00 0x7f24140ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.238+0000 7f241a800640 1 -- 172.21.15.28:0/1996995505 >> 172.21.15.28:0/1996995505 conn(0x7f241406c770 msgr2=0x7f24140cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:02.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.238+0000 7f241a800640 1 -- 172.21.15.28:0/1996995505 shutdown_connections 2024-04-26T20:57:02.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.238+0000 7f241a800640 1 -- 172.21.15.28:0/1996995505 wait complete. 2024-04-26T20:57:02.258 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:57:02.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.462+0000 7ff4ade00640 1 Processor -- start 2024-04-26T20:57:02.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.462+0000 7ff4ade00640 1 -- start start 2024-04-26T20:57:02.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.464+0000 7ff4ade00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4a80cfb30 0x7ff4a80cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.464+0000 7ff4ade00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff4a806d3f0 0x7ff4a80b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.464+0000 7ff4ade00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff4a80b7f20 0x7ff4a80d3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.464+0000 7ff4ade00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff4a80d0450 con 0x7ff4a80b7f20 2024-04-26T20:57:02.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.464+0000 7ff4ade00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7ff4a80ba890 con 0x7ff4a806d3f0 2024-04-26T20:57:02.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.464+0000 7ff4ade00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff4a80bab70 con 0x7ff4a80cfb30 2024-04-26T20:57:02.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.464+0000 7ff4ad400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff4a80b7f20 0x7ff4a80d3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.464+0000 7ff4ad400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff4a80b7f20 0x7ff4a80d3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35164/0 (socket says 172.21.15.28:35164) 2024-04-26T20:57:02.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.464+0000 7ff4ad400640 1 -- 172.21.15.28:0/919728733 learned_addr learned my addr 172.21.15.28:0/919728733 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:02.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.465+0000 7ff4a7e00640 1 --2- 172.21.15.28:0/919728733 >> v2:172.21.15.103:3300/0 conn(0x7ff4a806d3f0 0x7ff4a80b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.465+0000 7ff4aca00640 1 --2- 172.21.15.28:0/919728733 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4a80cfb30 0x7ff4a80cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.465+0000 7ff4ad400640 1 -- 172.21.15.28:0/919728733 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4a80cfb30 msgr2=0x7ff4a80cff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:02.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.465+0000 7ff4ad400640 1 --2- 172.21.15.28:0/919728733 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4a80cfb30 0x7ff4a80cff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.465+0000 7ff4ad400640 1 -- 172.21.15.28:0/919728733 >> v2:172.21.15.103:3300/0 conn(0x7ff4a806d3f0 msgr2=0x7ff4a80b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:02.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.465+0000 7ff4ad400640 1 --2- 172.21.15.28:0/919728733 >> v2:172.21.15.103:3300/0 conn(0x7ff4a806d3f0 0x7ff4a80b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.465+0000 7ff4ad400640 1 -- 172.21.15.28:0/919728733 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a80d3570 con 0x7ff4a80b7f20 2024-04-26T20:57:02.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.465+0000 7ff4a7e00640 1 --2- 172.21.15.28:0/919728733 >> v2:172.21.15.103:3300/0 conn(0x7ff4a806d3f0 0x7ff4a80b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:02.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.466+0000 7ff4ad400640 1 --2- 172.21.15.28:0/919728733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4a80b7f20 0x7ff4a80d3d70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff4a808d3d0 tx=0x7ff4a002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc9d19e0e9d3f7c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:02.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.466+0000 7ff4a7400640 1 -- 172.21.15.28:0/919728733 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0036070 con 0x7ff4a80b7f20 2024-04-26T20:57:02.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.466+0000 7ff4a7400640 1 -- 172.21.15.28:0/919728733 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a0031600 con 0x7ff4a80b7f20 2024-04-26T20:57:02.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.466+0000 7ff4a7400640 1 -- 172.21.15.28:0/919728733 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a00319d0 con 0x7ff4a80b7f20 2024-04-26T20:57:02.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.467+0000 7ff4ade00640 1 -- 172.21.15.28:0/919728733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4a80b7f20 msgr2=0x7ff4a80d3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:02.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.467+0000 7ff4ade00640 1 --2- 172.21.15.28:0/919728733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4a80b7f20 0x7ff4a80d3d70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff4a808d3d0 tx=0x7ff4a002a610 comp rx=0 tx=0).stop 2024-04-26T20:57:02.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.467+0000 7ff4ade00640 1 -- 172.21.15.28:0/919728733 shutdown_connections 2024-04-26T20:57:02.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.467+0000 7ff4ade00640 1 --2- 172.21.15.28:0/919728733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4a80b7f20 0x7ff4a80d3d70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.467+0000 7ff4ade00640 1 --2- 172.21.15.28:0/919728733 >> v2:172.21.15.103:3300/0 conn(0x7ff4a806d3f0 0x7ff4a80b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.467+0000 7ff4ade00640 1 --2- 172.21.15.28:0/919728733 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4a80cfb30 0x7ff4a80cff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.467+0000 7ff4ade00640 1 -- 172.21.15.28:0/919728733 >> 172.21.15.28:0/919728733 conn(0x7ff4a80d4df0 msgr2=0x7ff4a80d5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:02.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.467+0000 7ff4ade00640 1 -- 172.21.15.28:0/919728733 shutdown_connections 2024-04-26T20:57:02.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.467+0000 7ff4ade00640 1 -- 172.21.15.28:0/919728733 wait complete. 2024-04-26T20:57:02.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.469+0000 7ff4ade00640 1 Processor -- start 2024-04-26T20:57:02.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.469+0000 7ff4ade00640 1 -- start start 2024-04-26T20:57:02.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.470+0000 7ff4ade00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4a806d3f0 0x7ff4a81f6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.470+0000 7ff4ade00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4a80b7f20 0x7ff4a80c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.470+0000 7ff4ade00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4a80cfb30 0x7ff4a8160d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.470+0000 7ff4ade00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a815f2c0 con 0x7ff4a80b7f20 2024-04-26T20:57:02.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.470+0000 7ff4ade00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a80b45a0 con 0x7ff4a806d3f0 2024-04-26T20:57:02.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.470+0000 7ff4ade00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff4a815f440 con 0x7ff4a80cfb30 2024-04-26T20:57:02.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.470+0000 7ff4a7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4a80b7f20 0x7ff4a80c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.470+0000 7ff4a7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4a80b7f20 0x7ff4a80c9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35170/0 (socket says 172.21.15.28:35170) 2024-04-26T20:57:02.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.470+0000 7ff4a7e00640 1 -- 172.21.15.28:0/2355494834 learned_addr learned my addr 172.21.15.28:0/2355494834 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:02.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.470+0000 7ff4ad400640 1 --2- 172.21.15.28:0/2355494834 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4a80cfb30 0x7ff4a8160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.470+0000 7ff4aca00640 1 --2- 172.21.15.28:0/2355494834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4a806d3f0 0x7ff4a81f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.471+0000 7ff4a7e00640 1 -- 172.21.15.28:0/2355494834 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4a80cfb30 msgr2=0x7ff4a8160d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:02.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.471+0000 7ff4a7e00640 1 --2- 172.21.15.28:0/2355494834 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4a80cfb30 0x7ff4a8160d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.471+0000 7ff4a7e00640 1 -- 172.21.15.28:0/2355494834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4a806d3f0 msgr2=0x7ff4a81f6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:02.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.471+0000 7ff4a7e00640 1 --2- 172.21.15.28:0/2355494834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4a806d3f0 0x7ff4a81f6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.471+0000 7ff4a7e00640 1 -- 172.21.15.28:0/2355494834 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a81f68a0 con 0x7ff4a80b7f20 2024-04-26T20:57:02.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.471+0000 7ff4a7e00640 1 --2- 172.21.15.28:0/2355494834 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4a80b7f20 0x7ff4a80c9360 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff49400c7f0 tx=0x7ff49400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:02.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.471+0000 7ff4a4c00640 1 -- 172.21.15.28:0/2355494834 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494015070 con 0x7ff4a80b7f20 2024-04-26T20:57:02.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.471+0000 7ff4a4c00640 1 -- 172.21.15.28:0/2355494834 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff494004220 con 0x7ff4a80b7f20 2024-04-26T20:57:02.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.471+0000 7ff4a4c00640 1 -- 172.21.15.28:0/2355494834 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff494007620 con 0x7ff4a80b7f20 2024-04-26T20:57:02.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.472+0000 7ff4ade00640 1 -- 172.21.15.28:0/2355494834 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a80ca820 con 0x7ff4a80b7f20 2024-04-26T20:57:02.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.472+0000 7ff4ade00640 1 -- 172.21.15.28:0/2355494834 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a80c8880 con 0x7ff4a80b7f20 2024-04-26T20:57:02.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.473+0000 7ff4ade00640 1 -- 172.21.15.28:0/2355494834 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a8076710 con 0x7ff4a80b7f20 2024-04-26T20:57:02.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.474+0000 7ff4a4c00640 1 -- 172.21.15.28:0/2355494834 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4940079d0 con 0x7ff4a80b7f20 2024-04-26T20:57:02.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.475+0000 7ff4a4c00640 1 --2- 172.21.15.28:0/2355494834 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff48806c8f0 0x7ff48806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.475+0000 7ff4a4c00640 1 -- 172.21.15.28:0/2355494834 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff49408bf00 con 0x7ff4a80b7f20 2024-04-26T20:57:02.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.475+0000 7ff4aca00640 1 --2- 172.21.15.28:0/2355494834 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff48806c8f0 0x7ff48806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.476+0000 7ff4aca00640 1 --2- 172.21.15.28:0/2355494834 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff48806c8f0 0x7ff48806edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff49c006fd0 tx=0x7ff49c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:02.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.480+0000 7ff4a4c00640 1 -- 172.21.15.28:0/2355494834 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff4a8076710 con 0x7ff4a80b7f20 2024-04-26T20:57:02.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.683+0000 7ff4ade00640 1 -- 172.21.15.28:0/2355494834 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff4a80d2c50 con 0x7ff4a80b7f20 2024-04-26T20:57:02.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.684+0000 7ff4a4c00640 1 -- 172.21.15.28:0/2355494834 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7ff4a80d2c50 con 0x7ff4a80b7f20 2024-04-26T20:57:02.686 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-04-26T20:57:02.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.687+0000 7ff4ade00640 1 -- 172.21.15.28:0/2355494834 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff48806c8f0 msgr2=0x7ff48806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:02.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.687+0000 7ff4ade00640 1 --2- 172.21.15.28:0/2355494834 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff48806c8f0 0x7ff48806edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff49c006fd0 tx=0x7ff49c008040 comp rx=0 tx=0).stop 2024-04-26T20:57:02.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.687+0000 7ff4ade00640 1 -- 172.21.15.28:0/2355494834 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4a80b7f20 msgr2=0x7ff4a80c9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:02.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.687+0000 7ff4ade00640 1 --2- 172.21.15.28:0/2355494834 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4a80b7f20 0x7ff4a80c9360 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff49400c7f0 tx=0x7ff49400ccb0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.688+0000 7ff4ade00640 1 -- 172.21.15.28:0/2355494834 shutdown_connections 2024-04-26T20:57:02.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.688+0000 7ff4ade00640 1 --2- 172.21.15.28:0/2355494834 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff48806c8f0 0x7ff48806edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.688+0000 7ff4ade00640 1 --2- 172.21.15.28:0/2355494834 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff4a80cfb30 0x7ff4a8160d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.688+0000 7ff4ade00640 1 --2- 172.21.15.28:0/2355494834 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff4a80b7f20 0x7ff4a80c9360 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.688+0000 7ff4ade00640 1 --2- 172.21.15.28:0/2355494834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4a806d3f0 0x7ff4a81f6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.688+0000 7ff4ade00640 1 -- 172.21.15.28:0/2355494834 >> 172.21.15.28:0/2355494834 conn(0x7ff4a80d4df0 msgr2=0x7ff4a806cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:02.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.688+0000 7ff4ade00640 1 -- 172.21.15.28:0/2355494834 shutdown_connections 2024-04-26T20:57:02.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.688+0000 7ff4ade00640 1 -- 172.21.15.28:0/2355494834 wait complete. 2024-04-26T20:57:02.709 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:57:02.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.917+0000 7ff59e600640 1 Processor -- start 2024-04-26T20:57:02.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.917+0000 7ff59e600640 1 -- start start 2024-04-26T20:57:02.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.919+0000 7ff59e600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff598144b90 0x7ff5980cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.919+0000 7ff59e600640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff598073800 0x7ff598073c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.919+0000 7ff59e600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff598072760 con 0x7ff598145080 2024-04-26T20:57:02.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.919+0000 7ff59e600640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7ff598072a40 con 0x7ff598073800 2024-04-26T20:57:02.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.919+0000 7ff59e600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff598072d20 con 0x7ff598144b90 2024-04-26T20:57:02.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.919+0000 7ff58fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff598145080 0x7ff5980ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47766/0 (socket says 172.21.15.28:47766) 2024-04-26T20:57:02.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.919+0000 7ff58fe00640 1 -- 172.21.15.28:0/2037612225 learned_addr learned my addr 172.21.15.28:0/2037612225 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:02.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.920+0000 7ff59dc00640 1 --2- 172.21.15.28:0/2037612225 >> v2:172.21.15.103:3300/0 conn(0x7ff598073800 0x7ff598073c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.920+0000 7ff59d200640 1 --2- 172.21.15.28:0/2037612225 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff598144b90 0x7ff5980cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.920+0000 7ff597e00640 1 -- 172.21.15.28:0/2037612225 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3276250974 0 0) 0x7ff598072760 con 0x7ff598145080 2024-04-26T20:57:02.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.920+0000 7ff597e00640 1 -- 172.21.15.28:0/2037612225 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff57c005730 con 0x7ff598145080 2024-04-26T20:57:02.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.920+0000 7ff597e00640 1 -- 172.21.15.28:0/2037612225 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3906949100 0 0) 0x7ff57c005730 con 0x7ff598145080 2024-04-26T20:57:02.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.920+0000 7ff597e00640 1 -- 172.21.15.28:0/2037612225 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff598144b90 msgr2=0x7ff5980cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:02.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.920+0000 7ff597e00640 1 --2- 172.21.15.28:0/2037612225 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff598144b90 0x7ff5980cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.920+0000 7ff597e00640 1 -- 172.21.15.28:0/2037612225 >> v2:172.21.15.103:3300/0 conn(0x7ff598073800 msgr2=0x7ff598073c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:02.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.920+0000 7ff597e00640 1 --2- 172.21.15.28:0/2037612225 >> v2:172.21.15.103:3300/0 conn(0x7ff598073800 0x7ff598073c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.920+0000 7ff597e00640 1 -- 172.21.15.28:0/2037612225 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff598073000 con 0x7ff598145080 2024-04-26T20:57:02.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.920+0000 7ff597e00640 1 -- 172.21.15.28:0/2037612225 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7ff580001670 con 0x7ff598145080 2024-04-26T20:57:02.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.920+0000 7ff597e00640 1 -- 172.21.15.28:0/2037612225 >> v1:172.21.15.28:6789/0 conn(0x7ff598145080 legacy=0x7ff5980ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:02.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.920+0000 7ff597e00640 1 --2- 172.21.15.28:0/2037612225 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff598144b90 0x7ff57c005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.920+0000 7ff597e00640 1 --2- 172.21.15.28:0/2037612225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff57c006140 0x7ff57c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.920+0000 7ff597e00640 1 --2- 172.21.15.28:0/2037612225 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff57c006f50 0x7ff57c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.920+0000 7ff597e00640 1 -- 172.21.15.28:0/2037612225 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff598072a40 con 0x7ff57c006f50 2024-04-26T20:57:02.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.920+0000 7ff597e00640 1 -- 172.21.15.28:0/2037612225 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff598072d20 con 0x7ff57c006140 2024-04-26T20:57:02.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.920+0000 7ff597e00640 1 -- 172.21.15.28:0/2037612225 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff57c00dc20 con 0x7ff598144b90 2024-04-26T20:57:02.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.921+0000 7ff59dc00640 1 --2- 172.21.15.28:0/2037612225 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff57c006f50 0x7ff57c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.921+0000 7ff58fe00640 1 --2- 172.21.15.28:0/2037612225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff57c006140 0x7ff57c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.921+0000 7ff59dc00640 1 -- 172.21.15.28:0/2037612225 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff598144b90 msgr2=0x7ff57c005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:02.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.921+0000 7ff59dc00640 1 --2- 172.21.15.28:0/2037612225 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff598144b90 0x7ff57c005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.921+0000 7ff59dc00640 1 -- 172.21.15.28:0/2037612225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff57c006140 msgr2=0x7ff57c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:02.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.921+0000 7ff59dc00640 1 --2- 172.21.15.28:0/2037612225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff57c006140 0x7ff57c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.921+0000 7ff59dc00640 1 -- 172.21.15.28:0/2037612225 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57c005730 con 0x7ff57c006f50 2024-04-26T20:57:02.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.922+0000 7ff59dc00640 1 --2- 172.21.15.28:0/2037612225 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff57c006f50 0x7ff57c0073d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff590009af0 tx=0x7ff57c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e61877b10a3ef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:02.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.922+0000 7ff597e00640 1 -- 172.21.15.28:0/2037612225 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff590036070 con 0x7ff57c006f50 2024-04-26T20:57:02.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.922+0000 7ff597e00640 1 -- 172.21.15.28:0/2037612225 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff590031400 con 0x7ff57c006f50 2024-04-26T20:57:02.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.922+0000 7ff597e00640 1 -- 172.21.15.28:0/2037612225 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff590009690 con 0x7ff57c006f50 2024-04-26T20:57:02.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.923+0000 7ff59e600640 1 -- 172.21.15.28:0/2037612225 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff57c006f50 msgr2=0x7ff57c0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:02.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.923+0000 7ff59e600640 1 --2- 172.21.15.28:0/2037612225 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff57c006f50 0x7ff57c0073d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff590009af0 tx=0x7ff57c007a40 comp rx=0 tx=0).stop 2024-04-26T20:57:02.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.923+0000 7ff59dc00640 1 -- 172.21.15.28:0/2037612225 reap_dead start 2024-04-26T20:57:02.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.923+0000 7ff59e600640 1 -- 172.21.15.28:0/2037612225 shutdown_connections 2024-04-26T20:57:02.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.923+0000 7ff59e600640 1 -- 172.21.15.28:0/2037612225 >> 172.21.15.28:0/2037612225 conn(0x7ff5980d0a50 msgr2=0x7ff5980d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:02.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.923+0000 7ff59e600640 1 -- 172.21.15.28:0/2037612225 shutdown_connections 2024-04-26T20:57:02.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.923+0000 7ff59e600640 1 -- 172.21.15.28:0/2037612225 wait complete. 2024-04-26T20:57:02.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.925+0000 7ff59e600640 1 Processor -- start 2024-04-26T20:57:02.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.925+0000 7ff59e600640 1 -- start start 2024-04-26T20:57:02.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.926+0000 7ff59e600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff57c006f50 0x7ff59813e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.926+0000 7ff59e600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5980b6310 0x7ff5980b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.926+0000 7ff59e600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5980b4d30 0x7ff5980b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.926+0000 7ff59e600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff5980ced30 con 0x7ff57c006f50 2024-04-26T20:57:02.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.926+0000 7ff59e600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff5980cb0a0 con 0x7ff5980b4d30 2024-04-26T20:57:02.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.926+0000 7ff59e600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff5980b5270 con 0x7ff5980b6310 2024-04-26T20:57:02.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.926+0000 7ff59d200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff57c006f50 0x7ff59813e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.926+0000 7ff59d200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff57c006f50 0x7ff59813e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35178/0 (socket says 172.21.15.28:35178) 2024-04-26T20:57:02.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.926+0000 7ff59d200640 1 -- 172.21.15.28:0/1005033077 learned_addr learned my addr 172.21.15.28:0/1005033077 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:02.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.927+0000 7ff58fe00640 1 --2- 172.21.15.28:0/1005033077 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5980b6310 0x7ff5980b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.927+0000 7ff59dc00640 1 --2- 172.21.15.28:0/1005033077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5980b4d30 0x7ff5980b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.927+0000 7ff59d200640 1 -- 172.21.15.28:0/1005033077 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5980b6310 msgr2=0x7ff5980b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:02.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.927+0000 7ff59d200640 1 --2- 172.21.15.28:0/1005033077 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5980b6310 0x7ff5980b47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.927+0000 7ff59d200640 1 -- 172.21.15.28:0/1005033077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5980b4d30 msgr2=0x7ff5980b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:02.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.927+0000 7ff59d200640 1 --2- 172.21.15.28:0/1005033077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5980b4d30 0x7ff5980b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:02.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.927+0000 7ff59d200640 1 -- 172.21.15.28:0/1005033077 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5980b5550 con 0x7ff57c006f50 2024-04-26T20:57:02.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.927+0000 7ff59dc00640 1 --2- 172.21.15.28:0/1005033077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5980b4d30 0x7ff5980b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:02.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.927+0000 7ff59d200640 1 --2- 172.21.15.28:0/1005033077 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff57c006f50 0x7ff59813e4e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff58800ef40 tx=0x7ff58800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:02.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.927+0000 7ff595600640 1 -- 172.21.15.28:0/1005033077 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588009e60 con 0x7ff57c006f50 2024-04-26T20:57:02.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.927+0000 7ff595600640 1 -- 172.21.15.28:0/1005033077 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff588019070 con 0x7ff57c006f50 2024-04-26T20:57:02.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.927+0000 7ff59e600640 1 -- 172.21.15.28:0/1005033077 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5980b5830 con 0x7ff57c006f50 2024-04-26T20:57:02.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.927+0000 7ff595600640 1 -- 172.21.15.28:0/1005033077 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff588003bb0 con 0x7ff57c006f50 2024-04-26T20:57:02.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.927+0000 7ff59e600640 1 -- 172.21.15.28:0/1005033077 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5980b5df0 con 0x7ff57c006f50 2024-04-26T20:57:02.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.929+0000 7ff59e600640 1 -- 172.21.15.28:0/1005033077 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff59806a360 con 0x7ff57c006f50 2024-04-26T20:57:02.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.930+0000 7ff595600640 1 -- 172.21.15.28:0/1005033077 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff588005d10 con 0x7ff57c006f50 2024-04-26T20:57:02.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.931+0000 7ff595600640 1 --2- 172.21.15.28:0/1005033077 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff57406c8f0 0x7ff57406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:02.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.931+0000 7ff595600640 1 -- 172.21.15.28:0/1005033077 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff58808c030 con 0x7ff57c006f50 2024-04-26T20:57:02.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.931+0000 7ff58fe00640 1 --2- 172.21.15.28:0/1005033077 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff57406c8f0 0x7ff57406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:02.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.932+0000 7ff58fe00640 1 --2- 172.21.15.28:0/1005033077 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff57406c8f0 0x7ff57406edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff5980d0700 tx=0x7ff580009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:02.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:02.935+0000 7ff595600640 1 -- 172.21.15.28:0/1005033077 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff588023020 con 0x7ff57c006f50 2024-04-26T20:57:03.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.181+0000 7ff59e600640 1 -- 172.21.15.28:0/1005033077 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff598143480 con 0x7ff57c006f50 2024-04-26T20:57:03.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.182+0000 7ff595600640 1 -- 172.21.15.28:0/1005033077 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7ff58805a460 con 0x7ff57c006f50 2024-04-26T20:57:03.184 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:03.184 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/2818824721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2818824721},{"type":"v1","addr":"172.21.15.103:6834","nonce":2818824721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/259712913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":259712913},{"type":"v1","addr":"172.21.15.28:6836","nonce":259712913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/989575415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":989575415},{"type":"v1","addr":"172.21.15.28:6837","nonce":989575415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:56:51.143688+0000","modified":"2024-04-26T20:56:54.228571+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6835/2361950169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":2361950169},{"type":"v1","addr":"172.21.15.103:6835","nonce":2361950169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-26T20:57:03.184 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 8 2024-04-26T20:57:03.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.185+0000 7ff59e600640 1 -- 172.21.15.28:0/1005033077 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff57406c8f0 msgr2=0x7ff57406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:03.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.185+0000 7ff59e600640 1 --2- 172.21.15.28:0/1005033077 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff57406c8f0 0x7ff57406edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff5980d0700 tx=0x7ff580009040 comp rx=0 tx=0).stop 2024-04-26T20:57:03.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.186+0000 7ff59e600640 1 -- 172.21.15.28:0/1005033077 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff57c006f50 msgr2=0x7ff59813e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:03.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.186+0000 7ff59e600640 1 --2- 172.21.15.28:0/1005033077 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff57c006f50 0x7ff59813e4e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff58800ef40 tx=0x7ff58800b480 comp rx=0 tx=0).stop 2024-04-26T20:57:03.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.186+0000 7ff59e600640 1 -- 172.21.15.28:0/1005033077 shutdown_connections 2024-04-26T20:57:03.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.186+0000 7ff59e600640 1 --2- 172.21.15.28:0/1005033077 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff57406c8f0 0x7ff57406edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:03.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.186+0000 7ff59e600640 1 --2- 172.21.15.28:0/1005033077 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff5980b4d30 0x7ff5980b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:03.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.186+0000 7ff59e600640 1 --2- 172.21.15.28:0/1005033077 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff5980b6310 0x7ff5980b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:03.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.186+0000 7ff59e600640 1 --2- 172.21.15.28:0/1005033077 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff57c006f50 0x7ff59813e4e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:03.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.186+0000 7ff59e600640 1 -- 172.21.15.28:0/1005033077 >> 172.21.15.28:0/1005033077 conn(0x7ff5980d0a50 msgr2=0x7ff5980d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:03.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.186+0000 7ff59e600640 1 -- 172.21.15.28:0/1005033077 shutdown_connections 2024-04-26T20:57:03.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.186+0000 7ff59e600640 1 -- 172.21.15.28:0/1005033077 wait complete. 2024-04-26T20:57:03.204 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:57:03.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.412+0000 7f32ade00640 1 Processor -- start 2024-04-26T20:57:03.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.413+0000 7f32ade00640 1 -- start start 2024-04-26T20:57:03.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.415+0000 7f32ade00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f32a80d4700 0x7f32a80d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:03.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.415+0000 7f32ade00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f32a80d5370 0x7f32a80b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:03.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.415+0000 7f32ade00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32a80d0710 con 0x7f32a815f1b0 2024-04-26T20:57:03.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.415+0000 7f32ade00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f32a80d57e0 con 0x7f32a80d4700 2024-04-26T20:57:03.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.415+0000 7f32ade00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f32a80cff00 con 0x7f32a80d5370 2024-04-26T20:57:03.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.415+0000 7f32a7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f32a80d4700 0x7f32a80d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:03.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.415+0000 7f32a7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f32a80d4700 0x7f32a80d4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35194/0 (socket says 172.21.15.28:35194) 2024-04-26T20:57:03.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.415+0000 7f32a7e00640 1 -- 172.21.15.28:0/3888926530 learned_addr learned my addr 172.21.15.28:0/3888926530 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:03.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.415+0000 7f32ad400640 1 --2- 172.21.15.28:0/3888926530 >> v2:172.21.15.103:3300/0 conn(0x7f32a80d5370 0x7f32a80b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:03.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.415+0000 7f32a7e00640 1 -- 172.21.15.28:0/3888926530 >> v2:172.21.15.103:3300/0 conn(0x7f32a80d5370 msgr2=0x7f32a80b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:03.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.415+0000 7f32a7e00640 1 --2- 172.21.15.28:0/3888926530 >> v2:172.21.15.103:3300/0 conn(0x7f32a80d5370 0x7f32a80b80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:03.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.415+0000 7f32a7e00640 1 -- 172.21.15.28:0/3888926530 >> v1:172.21.15.103:6789/0 conn(0x7f32a815f1b0 legacy=0x7f32a80cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:03.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.415+0000 7f32a7e00640 1 -- 172.21.15.28:0/3888926530 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a80d4b40 con 0x7f32a80d4700 2024-04-26T20:57:03.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.416+0000 7f32a7e00640 1 --2- 172.21.15.28:0/3888926530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32a80d4700 0x7f32a80d4e30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f32a80b7a90 tx=0x7f329002a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa14981f53cfea43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:03.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.417+0000 7f32a7400640 1 -- 172.21.15.28:0/3888926530 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290036070 con 0x7f32a80d4700 2024-04-26T20:57:03.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.417+0000 7f32a7400640 1 -- 172.21.15.28:0/3888926530 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3290031600 con 0x7f32a80d4700 2024-04-26T20:57:03.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.417+0000 7f32a7400640 1 -- 172.21.15.28:0/3888926530 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32900319d0 con 0x7f32a80d4700 2024-04-26T20:57:03.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.417+0000 7f32ade00640 1 -- 172.21.15.28:0/3888926530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32a80d4700 msgr2=0x7f32a80d4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:03.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.417+0000 7f32ade00640 1 --2- 172.21.15.28:0/3888926530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32a80d4700 0x7f32a80d4e30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f32a80b7a90 tx=0x7f329002a760 comp rx=0 tx=0).stop 2024-04-26T20:57:03.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.417+0000 7f32ade00640 1 -- 172.21.15.28:0/3888926530 shutdown_connections 2024-04-26T20:57:03.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.417+0000 7f32ade00640 1 --2- 172.21.15.28:0/3888926530 >> v2:172.21.15.103:3300/0 conn(0x7f32a80d5370 0x7f32a80b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:03.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.417+0000 7f32ade00640 1 --2- 172.21.15.28:0/3888926530 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32a80d4700 0x7f32a80d4e30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:03.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.417+0000 7f32ade00640 1 -- 172.21.15.28:0/3888926530 >> 172.21.15.28:0/3888926530 conn(0x7f32a806c770 msgr2=0x7f32a806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:03.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.418+0000 7f32ade00640 1 -- 172.21.15.28:0/3888926530 shutdown_connections 2024-04-26T20:57:03.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.418+0000 7f32ade00640 1 -- 172.21.15.28:0/3888926530 wait complete. 2024-04-26T20:57:03.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.419+0000 7f32ade00640 1 Processor -- start 2024-04-26T20:57:03.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.419+0000 7f32ade00640 1 -- start start 2024-04-26T20:57:03.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.420+0000 7f32ade00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32a80d4700 0x7f32a8152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:03.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.420+0000 7f32ade00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32a80d5370 0x7f32a80ce480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:03.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.420+0000 7f32ade00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f32a815f1b0 0x7f32a8161dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:03.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.420+0000 7f32ade00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f32a80d0430 con 0x7f32a80d5370 2024-04-26T20:57:03.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.420+0000 7f32ade00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f32a80d4270 con 0x7f32a80d4700 2024-04-26T20:57:03.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.420+0000 7f32ade00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f32a80d3f00 con 0x7f32a815f1b0 2024-04-26T20:57:03.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.420+0000 7f32a7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32a80d5370 0x7f32a80ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:03.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.421+0000 7f32a7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32a80d5370 0x7f32a80ce480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35210/0 (socket says 172.21.15.28:35210) 2024-04-26T20:57:03.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.421+0000 7f32a7e00640 1 -- 172.21.15.28:0/819917872 learned_addr learned my addr 172.21.15.28:0/819917872 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:03.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.421+0000 7f32aca00640 1 --2- 172.21.15.28:0/819917872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32a80d4700 0x7f32a8152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:03.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.421+0000 7f32ad400640 1 --2- 172.21.15.28:0/819917872 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f32a815f1b0 0x7f32a8161dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:03.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.421+0000 7f32a7e00640 1 -- 172.21.15.28:0/819917872 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f32a815f1b0 msgr2=0x7f32a8161dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:03.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.421+0000 7f32a7e00640 1 --2- 172.21.15.28:0/819917872 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f32a815f1b0 0x7f32a8161dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:03.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.421+0000 7f32a7e00640 1 -- 172.21.15.28:0/819917872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32a80d4700 msgr2=0x7f32a8152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:03.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.421+0000 7f32a7e00640 1 --2- 172.21.15.28:0/819917872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32a80d4700 0x7f32a8152f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:03.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.421+0000 7f32a7e00640 1 -- 172.21.15.28:0/819917872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a80ced80 con 0x7f32a80d5370 2024-04-26T20:57:03.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.421+0000 7f32aca00640 1 --2- 172.21.15.28:0/819917872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32a80d4700 0x7f32a8152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:03.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.421+0000 7f32ad400640 1 --2- 172.21.15.28:0/819917872 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f32a815f1b0 0x7f32a8161dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:03.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.421+0000 7f32a7e00640 1 --2- 172.21.15.28:0/819917872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32a80d5370 0x7f32a80ce480 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3290009f10 tx=0x7f3290004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:03.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.422+0000 7f32a4c00640 1 -- 172.21.15.28:0/819917872 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290036050 con 0x7f32a80d5370 2024-04-26T20:57:03.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.422+0000 7f32a4c00640 1 -- 172.21.15.28:0/819917872 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3290042070 con 0x7f32a80d5370 2024-04-26T20:57:03.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.422+0000 7f32a4c00640 1 -- 172.21.15.28:0/819917872 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3290007c30 con 0x7f32a80d5370 2024-04-26T20:57:03.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.422+0000 7f32ade00640 1 -- 172.21.15.28:0/819917872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a8161010 con 0x7f32a80d5370 2024-04-26T20:57:03.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.422+0000 7f32ade00640 1 -- 172.21.15.28:0/819917872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a81615d0 con 0x7f32a80d5370 2024-04-26T20:57:03.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.423+0000 7f32ade00640 1 -- 172.21.15.28:0/819917872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32a806a360 con 0x7f32a80d5370 2024-04-26T20:57:03.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.424+0000 7f32a4c00640 1 -- 172.21.15.28:0/819917872 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3290047460 con 0x7f32a80d5370 2024-04-26T20:57:03.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.425+0000 7f32a4c00640 1 --2- 172.21.15.28:0/819917872 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f328406c8f0 0x7f328406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:03.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.425+0000 7f32a4c00640 1 -- 172.21.15.28:0/819917872 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f32900acb00 con 0x7f32a80d5370 2024-04-26T20:57:03.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.425+0000 7f32aca00640 1 --2- 172.21.15.28:0/819917872 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f328406c8f0 0x7f328406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:03.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.426+0000 7f32aca00640 1 --2- 172.21.15.28:0/819917872 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f328406c8f0 0x7f328406edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f32a80ba270 tx=0x7f3298008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:03.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.429+0000 7f32a4c00640 1 -- 172.21.15.28:0/819917872 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f32a80d4b40 con 0x7f32a80d5370 2024-04-26T20:57:03.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.635+0000 7f32ade00640 1 -- 172.21.15.28:0/819917872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f32a8068bd0 con 0x7f32a80d5370 2024-04-26T20:57:03.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.638+0000 7f32a4c00640 1 -- 172.21.15.28:0/819917872 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7f32a80d0710 con 0x7f32a80d5370 2024-04-26T20:57:03.640 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:03.640 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":21,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:56:52.699663+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:57:03.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.641+0000 7f32ade00640 1 -- 172.21.15.28:0/819917872 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f328406c8f0 msgr2=0x7f328406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:03.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.641+0000 7f32ade00640 1 --2- 172.21.15.28:0/819917872 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f328406c8f0 0x7f328406edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f32a80ba270 tx=0x7f3298008040 comp rx=0 tx=0).stop 2024-04-26T20:57:03.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.641+0000 7f32ade00640 1 -- 172.21.15.28:0/819917872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32a80d5370 msgr2=0x7f32a80ce480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:03.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.641+0000 7f32ade00640 1 --2- 172.21.15.28:0/819917872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32a80d5370 0x7f32a80ce480 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3290009f10 tx=0x7f3290004a40 comp rx=0 tx=0).stop 2024-04-26T20:57:03.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.641+0000 7f32ade00640 1 -- 172.21.15.28:0/819917872 shutdown_connections 2024-04-26T20:57:03.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.641+0000 7f32ade00640 1 --2- 172.21.15.28:0/819917872 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f328406c8f0 0x7f328406edb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:03.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.641+0000 7f32ade00640 1 --2- 172.21.15.28:0/819917872 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f32a815f1b0 0x7f32a8161dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:03.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.641+0000 7f32ade00640 1 --2- 172.21.15.28:0/819917872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32a80d5370 0x7f32a80ce480 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:03.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.641+0000 7f32ade00640 1 --2- 172.21.15.28:0/819917872 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32a80d4700 0x7f32a8152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:03.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.641+0000 7f32ade00640 1 -- 172.21.15.28:0/819917872 >> 172.21.15.28:0/819917872 conn(0x7f32a806c770 msgr2=0x7f32a815a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:03.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.642+0000 7f32ade00640 1 -- 172.21.15.28:0/819917872 shutdown_connections 2024-04-26T20:57:03.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.642+0000 7f32ade00640 1 -- 172.21.15.28:0/819917872 wait complete. 2024-04-26T20:57:03.662 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:56:48.688498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:56:50.117541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:57:03.663 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:57:03.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.887+0000 7f6200600640 1 Processor -- start 2024-04-26T20:57:03.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.887+0000 7f6200600640 1 -- start start 2024-04-26T20:57:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.888+0000 7f6200600640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f61f8073f50 0x7f61f80ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.888+0000 7f6200600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f61f80ce8d0 0x7f61f806e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.888+0000 7f6200600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61f8074400 con 0x7f61f806fc10 2024-04-26T20:57:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.888+0000 7f6200600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f61f80ced00 con 0x7f61f80ce8d0 2024-04-26T20:57:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.888+0000 7f6200600640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f61f815e310 con 0x7f61f8073f50 2024-04-26T20:57:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.889+0000 7f61fe600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f61f80ce8d0 0x7f61f806e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.889+0000 7f61fe600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f61f80ce8d0 0x7f61f806e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35214/0 (socket says 172.21.15.28:35214) 2024-04-26T20:57:03.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.889+0000 7f61fe600640 1 -- 172.21.15.28:0/3484858018 learned_addr learned my addr 172.21.15.28:0/3484858018 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:03.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.889+0000 7f61fd200640 1 --2- 172.21.15.28:0/3484858018 >> v2:172.21.15.103:3300/0 conn(0x7f61f8073f50 0x7f61f80ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:03.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.889+0000 7f61fe600640 1 -- 172.21.15.28:0/3484858018 >> v2:172.21.15.103:3300/0 conn(0x7f61f8073f50 msgr2=0x7f61f80ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:03.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.889+0000 7f61fe600640 1 --2- 172.21.15.28:0/3484858018 >> v2:172.21.15.103:3300/0 conn(0x7f61f8073f50 0x7f61f80ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:03.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.889+0000 7f61fe600640 1 -- 172.21.15.28:0/3484858018 >> v1:172.21.15.28:6789/0 conn(0x7f61f806fc10 legacy=0x7f61f8073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:03.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.889+0000 7f61fe600640 1 -- 172.21.15.28:0/3484858018 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f8072aa0 con 0x7f61f80ce8d0 2024-04-26T20:57:03.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.891+0000 7f61fe600640 1 --2- 172.21.15.28:0/3484858018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61f80ce8d0 0x7f61f806e610 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f61f8070460 tx=0x7f61f002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532cea7606f78406 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:03.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.891+0000 7f61f7e00640 1 -- 172.21.15.28:0/3484858018 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f0036070 con 0x7f61f80ce8d0 2024-04-26T20:57:03.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.891+0000 7f61f7e00640 1 -- 172.21.15.28:0/3484858018 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61f0031600 con 0x7f61f80ce8d0 2024-04-26T20:57:03.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.891+0000 7f61f7e00640 1 -- 172.21.15.28:0/3484858018 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f00319d0 con 0x7f61f80ce8d0 2024-04-26T20:57:03.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.891+0000 7f6200600640 1 -- 172.21.15.28:0/3484858018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61f80ce8d0 msgr2=0x7f61f806e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:03.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.891+0000 7f6200600640 1 --2- 172.21.15.28:0/3484858018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61f80ce8d0 0x7f61f806e610 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f61f8070460 tx=0x7f61f002a6a0 comp rx=0 tx=0).stop 2024-04-26T20:57:03.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.892+0000 7f6200600640 1 -- 172.21.15.28:0/3484858018 shutdown_connections 2024-04-26T20:57:03.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.892+0000 7f6200600640 1 --2- 172.21.15.28:0/3484858018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61f80ce8d0 0x7f61f806e610 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:03.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.892+0000 7f6200600640 1 --2- 172.21.15.28:0/3484858018 >> v2:172.21.15.103:3300/0 conn(0x7f61f8073f50 0x7f61f80ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:03.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.892+0000 7f6200600640 1 -- 172.21.15.28:0/3484858018 >> 172.21.15.28:0/3484858018 conn(0x7f61f806c770 msgr2=0x7f61f806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:03.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.892+0000 7f6200600640 1 -- 172.21.15.28:0/3484858018 shutdown_connections 2024-04-26T20:57:03.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.892+0000 7f6200600640 1 -- 172.21.15.28:0/3484858018 wait complete. 2024-04-26T20:57:03.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.894+0000 7f6200600640 1 Processor -- start 2024-04-26T20:57:03.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.894+0000 7f6200600640 1 -- start start 2024-04-26T20:57:03.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.895+0000 7f6200600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61f806fc10 0x7f61f80ab580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:03.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.895+0000 7f6200600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61f8073f50 0x7f61f8152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:03.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.895+0000 7f6200600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61f80ce8d0 0x7f61f81557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:03.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.895+0000 7f6200600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f61f815dfa0 con 0x7f61f8073f50 2024-04-26T20:57:03.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.895+0000 7f6200600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f61f8073060 con 0x7f61f806fc10 2024-04-26T20:57:03.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.895+0000 7f6200600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f61f80d07d0 con 0x7f61f80ce8d0 2024-04-26T20:57:03.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.895+0000 7f61fd200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61f8073f50 0x7f61f8152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:03.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.895+0000 7f61fd200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61f8073f50 0x7f61f8152510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35228/0 (socket says 172.21.15.28:35228) 2024-04-26T20:57:03.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.895+0000 7f61fd200640 1 -- 172.21.15.28:0/3161984162 learned_addr learned my addr 172.21.15.28:0/3161984162 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:03.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.895+0000 7f61fdc00640 1 --2- 172.21.15.28:0/3161984162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61f806fc10 0x7f61f80ab580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:03.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.895+0000 7f61fe600640 1 --2- 172.21.15.28:0/3161984162 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61f80ce8d0 0x7f61f81557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:03.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.895+0000 7f61fd200640 1 -- 172.21.15.28:0/3161984162 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61f80ce8d0 msgr2=0x7f61f81557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:03.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.895+0000 7f61fd200640 1 --2- 172.21.15.28:0/3161984162 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61f80ce8d0 0x7f61f81557c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:03.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.895+0000 7f61fd200640 1 -- 172.21.15.28:0/3161984162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61f806fc10 msgr2=0x7f61f80ab580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:03.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.895+0000 7f61fd200640 1 --2- 172.21.15.28:0/3161984162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61f806fc10 0x7f61f80ab580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:03.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.895+0000 7f61fd200640 1 -- 172.21.15.28:0/3161984162 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f8152a50 con 0x7f61f8073f50 2024-04-26T20:57:03.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.895+0000 7f61fdc00640 1 --2- 172.21.15.28:0/3161984162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61f806fc10 0x7f61f80ab580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:03.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.896+0000 7f61fd200640 1 --2- 172.21.15.28:0/3161984162 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61f8073f50 0x7f61f8152510 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f61e400d5f0 tx=0x7f61e400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:03.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.896+0000 7f61f5600640 1 -- 172.21.15.28:0/3161984162 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e40125f0 con 0x7f61f8073f50 2024-04-26T20:57:03.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.896+0000 7f61f5600640 1 -- 172.21.15.28:0/3161984162 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61e4019070 con 0x7f61f8073f50 2024-04-26T20:57:03.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.896+0000 7f61f5600640 1 -- 172.21.15.28:0/3161984162 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e40129d0 con 0x7f61f8073f50 2024-04-26T20:57:03.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.896+0000 7f6200600640 1 -- 172.21.15.28:0/3161984162 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f8152cd0 con 0x7f61f8073f50 2024-04-26T20:57:03.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.896+0000 7f6200600640 1 -- 172.21.15.28:0/3161984162 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f8153290 con 0x7f61f8073f50 2024-04-26T20:57:03.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.897+0000 7f6200600640 1 -- 172.21.15.28:0/3161984162 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f806a360 con 0x7f61f8073f50 2024-04-26T20:57:03.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.899+0000 7f61f5600640 1 -- 172.21.15.28:0/3161984162 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61e4012b70 con 0x7f61f8073f50 2024-04-26T20:57:03.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.899+0000 7f61f5600640 1 --2- 172.21.15.28:0/3161984162 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f61d806c8f0 0x7f61d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:03.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.900+0000 7f61f5600640 1 -- 172.21.15.28:0/3161984162 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f61e408bf10 con 0x7f61f8073f50 2024-04-26T20:57:03.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.900+0000 7f61fdc00640 1 --2- 172.21.15.28:0/3161984162 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f61d806c8f0 0x7f61d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:03.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.900+0000 7f61fdc00640 1 --2- 172.21.15.28:0/3161984162 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f61d806c8f0 0x7f61d806edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f61f8153c50 tx=0x7f61ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:03.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:03.904+0000 7f61f5600640 1 -- 172.21.15.28:0/3161984162 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f61f8152a50 con 0x7f61f8073f50 2024-04-26T20:57:04.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.101+0000 7f6200600640 1 -- 172.21.15.28:0/3161984162 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f61f80b05a0 con 0x7f61f8073f50 2024-04-26T20:57:04.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.102+0000 7f61f5600640 1 -- 172.21.15.28:0/3161984162 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f61e405a3d0 con 0x7f61f8073f50 2024-04-26T20:57:04.103 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:57:04.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.105+0000 7f6200600640 1 -- 172.21.15.28:0/3161984162 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f61d806c8f0 msgr2=0x7f61d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:04.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.105+0000 7f6200600640 1 --2- 172.21.15.28:0/3161984162 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f61d806c8f0 0x7f61d806edb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f61f8153c50 tx=0x7f61ec009040 comp rx=0 tx=0).stop 2024-04-26T20:57:04.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.105+0000 7f6200600640 1 -- 172.21.15.28:0/3161984162 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61f8073f50 msgr2=0x7f61f8152510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:04.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.105+0000 7f6200600640 1 --2- 172.21.15.28:0/3161984162 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61f8073f50 0x7f61f8152510 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f61e400d5f0 tx=0x7f61e400dab0 comp rx=0 tx=0).stop 2024-04-26T20:57:04.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.105+0000 7f6200600640 1 -- 172.21.15.28:0/3161984162 shutdown_connections 2024-04-26T20:57:04.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.105+0000 7f6200600640 1 --2- 172.21.15.28:0/3161984162 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f61d806c8f0 0x7f61d806edb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:04.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.105+0000 7f6200600640 1 --2- 172.21.15.28:0/3161984162 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f61f80ce8d0 0x7f61f81557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:04.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.105+0000 7f6200600640 1 --2- 172.21.15.28:0/3161984162 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f61f8073f50 0x7f61f8152510 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:04.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.105+0000 7f6200600640 1 --2- 172.21.15.28:0/3161984162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61f806fc10 0x7f61f80ab580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:04.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.105+0000 7f6200600640 1 -- 172.21.15.28:0/3161984162 >> 172.21.15.28:0/3161984162 conn(0x7f61f806c770 msgr2=0x7f61f80b09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:04.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.105+0000 7f6200600640 1 -- 172.21.15.28:0/3161984162 shutdown_connections 2024-04-26T20:57:04.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.105+0000 7f6200600640 1 -- 172.21.15.28:0/3161984162 wait complete. 2024-04-26T20:57:04.126 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:57:04.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.331+0000 7fbd61200640 1 Processor -- start 2024-04-26T20:57:04.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.331+0000 7fbd61200640 1 -- start start 2024-04-26T20:57:04.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.333+0000 7fbd61200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbd5c073f50 0x7fbd5c0ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:04.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.333+0000 7fbd61200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbd5c0ce8d0 0x7fbd5c06e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:04.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.333+0000 7fbd61200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd5c074400 con 0x7fbd5c06fc10 2024-04-26T20:57:04.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.333+0000 7fbd61200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbd5c0ced00 con 0x7fbd5c073f50 2024-04-26T20:57:04.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.333+0000 7fbd61200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fbd5c15e310 con 0x7fbd5c0ce8d0 2024-04-26T20:57:04.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.333+0000 7fbd59400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbd5c073f50 0x7fbd5c0ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:04.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.333+0000 7fbd59400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbd5c073f50 0x7fbd5c0ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35234/0 (socket says 172.21.15.28:35234) 2024-04-26T20:57:04.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.333+0000 7fbd59400640 1 -- 172.21.15.28:0/92776068 learned_addr learned my addr 172.21.15.28:0/92776068 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:04.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.334+0000 7fbd5a800640 1 --2- 172.21.15.28:0/92776068 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbd5c0ce8d0 0x7fbd5c06e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:04.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.334+0000 7fbd59400640 1 -- 172.21.15.28:0/92776068 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbd5c0ce8d0 msgr2=0x7fbd5c06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:04.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.334+0000 7fbd59400640 1 --2- 172.21.15.28:0/92776068 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbd5c0ce8d0 0x7fbd5c06e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:04.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.334+0000 7fbd59400640 1 -- 172.21.15.28:0/92776068 >> v1:172.21.15.103:6789/0 conn(0x7fbd5c06fc10 legacy=0x7fbd5c073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:04.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.334+0000 7fbd59400640 1 -- 172.21.15.28:0/92776068 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5c072aa0 con 0x7fbd5c073f50 2024-04-26T20:57:04.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.335+0000 7fbd59400640 1 --2- 172.21.15.28:0/92776068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd5c073f50 0x7fbd5c0ce390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c070460 tx=0x7fbd4802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac411e9c1d7f6af9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:04.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.335+0000 7fbd58a00640 1 -- 172.21.15.28:0/92776068 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48037070 con 0x7fbd5c073f50 2024-04-26T20:57:04.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.335+0000 7fbd58a00640 1 -- 172.21.15.28:0/92776068 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd4802ae50 con 0x7fbd5c073f50 2024-04-26T20:57:04.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.335+0000 7fbd58a00640 1 -- 172.21.15.28:0/92776068 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48032660 con 0x7fbd5c073f50 2024-04-26T20:57:04.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.336+0000 7fbd61200640 1 -- 172.21.15.28:0/92776068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd5c073f50 msgr2=0x7fbd5c0ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:04.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.336+0000 7fbd61200640 1 --2- 172.21.15.28:0/92776068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd5c073f50 0x7fbd5c0ce390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c070460 tx=0x7fbd4802a140 comp rx=0 tx=0).stop 2024-04-26T20:57:04.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.336+0000 7fbd61200640 1 -- 172.21.15.28:0/92776068 shutdown_connections 2024-04-26T20:57:04.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.336+0000 7fbd61200640 1 --2- 172.21.15.28:0/92776068 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbd5c0ce8d0 0x7fbd5c06e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:04.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.336+0000 7fbd61200640 1 --2- 172.21.15.28:0/92776068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd5c073f50 0x7fbd5c0ce390 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:04.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.336+0000 7fbd61200640 1 -- 172.21.15.28:0/92776068 >> 172.21.15.28:0/92776068 conn(0x7fbd5c06c770 msgr2=0x7fbd5c06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:04.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.336+0000 7fbd61200640 1 -- 172.21.15.28:0/92776068 shutdown_connections 2024-04-26T20:57:04.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.336+0000 7fbd61200640 1 -- 172.21.15.28:0/92776068 wait complete. 2024-04-26T20:57:04.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.338+0000 7fbd61200640 1 Processor -- start 2024-04-26T20:57:04.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.338+0000 7fbd61200640 1 -- start start 2024-04-26T20:57:04.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.339+0000 7fbd61200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd5c06fc10 0x7fbd5c0aa600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:04.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.339+0000 7fbd61200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd5c073f50 0x7fbd5c152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:04.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.339+0000 7fbd61200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbd5c0ce8d0 0x7fbd5c1465a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:04.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.339+0000 7fbd61200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c15dfa0 con 0x7fbd5c073f50 2024-04-26T20:57:04.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.339+0000 7fbd61200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c073060 con 0x7fbd5c06fc10 2024-04-26T20:57:04.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.339+0000 7fbd61200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fbd5c0d07d0 con 0x7fbd5c0ce8d0 2024-04-26T20:57:04.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.339+0000 7fbd59400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd5c073f50 0x7fbd5c152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:04.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.339+0000 7fbd59400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd5c073f50 0x7fbd5c152c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35246/0 (socket says 172.21.15.28:35246) 2024-04-26T20:57:04.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.340+0000 7fbd59400640 1 -- 172.21.15.28:0/519514237 learned_addr learned my addr 172.21.15.28:0/519514237 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:04.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.340+0000 7fbd59e00640 1 --2- 172.21.15.28:0/519514237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd5c06fc10 0x7fbd5c0aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:04.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.340+0000 7fbd5a800640 1 --2- 172.21.15.28:0/519514237 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbd5c0ce8d0 0x7fbd5c1465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:04.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.340+0000 7fbd59400640 1 -- 172.21.15.28:0/519514237 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbd5c0ce8d0 msgr2=0x7fbd5c1465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:04.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.340+0000 7fbd59400640 1 --2- 172.21.15.28:0/519514237 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbd5c0ce8d0 0x7fbd5c1465a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:04.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.340+0000 7fbd59400640 1 -- 172.21.15.28:0/519514237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd5c06fc10 msgr2=0x7fbd5c0aa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:04.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.340+0000 7fbd59400640 1 --2- 172.21.15.28:0/519514237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd5c06fc10 0x7fbd5c0aa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:04.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.340+0000 7fbd59400640 1 -- 172.21.15.28:0/519514237 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5c1533b0 con 0x7fbd5c073f50 2024-04-26T20:57:04.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.340+0000 7fbd5a800640 1 --2- 172.21.15.28:0/519514237 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbd5c0ce8d0 0x7fbd5c1465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:04.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.340+0000 7fbd59400640 1 --2- 172.21.15.28:0/519514237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd5c073f50 0x7fbd5c152c40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbd48000c00 tx=0x7fbd480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:04.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.341+0000 7fbd52000640 1 -- 172.21.15.28:0/519514237 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48041070 con 0x7fbd5c073f50 2024-04-26T20:57:04.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.341+0000 7fbd52000640 1 -- 172.21.15.28:0/519514237 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd4803c670 con 0x7fbd5c073f50 2024-04-26T20:57:04.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.341+0000 7fbd52000640 1 -- 172.21.15.28:0/519514237 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd48037040 con 0x7fbd5c073f50 2024-04-26T20:57:04.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.341+0000 7fbd61200640 1 -- 172.21.15.28:0/519514237 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5c153690 con 0x7fbd5c073f50 2024-04-26T20:57:04.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.341+0000 7fbd61200640 1 -- 172.21.15.28:0/519514237 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd5c153c50 con 0x7fbd5c073f50 2024-04-26T20:57:04.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.342+0000 7fbd61200640 1 -- 172.21.15.28:0/519514237 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd5c06a360 con 0x7fbd5c073f50 2024-04-26T20:57:04.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.343+0000 7fbd52000640 1 -- 172.21.15.28:0/519514237 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd48044070 con 0x7fbd5c073f50 2024-04-26T20:57:04.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.344+0000 7fbd52000640 1 --2- 172.21.15.28:0/519514237 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbd3c06c8f0 0x7fbd3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:04.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.344+0000 7fbd52000640 1 -- 172.21.15.28:0/519514237 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbd480ac030 con 0x7fbd5c073f50 2024-04-26T20:57:04.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.344+0000 7fbd59e00640 1 --2- 172.21.15.28:0/519514237 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbd3c06c8f0 0x7fbd3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:04.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.345+0000 7fbd59e00640 1 --2- 172.21.15.28:0/519514237 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbd3c06c8f0 0x7fbd3c06edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c006fd0 tx=0x7fbd4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:04.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.349+0000 7fbd52000640 1 -- 172.21.15.28:0/519514237 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbd4807a450 con 0x7fbd5c073f50 2024-04-26T20:57:04.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.554+0000 7fbd61200640 1 -- 172.21.15.28:0/519514237 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbd5c15f3d0 con 0x7fbd5c073f50 2024-04-26T20:57:04.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.555+0000 7fbd52000640 1 -- 172.21.15.28:0/519514237 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fbd48079d50 con 0x7fbd5c073f50 2024-04-26T20:57:04.556 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-04-26T20:57:04.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.557+0000 7fbd61200640 1 -- 172.21.15.28:0/519514237 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbd3c06c8f0 msgr2=0x7fbd3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:04.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.557+0000 7fbd61200640 1 --2- 172.21.15.28:0/519514237 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbd3c06c8f0 0x7fbd3c06edb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c006fd0 tx=0x7fbd4c008040 comp rx=0 tx=0).stop 2024-04-26T20:57:04.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.557+0000 7fbd61200640 1 -- 172.21.15.28:0/519514237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd5c073f50 msgr2=0x7fbd5c152c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:04.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.557+0000 7fbd61200640 1 --2- 172.21.15.28:0/519514237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd5c073f50 0x7fbd5c152c40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbd48000c00 tx=0x7fbd480046c0 comp rx=0 tx=0).stop 2024-04-26T20:57:04.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.557+0000 7fbd61200640 1 -- 172.21.15.28:0/519514237 shutdown_connections 2024-04-26T20:57:04.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.557+0000 7fbd61200640 1 --2- 172.21.15.28:0/519514237 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbd3c06c8f0 0x7fbd3c06edb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:04.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.557+0000 7fbd61200640 1 --2- 172.21.15.28:0/519514237 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbd5c0ce8d0 0x7fbd5c1465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:04.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.557+0000 7fbd61200640 1 --2- 172.21.15.28:0/519514237 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd5c073f50 0x7fbd5c152c40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:04.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.557+0000 7fbd61200640 1 --2- 172.21.15.28:0/519514237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd5c06fc10 0x7fbd5c0aa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:04.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.557+0000 7fbd61200640 1 -- 172.21.15.28:0/519514237 >> 172.21.15.28:0/519514237 conn(0x7fbd5c06c770 msgr2=0x7fbd5c0ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:04.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.558+0000 7fbd61200640 1 -- 172.21.15.28:0/519514237 shutdown_connections 2024-04-26T20:57:04.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.558+0000 7fbd61200640 1 -- 172.21.15.28:0/519514237 wait complete. 2024-04-26T20:57:04.578 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:57:04.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.786+0000 7fa3b4200640 1 Processor -- start 2024-04-26T20:57:04.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.786+0000 7fa3b4200640 1 -- start start 2024-04-26T20:57:04.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.787+0000 7fa3b4200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa3ac146860 0x7fa3ac06f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:04.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.787+0000 7fa3b4200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3ac0b4d90 con 0x7fa3ac0b4730 2024-04-26T20:57:04.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.787+0000 7fa3b4200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3ac15d850 con 0x7fa3ac06f810 2024-04-26T20:57:04.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.787+0000 7fa3b4200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa3ac06fc70 con 0x7fa3ac146860 2024-04-26T20:57:04.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.788+0000 7fa3b1800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa3ac146860 0x7fa3ac06f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:04.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.788+0000 7fa3b1800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa3ac146860 0x7fa3ac06f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35256/0 (socket says 172.21.15.28:35256) 2024-04-26T20:57:04.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.788+0000 7fa3b1800640 1 -- 172.21.15.28:0/3722571027 learned_addr learned my addr 172.21.15.28:0/3722571027 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:04.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.788+0000 7fa3b2200640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa3ac0b4730 0x7fa3ac15e330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47774/0 (socket says 172.21.15.28:47774) 2024-04-26T20:57:04.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.788+0000 7fa3b1800640 1 -- 172.21.15.28:0/3722571027 >> v1:172.21.15.103:6789/0 conn(0x7fa3ac06f810 legacy=0x7fa3ac0b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:04.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.788+0000 7fa3b1800640 1 -- 172.21.15.28:0/3722571027 >> v1:172.21.15.28:6789/0 conn(0x7fa3ac0b4730 legacy=0x7fa3ac15e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:04.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.788+0000 7fa3abe00640 1 -- 172.21.15.28:0/3722571027 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1477839734 0 0) 0x7fa3ac0b4d90 con 0x7fa3ac0b4730 2024-04-26T20:57:04.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.788+0000 7fa3b1800640 1 -- 172.21.15.28:0/3722571027 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ac15db30 con 0x7fa3ac146860 2024-04-26T20:57:04.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.790+0000 7fa3b1800640 1 --2- 172.21.15.28:0/3722571027 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac146860 0x7fa3ac06f2d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009870 tx=0x7fa3a002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa2b1d0d4066aec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:04.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.790+0000 7fa3abe00640 1 -- 172.21.15.28:0/3722571027 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0037070 con 0x7fa3ac146860 2024-04-26T20:57:04.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.790+0000 7fa3abe00640 1 -- 172.21.15.28:0/3722571027 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a002ae50 con 0x7fa3ac146860 2024-04-26T20:57:04.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.790+0000 7fa3abe00640 1 -- 172.21.15.28:0/3722571027 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0032860 con 0x7fa3ac146860 2024-04-26T20:57:04.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.790+0000 7fa3b4200640 1 -- 172.21.15.28:0/3722571027 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac146860 msgr2=0x7fa3ac06f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:04.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.790+0000 7fa3b4200640 1 --2- 172.21.15.28:0/3722571027 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac146860 0x7fa3ac06f2d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009870 tx=0x7fa3a002a300 comp rx=0 tx=0).stop 2024-04-26T20:57:04.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.791+0000 7fa3b4200640 1 -- 172.21.15.28:0/3722571027 shutdown_connections 2024-04-26T20:57:04.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.791+0000 7fa3b4200640 1 --2- 172.21.15.28:0/3722571027 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac146860 0x7fa3ac06f2d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:04.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.791+0000 7fa3b4200640 1 -- 172.21.15.28:0/3722571027 >> 172.21.15.28:0/3722571027 conn(0x7fa3ac0b2b60 msgr2=0x7fa3ac0b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:04.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.791+0000 7fa3b4200640 1 -- 172.21.15.28:0/3722571027 shutdown_connections 2024-04-26T20:57:04.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.791+0000 7fa3b4200640 1 -- 172.21.15.28:0/3722571027 wait complete. 2024-04-26T20:57:04.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.792+0000 7fa3b4200640 1 Processor -- start 2024-04-26T20:57:04.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.792+0000 7fa3b4200640 1 -- start start 2024-04-26T20:57:04.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.793+0000 7fa3b4200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3ac06f810 0x7fa3ac1429e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:04.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.793+0000 7fa3b4200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac0b4730 0x7fa3ac072d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:04.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.793+0000 7fa3b4200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3ac146860 0x7fa3ac1360c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:04.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.793+0000 7fa3b4200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac15ebc0 con 0x7fa3ac0b4730 2024-04-26T20:57:04.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.793+0000 7fa3b4200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac15ea40 con 0x7fa3ac06f810 2024-04-26T20:57:04.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.793+0000 7fa3b4200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa3ac15ed40 con 0x7fa3ac146860 2024-04-26T20:57:04.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.794+0000 7fa3b0e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac0b4730 0x7fa3ac072d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:04.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.794+0000 7fa3b0e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac0b4730 0x7fa3ac072d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35266/0 (socket says 172.21.15.28:35266) 2024-04-26T20:57:04.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.794+0000 7fa3b0e00640 1 -- 172.21.15.28:0/2636700713 learned_addr learned my addr 172.21.15.28:0/2636700713 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:04.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.794+0000 7fa3b1800640 1 --2- 172.21.15.28:0/2636700713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3ac06f810 0x7fa3ac1429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:04.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.794+0000 7fa3b2200640 1 --2- 172.21.15.28:0/2636700713 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3ac146860 0x7fa3ac1360c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:04.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.794+0000 7fa3b0e00640 1 -- 172.21.15.28:0/2636700713 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3ac146860 msgr2=0x7fa3ac1360c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:04.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.794+0000 7fa3b0e00640 1 --2- 172.21.15.28:0/2636700713 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3ac146860 0x7fa3ac1360c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:04.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.794+0000 7fa3b0e00640 1 -- 172.21.15.28:0/2636700713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3ac06f810 msgr2=0x7fa3ac1429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:04.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.794+0000 7fa3b0e00640 1 --2- 172.21.15.28:0/2636700713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3ac06f810 0x7fa3ac1429e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:04.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.794+0000 7fa3b0e00640 1 -- 172.21.15.28:0/2636700713 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ac157a50 con 0x7fa3ac0b4730 2024-04-26T20:57:04.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.795+0000 7fa3b0e00640 1 --2- 172.21.15.28:0/2636700713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac0b4730 0x7fa3ac072d70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa39800c480 tx=0x7fa39800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:04.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.795+0000 7fa3a9600640 1 -- 172.21.15.28:0/2636700713 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3980052d0 con 0x7fa3ac0b4730 2024-04-26T20:57:04.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.795+0000 7fa3a9600640 1 -- 172.21.15.28:0/2636700713 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa398005470 con 0x7fa3ac0b4730 2024-04-26T20:57:04.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.795+0000 7fa3a9600640 1 -- 172.21.15.28:0/2636700713 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398012850 con 0x7fa3ac0b4730 2024-04-26T20:57:04.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.795+0000 7fa3b4200640 1 -- 172.21.15.28:0/2636700713 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3ac157d30 con 0x7fa3ac0b4730 2024-04-26T20:57:04.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.795+0000 7fa3b4200640 1 -- 172.21.15.28:0/2636700713 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3ac1582f0 con 0x7fa3ac0b4730 2024-04-26T20:57:04.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.796+0000 7fa3b4200640 1 -- 172.21.15.28:0/2636700713 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3ac076520 con 0x7fa3ac0b4730 2024-04-26T20:57:04.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.797+0000 7fa3a9600640 1 -- 172.21.15.28:0/2636700713 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3980129f0 con 0x7fa3ac0b4730 2024-04-26T20:57:04.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.798+0000 7fa3a9600640 1 --2- 172.21.15.28:0/2636700713 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa38c06c8f0 0x7fa38c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:04.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.798+0000 7fa3a9600640 1 -- 172.21.15.28:0/2636700713 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa39808b890 con 0x7fa3ac0b4730 2024-04-26T20:57:04.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.798+0000 7fa3b1800640 1 --2- 172.21.15.28:0/2636700713 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa38c06c8f0 0x7fa38c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:04.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.799+0000 7fa3b1800640 1 --2- 172.21.15.28:0/2636700713 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa38c06c8f0 0x7fa38c06edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0034000 tx=0x7fa3a002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:04.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:04.803+0000 7fa3a9600640 1 -- 172.21.15.28:0/2636700713 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa398059cb0 con 0x7fa3ac0b4730 2024-04-26T20:57:05.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.007+0000 7fa3b4200640 1 -- 172.21.15.28:0/2636700713 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa3ac157780 con 0x7fa3ac0b4730 2024-04-26T20:57:05.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.008+0000 7fa3a9600640 1 -- 172.21.15.28:0/2636700713 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fa3980595b0 con 0x7fa3ac0b4730 2024-04-26T20:57:05.009 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-04-26T20:57:05.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.010+0000 7fa3b4200640 1 -- 172.21.15.28:0/2636700713 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa38c06c8f0 msgr2=0x7fa38c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:05.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.010+0000 7fa3b4200640 1 --2- 172.21.15.28:0/2636700713 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa38c06c8f0 0x7fa38c06edb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0034000 tx=0x7fa3a002e000 comp rx=0 tx=0).stop 2024-04-26T20:57:05.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.010+0000 7fa3b4200640 1 -- 172.21.15.28:0/2636700713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac0b4730 msgr2=0x7fa3ac072d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:05.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.010+0000 7fa3b4200640 1 --2- 172.21.15.28:0/2636700713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac0b4730 0x7fa3ac072d70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa39800c480 tx=0x7fa39800c940 comp rx=0 tx=0).stop 2024-04-26T20:57:05.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.011+0000 7fa3b4200640 1 -- 172.21.15.28:0/2636700713 shutdown_connections 2024-04-26T20:57:05.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.011+0000 7fa3b4200640 1 --2- 172.21.15.28:0/2636700713 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa38c06c8f0 0x7fa38c06edb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.011+0000 7fa3b4200640 1 --2- 172.21.15.28:0/2636700713 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3ac146860 0x7fa3ac1360c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.011+0000 7fa3b4200640 1 --2- 172.21.15.28:0/2636700713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3ac0b4730 0x7fa3ac072d70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.011+0000 7fa3b4200640 1 --2- 172.21.15.28:0/2636700713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3ac06f810 0x7fa3ac1429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.011+0000 7fa3b4200640 1 -- 172.21.15.28:0/2636700713 >> 172.21.15.28:0/2636700713 conn(0x7fa3ac0b2b60 msgr2=0x7fa3ac15cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:05.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.011+0000 7fa3b4200640 1 -- 172.21.15.28:0/2636700713 shutdown_connections 2024-04-26T20:57:05.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.011+0000 7fa3b4200640 1 -- 172.21.15.28:0/2636700713 wait complete. 2024-04-26T20:57:05.030 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:57:05.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.230+0000 7f009dc00640 1 Processor -- start 2024-04-26T20:57:05.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.230+0000 7f009dc00640 1 -- start start 2024-04-26T20:57:05.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.232+0000 7f009dc00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0098146670 0x7f0098071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:05.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.232+0000 7f009dc00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f00981610c0 0x7f00980cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:05.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.232+0000 7f009dc00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0098161530 con 0x7f0098146060 2024-04-26T20:57:05.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.232+0000 7f009dc00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f009806f510 con 0x7f00981610c0 2024-04-26T20:57:05.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.232+0000 7f009dc00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f0098146440 con 0x7f0098146670 2024-04-26T20:57:05.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.232+0000 7f0097e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f0098146060 0x7f00981609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47780/0 (socket says 172.21.15.28:47780) 2024-04-26T20:57:05.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.232+0000 7f0097e00640 1 -- 172.21.15.28:0/3931295225 learned_addr learned my addr 172.21.15.28:0/3931295225 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:05.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.233+0000 7f009d200640 1 --2- 172.21.15.28:0/3931295225 >> v2:172.21.15.103:3300/0 conn(0x7f00981610c0 0x7f00980cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:05.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.233+0000 7f008f400640 1 --2- 172.21.15.28:0/3931295225 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0098146670 0x7f0098071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:05.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.233+0000 7f0097400640 1 -- 172.21.15.28:0/3931295225 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3270501334 0 0) 0x7f0098161530 con 0x7f0098146060 2024-04-26T20:57:05.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.233+0000 7f0097400640 1 -- 172.21.15.28:0/3931295225 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f007c003520 con 0x7f0098146060 2024-04-26T20:57:05.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.233+0000 7f0097400640 1 -- 172.21.15.28:0/3931295225 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2145899422 0 0) 0x7f007c003520 con 0x7f0098146060 2024-04-26T20:57:05.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.233+0000 7f0097400640 1 -- 172.21.15.28:0/3931295225 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0098146670 msgr2=0x7f0098071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:05.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.233+0000 7f0097400640 1 --2- 172.21.15.28:0/3931295225 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0098146670 0x7f0098071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.233+0000 7f0097400640 1 -- 172.21.15.28:0/3931295225 >> v2:172.21.15.103:3300/0 conn(0x7f00981610c0 msgr2=0x7f00980cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:05.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.233+0000 7f0097400640 1 --2- 172.21.15.28:0/3931295225 >> v2:172.21.15.103:3300/0 conn(0x7f00981610c0 0x7f00980cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.233+0000 7f0097400640 1 -- 172.21.15.28:0/3931295225 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009806f010 con 0x7f0098146060 2024-04-26T20:57:05.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.233+0000 7f0097400640 1 -- 172.21.15.28:0/3931295225 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f0088001670 con 0x7f0098146060 2024-04-26T20:57:05.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.233+0000 7f0097400640 1 -- 172.21.15.28:0/3931295225 >> v1:172.21.15.28:6789/0 conn(0x7f0098146060 legacy=0x7f00981609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:05.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.233+0000 7f0097400640 1 --2- 172.21.15.28:0/3931295225 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f007c0039b0 0x7f007c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:05.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.233+0000 7f0097400640 1 --2- 172.21.15.28:0/3931295225 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0098146670 0x7f007c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:05.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.233+0000 7f0097400640 1 --2- 172.21.15.28:0/3931295225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f007c004d40 0x7f007c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:05.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.233+0000 7f0097400640 1 -- 172.21.15.28:0/3931295225 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f009806f510 con 0x7f007c0039b0 2024-04-26T20:57:05.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.233+0000 7f0097400640 1 -- 172.21.15.28:0/3931295225 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0098146440 con 0x7f007c004d40 2024-04-26T20:57:05.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.233+0000 7f0097400640 1 -- 172.21.15.28:0/3931295225 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f007c00baa0 con 0x7f0098146670 2024-04-26T20:57:05.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.233+0000 7f009d200640 1 --2- 172.21.15.28:0/3931295225 >> v2:172.21.15.103:3300/0 conn(0x7f00981610c0 0x7f00980cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:57:05.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.234+0000 7f0097e00640 1 --2- 172.21.15.28:0/3931295225 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f007c0039b0 0x7f007c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:05.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.234+0000 7f008f400640 1 --2- 172.21.15.28:0/3931295225 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0098146670 0x7f007c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:05.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.234+0000 7f009d200640 1 --2- 172.21.15.28:0/3931295225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f007c004d40 0x7f007c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:05.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.234+0000 7f0097e00640 1 -- 172.21.15.28:0/3931295225 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0098146670 msgr2=0x7f007c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:05.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.234+0000 7f0097e00640 1 --2- 172.21.15.28:0/3931295225 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0098146670 0x7f007c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.234+0000 7f0097e00640 1 -- 172.21.15.28:0/3931295225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f007c004d40 msgr2=0x7f007c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:05.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.234+0000 7f0097e00640 1 --2- 172.21.15.28:0/3931295225 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f007c004d40 0x7f007c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.234+0000 7f0097e00640 1 -- 172.21.15.28:0/3931295225 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007c003520 con 0x7f007c0039b0 2024-04-26T20:57:05.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.235+0000 7f0097e00640 1 --2- 172.21.15.28:0/3931295225 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f007c0039b0 0x7f007c003dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f008800c210 tx=0x7f007c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b9151b2950b6f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:05.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.235+0000 7f0097400640 1 -- 172.21.15.28:0/3931295225 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0088007d50 con 0x7f007c0039b0 2024-04-26T20:57:05.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.236+0000 7f0097400640 1 -- 172.21.15.28:0/3931295225 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0088005af0 con 0x7f007c0039b0 2024-04-26T20:57:05.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.236+0000 7f0097400640 1 -- 172.21.15.28:0/3931295225 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0088007070 con 0x7f007c0039b0 2024-04-26T20:57:05.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.236+0000 7f009dc00640 1 -- 172.21.15.28:0/3931295225 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f007c0039b0 msgr2=0x7f007c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:05.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.236+0000 7f009dc00640 1 --2- 172.21.15.28:0/3931295225 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f007c0039b0 0x7f007c003dd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f008800c210 tx=0x7f007c005830 comp rx=0 tx=0).stop 2024-04-26T20:57:05.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.236+0000 7f009d200640 1 -- 172.21.15.28:0/3931295225 reap_dead start 2024-04-26T20:57:05.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.236+0000 7f009dc00640 1 -- 172.21.15.28:0/3931295225 shutdown_connections 2024-04-26T20:57:05.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.236+0000 7f009dc00640 1 -- 172.21.15.28:0/3931295225 >> 172.21.15.28:0/3931295225 conn(0x7f00980d0a50 msgr2=0x7f00980d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:05.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.236+0000 7f009dc00640 1 -- 172.21.15.28:0/3931295225 shutdown_connections 2024-04-26T20:57:05.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.236+0000 7f009dc00640 1 -- 172.21.15.28:0/3931295225 wait complete. 2024-04-26T20:57:05.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.238+0000 7f009dc00640 1 Processor -- start 2024-04-26T20:57:05.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.239+0000 7f009dc00640 1 -- start start 2024-04-26T20:57:05.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.240+0000 7f009dc00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f007c0039b0 0x7f0098142a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:05.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.240+0000 7f009dc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f009813db60 0x7f009815d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:05.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.240+0000 7f009dc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f009815da20 0x7f0098143660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:05.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.240+0000 7f009dc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f009806ff60 con 0x7f009813db60 2024-04-26T20:57:05.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.240+0000 7f009dc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f009806fc80 con 0x7f009815da20 2024-04-26T20:57:05.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.240+0000 7f009dc00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f0098071e20 con 0x7f007c0039b0 2024-04-26T20:57:05.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.240+0000 7f008f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f009813db60 0x7f009815d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:05.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.240+0000 7f008f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f009813db60 0x7f009815d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35280/0 (socket says 172.21.15.28:35280) 2024-04-26T20:57:05.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.240+0000 7f008f400640 1 -- 172.21.15.28:0/3331705218 learned_addr learned my addr 172.21.15.28:0/3331705218 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:05.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.240+0000 7f0097e00640 1 --2- 172.21.15.28:0/3331705218 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f007c0039b0 0x7f0098142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:05.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.240+0000 7f009d200640 1 --2- 172.21.15.28:0/3331705218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f009815da20 0x7f0098143660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:05.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.241+0000 7f008f400640 1 -- 172.21.15.28:0/3331705218 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f007c0039b0 msgr2=0x7f0098142a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:05.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.241+0000 7f008f400640 1 --2- 172.21.15.28:0/3331705218 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f007c0039b0 0x7f0098142a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.241+0000 7f008f400640 1 -- 172.21.15.28:0/3331705218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f009815da20 msgr2=0x7f0098143660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:05.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.241+0000 7f008f400640 1 --2- 172.21.15.28:0/3331705218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f009815da20 0x7f0098143660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.241+0000 7f008f400640 1 -- 172.21.15.28:0/3331705218 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00981419f0 con 0x7f009813db60 2024-04-26T20:57:05.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.241+0000 7f008f400640 1 --2- 172.21.15.28:0/3331705218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f009813db60 0x7f009815d4e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0080011600 tx=0x7f0080011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:05.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.241+0000 7f0094c00640 1 -- 172.21.15.28:0/3331705218 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008000f8c0 con 0x7f009813db60 2024-04-26T20:57:05.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.241+0000 7f0094c00640 1 -- 172.21.15.28:0/3331705218 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f008000fa60 con 0x7f009813db60 2024-04-26T20:57:05.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.241+0000 7f0094c00640 1 -- 172.21.15.28:0/3331705218 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008000fe10 con 0x7f009813db60 2024-04-26T20:57:05.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.241+0000 7f009dc00640 1 -- 172.21.15.28:0/3331705218 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0098141c90 con 0x7f009813db60 2024-04-26T20:57:05.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.241+0000 7f009dc00640 1 -- 172.21.15.28:0/3331705218 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0098142250 con 0x7f009813db60 2024-04-26T20:57:05.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.242+0000 7f009dc00640 1 -- 172.21.15.28:0/3331705218 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f009806a360 con 0x7f009813db60 2024-04-26T20:57:05.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.243+0000 7f0094c00640 1 -- 172.21.15.28:0/3331705218 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0080015460 con 0x7f009813db60 2024-04-26T20:57:05.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.244+0000 7f0094c00640 1 --2- 172.21.15.28:0/3331705218 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f007406c920 0x7f007406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:05.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.245+0000 7f0094c00640 1 -- 172.21.15.28:0/3331705218 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f008008c510 con 0x7f009813db60 2024-04-26T20:57:05.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.245+0000 7f0097e00640 1 --2- 172.21.15.28:0/3331705218 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f007406c920 0x7f007406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:05.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.245+0000 7f0097e00640 1 --2- 172.21.15.28:0/3331705218 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f007406c920 0x7f007406ede0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0088033000 tx=0x7f0088009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:05.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.249+0000 7f0094c00640 1 -- 172.21.15.28:0/3331705218 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0080019030 con 0x7f009813db60 2024-04-26T20:57:05.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.484+0000 7f009dc00640 1 -- 172.21.15.28:0/3331705218 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f009815fc60 con 0x7f009813db60 2024-04-26T20:57:05.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.485+0000 7f0094c00640 1 -- 172.21.15.28:0/3331705218 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7f008005a8b0 con 0x7f009813db60 2024-04-26T20:57:05.487 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:05.487 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/2818824721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2818824721},{"type":"v1","addr":"172.21.15.103:6834","nonce":2818824721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/259712913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":259712913},{"type":"v1","addr":"172.21.15.28:6836","nonce":259712913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/989575415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":989575415},{"type":"v1","addr":"172.21.15.28:6837","nonce":989575415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:56:51.143688+0000","modified":"2024-04-26T20:56:54.228571+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6835/2361950169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":2361950169},{"type":"v1","addr":"172.21.15.103:6835","nonce":2361950169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-26T20:57:05.487 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 8 2024-04-26T20:57:05.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.488+0000 7f009dc00640 1 -- 172.21.15.28:0/3331705218 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f007406c920 msgr2=0x7f007406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:05.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.488+0000 7f009dc00640 1 --2- 172.21.15.28:0/3331705218 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f007406c920 0x7f007406ede0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0088033000 tx=0x7f0088009030 comp rx=0 tx=0).stop 2024-04-26T20:57:05.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.488+0000 7f009dc00640 1 -- 172.21.15.28:0/3331705218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f009813db60 msgr2=0x7f009815d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:05.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.488+0000 7f009dc00640 1 --2- 172.21.15.28:0/3331705218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f009813db60 0x7f009815d4e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0080011600 tx=0x7f0080011ac0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.488+0000 7f009dc00640 1 -- 172.21.15.28:0/3331705218 shutdown_connections 2024-04-26T20:57:05.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.488+0000 7f009dc00640 1 --2- 172.21.15.28:0/3331705218 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f007406c920 0x7f007406ede0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.488+0000 7f009dc00640 1 --2- 172.21.15.28:0/3331705218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f009815da20 0x7f0098143660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.488+0000 7f009dc00640 1 --2- 172.21.15.28:0/3331705218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f009813db60 0x7f009815d4e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.488+0000 7f009dc00640 1 --2- 172.21.15.28:0/3331705218 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f007c0039b0 0x7f0098142a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.488+0000 7f009dc00640 1 -- 172.21.15.28:0/3331705218 >> 172.21.15.28:0/3331705218 conn(0x7f00980d0a50 msgr2=0x7f009806de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:05.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.488+0000 7f009dc00640 1 -- 172.21.15.28:0/3331705218 shutdown_connections 2024-04-26T20:57:05.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.488+0000 7f009dc00640 1 -- 172.21.15.28:0/3331705218 wait complete. 2024-04-26T20:57:05.507 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:57:05.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.714+0000 7fee7f400640 1 Processor -- start 2024-04-26T20:57:05.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.715+0000 7fee7f400640 1 -- start start 2024-04-26T20:57:05.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.716+0000 7fee7f400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee780b6810 0x7fee7815d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:05.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.716+0000 7fee7f400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee780d0a30 con 0x7fee78146980 2024-04-26T20:57:05.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.716+0000 7fee7f400640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee7815e230 con 0x7fee780740e0 2024-04-26T20:57:05.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.716+0000 7fee7f400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fee7815e510 con 0x7fee780b6810 2024-04-26T20:57:05.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.716+0000 7fee7ca00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fee78146980 0x7fee780b6100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47784/0 (socket says 172.21.15.28:47784) 2024-04-26T20:57:05.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.716+0000 7fee7ca00640 1 -- 172.21.15.28:0/1414365751 learned_addr learned my addr 172.21.15.28:0/1414365751 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:05.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.717+0000 7fee77e00640 1 --2- 172.21.15.28:0/1414365751 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee780b6810 0x7fee7815d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:05.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.717+0000 7fee77400640 1 -- 172.21.15.28:0/1414365751 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2838210511 0 0) 0x7fee68001490 con 0x7fee78146980 2024-04-26T20:57:05.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.717+0000 7fee77400640 1 -- 172.21.15.28:0/1414365751 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fee5c003520 con 0x7fee78146980 2024-04-26T20:57:05.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.717+0000 7fee77400640 1 -- 172.21.15.28:0/1414365751 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3361202552 0 0) 0x7fee5c003520 con 0x7fee78146980 2024-04-26T20:57:05.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.717+0000 7fee77400640 1 -- 172.21.15.28:0/1414365751 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee780b6810 msgr2=0x7fee7815d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:05.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.717+0000 7fee77400640 1 --2- 172.21.15.28:0/1414365751 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee780b6810 0x7fee7815d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.717+0000 7fee77400640 1 -- 172.21.15.28:0/1414365751 >> v1:172.21.15.103:6789/0 conn(0x7fee780740e0 legacy=0x7fee78071ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:05.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.717+0000 7fee77400640 1 -- 172.21.15.28:0/1414365751 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee78073600 con 0x7fee78146980 2024-04-26T20:57:05.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.717+0000 7fee77400640 1 -- 172.21.15.28:0/1414365751 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fee68001bc0 con 0x7fee78146980 2024-04-26T20:57:05.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.717+0000 7fee77400640 1 -- 172.21.15.28:0/1414365751 >> v1:172.21.15.28:6789/0 conn(0x7fee78146980 legacy=0x7fee780b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:05.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.717+0000 7fee77400640 1 --2- 172.21.15.28:0/1414365751 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee780b6810 0x7fee5c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:05.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.717+0000 7fee77400640 1 --2- 172.21.15.28:0/1414365751 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee5c003f60 0x7fee5c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:05.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.717+0000 7fee77400640 1 --2- 172.21.15.28:0/1414365751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee5c004d40 0x7fee5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:05.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.717+0000 7fee77400640 1 -- 172.21.15.28:0/1414365751 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fee7815e510 con 0x7fee5c003f60 2024-04-26T20:57:05.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.717+0000 7fee77400640 1 -- 172.21.15.28:0/1414365751 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee5c00bb50 con 0x7fee5c004d40 2024-04-26T20:57:05.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.717+0000 7fee77400640 1 -- 172.21.15.28:0/1414365751 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fee5c00bd50 con 0x7fee780b6810 2024-04-26T20:57:05.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.717+0000 7fee77e00640 1 --2- 172.21.15.28:0/1414365751 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee5c003f60 0x7fee5c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:05.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.718+0000 7fee7ca00640 1 --2- 172.21.15.28:0/1414365751 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee780b6810 0x7fee5c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:05.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.718+0000 7fee7d400640 1 --2- 172.21.15.28:0/1414365751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee5c004d40 0x7fee5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:05.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.718+0000 7fee77e00640 1 -- 172.21.15.28:0/1414365751 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee780b6810 msgr2=0x7fee5c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:05.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.718+0000 7fee77e00640 1 --2- 172.21.15.28:0/1414365751 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee780b6810 0x7fee5c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.718+0000 7fee77e00640 1 -- 172.21.15.28:0/1414365751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee5c004d40 msgr2=0x7fee5c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:05.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.718+0000 7fee77e00640 1 --2- 172.21.15.28:0/1414365751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee5c004d40 0x7fee5c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.718+0000 7fee77e00640 1 -- 172.21.15.28:0/1414365751 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee5c003520 con 0x7fee5c003f60 2024-04-26T20:57:05.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.718+0000 7fee7d400640 1 --2- 172.21.15.28:0/1414365751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee5c004d40 0x7fee5c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:05.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.718+0000 7fee7ca00640 1 --2- 172.21.15.28:0/1414365751 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee780b6810 0x7fee5c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:05.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.719+0000 7fee77e00640 1 --2- 172.21.15.28:0/1414365751 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee5c003f60 0x7fee5c004800 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fee64009ac0 tx=0x7fee5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e807d629157c59 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:05.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.719+0000 7fee77400640 1 -- 172.21.15.28:0/1414365751 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64036070 con 0x7fee5c003f60 2024-04-26T20:57:05.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.719+0000 7fee77400640 1 -- 172.21.15.28:0/1414365751 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee64031400 con 0x7fee5c003f60 2024-04-26T20:57:05.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.719+0000 7fee77400640 1 -- 172.21.15.28:0/1414365751 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee64009690 con 0x7fee5c003f60 2024-04-26T20:57:05.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.720+0000 7fee7f400640 1 -- 172.21.15.28:0/1414365751 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee5c003f60 msgr2=0x7fee5c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:05.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.720+0000 7fee7f400640 1 --2- 172.21.15.28:0/1414365751 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee5c003f60 0x7fee5c004800 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fee64009ac0 tx=0x7fee5c005830 comp rx=0 tx=0).stop 2024-04-26T20:57:05.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.720+0000 7fee7d400640 1 -- 172.21.15.28:0/1414365751 reap_dead start 2024-04-26T20:57:05.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.720+0000 7fee7f400640 1 -- 172.21.15.28:0/1414365751 shutdown_connections 2024-04-26T20:57:05.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.720+0000 7fee7f400640 1 -- 172.21.15.28:0/1414365751 >> 172.21.15.28:0/1414365751 conn(0x7fee7806e220 msgr2=0x7fee7806e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:05.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.720+0000 7fee7f400640 1 -- 172.21.15.28:0/1414365751 shutdown_connections 2024-04-26T20:57:05.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.720+0000 7fee7f400640 1 -- 172.21.15.28:0/1414365751 wait complete. 2024-04-26T20:57:05.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.722+0000 7fee7f400640 1 Processor -- start 2024-04-26T20:57:05.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.722+0000 7fee7f400640 1 -- start start 2024-04-26T20:57:05.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.723+0000 7fee7f400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee5c003f60 0x7fee7814beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:05.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.723+0000 7fee7f400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee7815f800 0x7fee7815f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:05.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.723+0000 7fee7f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee781429b0 0x7fee7813d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:05.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.723+0000 7fee7f400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fee78072700 con 0x7fee781429b0 2024-04-26T20:57:05.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.723+0000 7fee7f400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fee78073bc0 con 0x7fee5c003f60 2024-04-26T20:57:05.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.723+0000 7fee7f400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fee78072880 con 0x7fee7815f800 2024-04-26T20:57:05.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.723+0000 7fee7d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee781429b0 0x7fee7813d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:05.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.723+0000 7fee7d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee781429b0 0x7fee7813d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35310/0 (socket says 172.21.15.28:35310) 2024-04-26T20:57:05.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.723+0000 7fee7d400640 1 -- 172.21.15.28:0/3167032638 learned_addr learned my addr 172.21.15.28:0/3167032638 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:05.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.723+0000 7fee7ca00640 1 --2- 172.21.15.28:0/3167032638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee5c003f60 0x7fee7814beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:05.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.723+0000 7fee77e00640 1 --2- 172.21.15.28:0/3167032638 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee7815f800 0x7fee7815f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:05.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.724+0000 7fee7d400640 1 -- 172.21.15.28:0/3167032638 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee7815f800 msgr2=0x7fee7815f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:05.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.724+0000 7fee7d400640 1 --2- 172.21.15.28:0/3167032638 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee7815f800 0x7fee7815f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.724+0000 7fee7d400640 1 -- 172.21.15.28:0/3167032638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee5c003f60 msgr2=0x7fee7814beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:05.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.724+0000 7fee7d400640 1 --2- 172.21.15.28:0/3167032638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee5c003f60 0x7fee7814beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.724+0000 7fee7d400640 1 -- 172.21.15.28:0/3167032638 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7815fdc0 con 0x7fee781429b0 2024-04-26T20:57:05.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.724+0000 7fee7d400640 1 --2- 172.21.15.28:0/3167032638 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee781429b0 0x7fee7813d470 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fee780704a0 tx=0x7fee7000d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:05.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.724+0000 7fee74c00640 1 -- 172.21.15.28:0/3167032638 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70005480 con 0x7fee781429b0 2024-04-26T20:57:05.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.724+0000 7fee74c00640 1 -- 172.21.15.28:0/3167032638 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee7000de80 con 0x7fee781429b0 2024-04-26T20:57:05.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.724+0000 7fee74c00640 1 -- 172.21.15.28:0/3167032638 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70007250 con 0x7fee781429b0 2024-04-26T20:57:05.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.724+0000 7fee7f400640 1 -- 172.21.15.28:0/3167032638 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee781600a0 con 0x7fee781429b0 2024-04-26T20:57:05.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.724+0000 7fee7f400640 1 -- 172.21.15.28:0/3167032638 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee78160660 con 0x7fee781429b0 2024-04-26T20:57:05.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.725+0000 7fee7f400640 1 -- 172.21.15.28:0/3167032638 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee7806a400 con 0x7fee781429b0 2024-04-26T20:57:05.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.726+0000 7fee74c00640 1 -- 172.21.15.28:0/3167032638 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee7001b020 con 0x7fee781429b0 2024-04-26T20:57:05.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.727+0000 7fee74c00640 1 --2- 172.21.15.28:0/3167032638 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fee5806ca90 0x7fee5806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:05.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.727+0000 7fee74c00640 1 -- 172.21.15.28:0/3167032638 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fee7008b4d0 con 0x7fee781429b0 2024-04-26T20:57:05.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.728+0000 7fee7ca00640 1 --2- 172.21.15.28:0/3167032638 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fee5806ca90 0x7fee5806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:05.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.728+0000 7fee7ca00640 1 --2- 172.21.15.28:0/3167032638 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fee5806ca90 0x7fee5806ef50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fee68007c40 tx=0x7fee68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:05.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.732+0000 7fee74c00640 1 -- 172.21.15.28:0/3167032638 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fee70059860 con 0x7fee781429b0 2024-04-26T20:57:05.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.943+0000 7fee7f400640 1 -- 172.21.15.28:0/3167032638 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee7815b270 con 0x7fee781429b0 2024-04-26T20:57:05.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.945+0000 7fee74c00640 1 -- 172.21.15.28:0/3167032638 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7fee70011e10 con 0x7fee781429b0 2024-04-26T20:57:05.947 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:05.947 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":21,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:56:52.699663+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:57:05.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.948+0000 7fee7f400640 1 -- 172.21.15.28:0/3167032638 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fee5806ca90 msgr2=0x7fee5806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:05.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.948+0000 7fee7f400640 1 --2- 172.21.15.28:0/3167032638 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fee5806ca90 0x7fee5806ef50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fee68007c40 tx=0x7fee68009040 comp rx=0 tx=0).stop 2024-04-26T20:57:05.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.948+0000 7fee7f400640 1 -- 172.21.15.28:0/3167032638 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee781429b0 msgr2=0x7fee7813d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:05.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.948+0000 7fee7f400640 1 --2- 172.21.15.28:0/3167032638 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee781429b0 0x7fee7813d470 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fee780704a0 tx=0x7fee7000d680 comp rx=0 tx=0).stop 2024-04-26T20:57:05.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.949+0000 7fee7f400640 1 -- 172.21.15.28:0/3167032638 shutdown_connections 2024-04-26T20:57:05.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.949+0000 7fee7f400640 1 --2- 172.21.15.28:0/3167032638 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fee5806ca90 0x7fee5806ef50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.949+0000 7fee7f400640 1 --2- 172.21.15.28:0/3167032638 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee781429b0 0x7fee7813d470 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.949+0000 7fee7f400640 1 --2- 172.21.15.28:0/3167032638 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fee7815f800 0x7fee7815f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.949+0000 7fee7f400640 1 --2- 172.21.15.28:0/3167032638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fee5c003f60 0x7fee7814beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:05.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.949+0000 7fee7f400640 1 -- 172.21.15.28:0/3167032638 >> 172.21.15.28:0/3167032638 conn(0x7fee7806e220 msgr2=0x7fee7806e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:05.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.949+0000 7fee7f400640 1 -- 172.21.15.28:0/3167032638 shutdown_connections 2024-04-26T20:57:05.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:05.949+0000 7fee7f400640 1 -- 172.21.15.28:0/3167032638 wait complete. 2024-04-26T20:57:05.969 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:57:06.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.180+0000 7fa264a00640 1 Processor -- start 2024-04-26T20:57:06.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.180+0000 7fa264a00640 1 -- start start 2024-04-26T20:57:06.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.182+0000 7fa264a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa260146060 0x7fa260071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:06.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.182+0000 7fa264a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa260161530 con 0x7fa2601610c0 2024-04-26T20:57:06.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.182+0000 7fa264a00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa260146440 con 0x7fa260146670 2024-04-26T20:57:06.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.182+0000 7fa264a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa26006f510 con 0x7fa260146060 2024-04-26T20:57:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.182+0000 7fa25fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa2601610c0 0x7fa2600cb0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47792/0 (socket says 172.21.15.28:47792) 2024-04-26T20:57:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.182+0000 7fa25fe00640 1 -- 172.21.15.28:0/3881528682 learned_addr learned my addr 172.21.15.28:0/3881528682 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.183+0000 7fa25f400640 1 --2- 172.21.15.28:0/3881528682 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa260146060 0x7fa260071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.183+0000 7fa25e000640 1 -- 172.21.15.28:0/3881528682 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1523743347 0 0) 0x7fa260161530 con 0x7fa2601610c0 2024-04-26T20:57:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.183+0000 7fa25e000640 1 -- 172.21.15.28:0/3881528682 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa244003520 con 0x7fa2601610c0 2024-04-26T20:57:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.183+0000 7fa25e000640 1 -- 172.21.15.28:0/3881528682 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1338972850 0 0) 0x7fa244003520 con 0x7fa2601610c0 2024-04-26T20:57:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.183+0000 7fa25e000640 1 -- 172.21.15.28:0/3881528682 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa260146060 msgr2=0x7fa260071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.183+0000 7fa25e000640 1 --2- 172.21.15.28:0/3881528682 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa260146060 0x7fa260071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.183+0000 7fa25e000640 1 -- 172.21.15.28:0/3881528682 >> v1:172.21.15.103:6789/0 conn(0x7fa260146670 legacy=0x7fa2601609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.183+0000 7fa25e000640 1 -- 172.21.15.28:0/3881528682 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26006f010 con 0x7fa2601610c0 2024-04-26T20:57:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.183+0000 7fa25e000640 1 -- 172.21.15.28:0/3881528682 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fa2580019f0 con 0x7fa2601610c0 2024-04-26T20:57:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.184+0000 7fa25e000640 1 -- 172.21.15.28:0/3881528682 >> v1:172.21.15.28:6789/0 conn(0x7fa2601610c0 legacy=0x7fa2600cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.184+0000 7fa25e000640 1 --2- 172.21.15.28:0/3881528682 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa260146060 0x7fa2440039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.184+0000 7fa25e000640 1 --2- 172.21.15.28:0/3881528682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa244003f60 0x7fa244004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.184+0000 7fa25e000640 1 --2- 172.21.15.28:0/3881528682 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa244004d40 0x7fa2440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.184+0000 7fa25e000640 1 -- 172.21.15.28:0/3881528682 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa26006f510 con 0x7fa244004d40 2024-04-26T20:57:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.184+0000 7fa25e000640 1 -- 172.21.15.28:0/3881528682 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa24400bb50 con 0x7fa244003f60 2024-04-26T20:57:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.184+0000 7fa25e000640 1 -- 172.21.15.28:0/3881528682 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa24400bd50 con 0x7fa260146060 2024-04-26T20:57:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.184+0000 7fa25fe00640 1 --2- 172.21.15.28:0/3881528682 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa244004d40 0x7fa2440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.184+0000 7fa25f400640 1 --2- 172.21.15.28:0/3881528682 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa260146060 0x7fa2440039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.184+0000 7fa25ea00640 1 --2- 172.21.15.28:0/3881528682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa244003f60 0x7fa244004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.184+0000 7fa25fe00640 1 -- 172.21.15.28:0/3881528682 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa260146060 msgr2=0x7fa2440039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.184+0000 7fa25fe00640 1 --2- 172.21.15.28:0/3881528682 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa260146060 0x7fa2440039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.184+0000 7fa25fe00640 1 -- 172.21.15.28:0/3881528682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa244003f60 msgr2=0x7fa244004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.184+0000 7fa25fe00640 1 --2- 172.21.15.28:0/3881528682 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa244003f60 0x7fa244004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.184+0000 7fa25fe00640 1 -- 172.21.15.28:0/3881528682 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa244003520 con 0x7fa244004d40 2024-04-26T20:57:06.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.186+0000 7fa25fe00640 1 --2- 172.21.15.28:0/3881528682 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa244004d40 0x7fa2440051c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa25800a850 tx=0x7fa244005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d095705ec3bfba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:06.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.186+0000 7fa25e000640 1 -- 172.21.15.28:0/3881528682 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25800a4a0 con 0x7fa244004d40 2024-04-26T20:57:06.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.186+0000 7fa25e000640 1 -- 172.21.15.28:0/3881528682 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa25802f070 con 0x7fa244004d40 2024-04-26T20:57:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.186+0000 7fa25e000640 1 -- 172.21.15.28:0/3881528682 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25802bce0 con 0x7fa244004d40 2024-04-26T20:57:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.186+0000 7fa264a00640 1 -- 172.21.15.28:0/3881528682 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa244004d40 msgr2=0x7fa2440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.186+0000 7fa264a00640 1 --2- 172.21.15.28:0/3881528682 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa244004d40 0x7fa2440051c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa25800a850 tx=0x7fa244005830 comp rx=0 tx=0).stop 2024-04-26T20:57:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.186+0000 7fa25fe00640 1 -- 172.21.15.28:0/3881528682 reap_dead start 2024-04-26T20:57:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.187+0000 7fa264a00640 1 -- 172.21.15.28:0/3881528682 shutdown_connections 2024-04-26T20:57:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.187+0000 7fa264a00640 1 -- 172.21.15.28:0/3881528682 >> 172.21.15.28:0/3881528682 conn(0x7fa2600d0a50 msgr2=0x7fa2600d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.187+0000 7fa264a00640 1 -- 172.21.15.28:0/3881528682 shutdown_connections 2024-04-26T20:57:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.187+0000 7fa264a00640 1 -- 172.21.15.28:0/3881528682 wait complete. 2024-04-26T20:57:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.188+0000 7fa264a00640 1 Processor -- start 2024-04-26T20:57:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.189+0000 7fa264a00640 1 -- start start 2024-04-26T20:57:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.190+0000 7fa264a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa244004d40 0x7fa2600aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.190+0000 7fa264a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa26015e2e0 0x7fa2601f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.190+0000 7fa264a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2601f8510 0x7fa2601f8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.190+0000 7fa264a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa26006ff60 con 0x7fa2601f8510 2024-04-26T20:57:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.190+0000 7fa264a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa26006fc80 con 0x7fa244004d40 2024-04-26T20:57:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.190+0000 7fa264a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa260071e20 con 0x7fa26015e2e0 2024-04-26T20:57:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.190+0000 7fa25fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2601f8510 0x7fa2601f8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.190+0000 7fa25fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2601f8510 0x7fa2601f8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35324/0 (socket says 172.21.15.28:35324) 2024-04-26T20:57:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.190+0000 7fa25fe00640 1 -- 172.21.15.28:0/750441139 learned_addr learned my addr 172.21.15.28:0/750441139 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.190+0000 7fa25ea00640 1 --2- 172.21.15.28:0/750441139 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa26015e2e0 0x7fa2601f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.190+0000 7fa25f400640 1 --2- 172.21.15.28:0/750441139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa244004d40 0x7fa2600aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.190+0000 7fa25fe00640 1 -- 172.21.15.28:0/750441139 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa26015e2e0 msgr2=0x7fa2601f7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.190+0000 7fa25fe00640 1 --2- 172.21.15.28:0/750441139 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa26015e2e0 0x7fa2601f7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.190+0000 7fa25fe00640 1 -- 172.21.15.28:0/750441139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa244004d40 msgr2=0x7fa2600aa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.190+0000 7fa25fe00640 1 --2- 172.21.15.28:0/750441139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa244004d40 0x7fa2600aa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.190+0000 7fa25fe00640 1 -- 172.21.15.28:0/750441139 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2600b53e0 con 0x7fa2601f8510 2024-04-26T20:57:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.190+0000 7fa25f400640 1 --2- 172.21.15.28:0/750441139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa244004d40 0x7fa2600aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.191+0000 7fa25fe00640 1 --2- 172.21.15.28:0/750441139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2601f8510 0x7fa2601f8990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa258000f80 tx=0x7fa25802d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.191+0000 7fa257400640 1 -- 172.21.15.28:0/750441139 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25800a480 con 0x7fa2601f8510 2024-04-26T20:57:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.191+0000 7fa257400640 1 -- 172.21.15.28:0/750441139 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa258004b60 con 0x7fa2601f8510 2024-04-26T20:57:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.191+0000 7fa257400640 1 -- 172.21.15.28:0/750441139 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa258034470 con 0x7fa2601f8510 2024-04-26T20:57:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.191+0000 7fa264a00640 1 -- 172.21.15.28:0/750441139 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2600b5680 con 0x7fa2601f8510 2024-04-26T20:57:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.191+0000 7fa264a00640 1 -- 172.21.15.28:0/750441139 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2600b5c40 con 0x7fa2601f8510 2024-04-26T20:57:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.192+0000 7fa264a00640 1 -- 172.21.15.28:0/750441139 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa26006a360 con 0x7fa2601f8510 2024-04-26T20:57:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.193+0000 7fa257400640 1 -- 172.21.15.28:0/750441139 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa258034610 con 0x7fa2601f8510 2024-04-26T20:57:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.194+0000 7fa257400640 1 --2- 172.21.15.28:0/750441139 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa24006c940 0x7fa24006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.194+0000 7fa257400640 1 -- 172.21.15.28:0/750441139 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa2580abc00 con 0x7fa2601f8510 2024-04-26T20:57:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.195+0000 7fa25f400640 1 --2- 172.21.15.28:0/750441139 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa24006c940 0x7fa24006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:06.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.195+0000 7fa25f400640 1 --2- 172.21.15.28:0/750441139 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa24006c940 0x7fa24006ee00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa26006ed50 tx=0x7fa250008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.199+0000 7fa257400640 1 -- 172.21.15.28:0/750441139 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa25807a0a0 con 0x7fa2601f8510 2024-04-26T20:57:06.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.441+0000 7fa264a00640 1 -- 172.21.15.28:0/750441139 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2600760c0 con 0x7fa2601f8510 2024-04-26T20:57:06.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.442+0000 7fa257400640 1 -- 172.21.15.28:0/750441139 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7fa2580799a0 con 0x7fa2601f8510 2024-04-26T20:57:06.444 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:06.444 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/2818824721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2818824721},{"type":"v1","addr":"172.21.15.103:6834","nonce":2818824721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/259712913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":259712913},{"type":"v1","addr":"172.21.15.28:6836","nonce":259712913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/989575415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":989575415},{"type":"v1","addr":"172.21.15.28:6837","nonce":989575415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:56:51.143688+0000","modified":"2024-04-26T20:56:54.228571+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6835/2361950169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":2361950169},{"type":"v1","addr":"172.21.15.103:6835","nonce":2361950169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-26T20:57:06.444 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 8 2024-04-26T20:57:06.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.445+0000 7fa264a00640 1 -- 172.21.15.28:0/750441139 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa24006c940 msgr2=0x7fa24006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:06.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.445+0000 7fa264a00640 1 --2- 172.21.15.28:0/750441139 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa24006c940 0x7fa24006ee00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa26006ed50 tx=0x7fa250008040 comp rx=0 tx=0).stop 2024-04-26T20:57:06.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.445+0000 7fa264a00640 1 -- 172.21.15.28:0/750441139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2601f8510 msgr2=0x7fa2601f8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:06.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.445+0000 7fa264a00640 1 --2- 172.21.15.28:0/750441139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2601f8510 0x7fa2601f8990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa258000f80 tx=0x7fa25802d070 comp rx=0 tx=0).stop 2024-04-26T20:57:06.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.445+0000 7fa264a00640 1 -- 172.21.15.28:0/750441139 shutdown_connections 2024-04-26T20:57:06.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.445+0000 7fa264a00640 1 --2- 172.21.15.28:0/750441139 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa24006c940 0x7fa24006ee00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:06.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.445+0000 7fa264a00640 1 --2- 172.21.15.28:0/750441139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2601f8510 0x7fa2601f8990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:06.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.445+0000 7fa264a00640 1 --2- 172.21.15.28:0/750441139 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa26015e2e0 0x7fa2601f7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:06.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.445+0000 7fa264a00640 1 --2- 172.21.15.28:0/750441139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa244004d40 0x7fa2600aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:06.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.445+0000 7fa264a00640 1 -- 172.21.15.28:0/750441139 >> 172.21.15.28:0/750441139 conn(0x7fa2600d0a50 msgr2=0x7fa2600cb010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:06.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.445+0000 7fa264a00640 1 -- 172.21.15.28:0/750441139 shutdown_connections 2024-04-26T20:57:06.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:06.445+0000 7fa264a00640 1 -- 172.21.15.28:0/750441139 wait complete. 2024-04-26T20:57:06.866 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:56:51.143688+0000', 'modified': '2024-04-26T20:56:54.228571+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4272}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4272': {'gid': 4272, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.103:6835/2361950169', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6833', 'nonce': 2361950169}, {'type': 'v1', 'addr': '172.21.15.103:6835', 'nonce': 2361950169}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4272, 'qdb_cluster': [4272]} 2024-04-26T20:57:06.867 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T20:57:06.867 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-26T20:57:07.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.087+0000 7fa701000640 1 Processor -- start 2024-04-26T20:57:07.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.088+0000 7fa701000640 1 -- start start 2024-04-26T20:57:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.089+0000 7fa701000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa6fc15f350 0x7fa6fc0d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.089+0000 7fa701000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa6fc0cf8a0 0x7fa6fc0d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.089+0000 7fa701000640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6fc0d0240 con 0x7fa6fc1638c0 2024-04-26T20:57:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.089+0000 7fa701000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa6fc0cffe0 con 0x7fa6fc0cf8a0 2024-04-26T20:57:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.089+0000 7fa701000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa6fc0d0520 con 0x7fa6fc15f350 2024-04-26T20:57:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.090+0000 7fa6f9400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa6fc0cf8a0 0x7fa6fc0d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.090+0000 7fa6f9400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa6fc0cf8a0 0x7fa6fc0d1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35336/0 (socket says 172.21.15.28:35336) 2024-04-26T20:57:07.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.090+0000 7fa6f9400640 1 -- 172.21.15.28:0/32151051 learned_addr learned my addr 172.21.15.28:0/32151051 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:07.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.090+0000 7fa6f9e00640 1 --2- 172.21.15.28:0/32151051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa6fc15f350 0x7fa6fc0d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.090+0000 7fa6f9400640 1 -- 172.21.15.28:0/32151051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa6fc15f350 msgr2=0x7fa6fc0d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:07.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.090+0000 7fa6f9400640 1 --2- 172.21.15.28:0/32151051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa6fc15f350 0x7fa6fc0d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.090+0000 7fa6f9400640 1 -- 172.21.15.28:0/32151051 >> v1:172.21.15.103:6789/0 conn(0x7fa6fc1638c0 legacy=0x7fa6fc06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:07.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.090+0000 7fa6f9400640 1 -- 172.21.15.28:0/32151051 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6fc15f730 con 0x7fa6fc0cf8a0 2024-04-26T20:57:07.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.090+0000 7fa6f9e00640 1 --2- 172.21.15.28:0/32151051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa6fc15f350 0x7fa6fc0d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:07.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.091+0000 7fa6f9400640 1 --2- 172.21.15.28:0/32151051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6fc0cf8a0 0x7fa6fc0d1d90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009870 tx=0x7fa6e802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c8a8ac30515e56 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:07.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.092+0000 7fa6f8a00640 1 -- 172.21.15.28:0/32151051 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8037070 con 0x7fa6fc0cf8a0 2024-04-26T20:57:07.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.092+0000 7fa6f8a00640 1 -- 172.21.15.28:0/32151051 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e802ae10 con 0x7fa6fc0cf8a0 2024-04-26T20:57:07.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.092+0000 7fa6f8a00640 1 -- 172.21.15.28:0/32151051 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8032660 con 0x7fa6fc0cf8a0 2024-04-26T20:57:07.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.092+0000 7fa701000640 1 -- 172.21.15.28:0/32151051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6fc0cf8a0 msgr2=0x7fa6fc0d1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:07.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.092+0000 7fa701000640 1 --2- 172.21.15.28:0/32151051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6fc0cf8a0 0x7fa6fc0d1d90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009870 tx=0x7fa6e802a100 comp rx=0 tx=0).stop 2024-04-26T20:57:07.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.092+0000 7fa701000640 1 -- 172.21.15.28:0/32151051 shutdown_connections 2024-04-26T20:57:07.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.092+0000 7fa701000640 1 --2- 172.21.15.28:0/32151051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6fc0cf8a0 0x7fa6fc0d1d90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.092+0000 7fa701000640 1 --2- 172.21.15.28:0/32151051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa6fc15f350 0x7fa6fc0d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.092+0000 7fa701000640 1 -- 172.21.15.28:0/32151051 >> 172.21.15.28:0/32151051 conn(0x7fa6fc0b6d10 msgr2=0x7fa6fc0b7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:07.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.092+0000 7fa701000640 1 -- 172.21.15.28:0/32151051 shutdown_connections 2024-04-26T20:57:07.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.093+0000 7fa701000640 1 -- 172.21.15.28:0/32151051 wait complete. 2024-04-26T20:57:07.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.094+0000 7fa701000640 1 Processor -- start 2024-04-26T20:57:07.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.094+0000 7fa701000640 1 -- start start 2024-04-26T20:57:07.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.095+0000 7fa701000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6fc0cf8a0 0x7fa6fc0c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.095+0000 7fa701000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6fc15f350 0x7fa6fc0d59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.095+0000 7fa701000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa6fc1638c0 0x7fa6fc150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.095+0000 7fa701000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc0d4aa0 con 0x7fa6fc15f350 2024-04-26T20:57:07.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.095+0000 7fa701000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc0d0800 con 0x7fa6fc0cf8a0 2024-04-26T20:57:07.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.095+0000 7fa701000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa6fc0d4c20 con 0x7fa6fc1638c0 2024-04-26T20:57:07.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.095+0000 7fa6f9400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6fc15f350 0x7fa6fc0d59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.096+0000 7fa6f9400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6fc15f350 0x7fa6fc0d59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35350/0 (socket says 172.21.15.28:35350) 2024-04-26T20:57:07.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.096+0000 7fa6f9400640 1 -- 172.21.15.28:0/1537881800 learned_addr learned my addr 172.21.15.28:0/1537881800 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.096+0000 7fa6f9e00640 1 --2- 172.21.15.28:0/1537881800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6fc0cf8a0 0x7fa6fc0c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.096+0000 7fa6fa800640 1 --2- 172.21.15.28:0/1537881800 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa6fc1638c0 0x7fa6fc150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.096+0000 7fa6f9400640 1 -- 172.21.15.28:0/1537881800 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa6fc1638c0 msgr2=0x7fa6fc150490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.096+0000 7fa6f9400640 1 --2- 172.21.15.28:0/1537881800 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa6fc1638c0 0x7fa6fc150490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.096+0000 7fa6f9400640 1 -- 172.21.15.28:0/1537881800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6fc0cf8a0 msgr2=0x7fa6fc0c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.096+0000 7fa6f9400640 1 --2- 172.21.15.28:0/1537881800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6fc0cf8a0 0x7fa6fc0c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.096+0000 7fa6f9400640 1 -- 172.21.15.28:0/1537881800 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6fc1509d0 con 0x7fa6fc15f350 2024-04-26T20:57:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.096+0000 7fa6f9e00640 1 --2- 172.21.15.28:0/1537881800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6fc0cf8a0 0x7fa6fc0c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.096+0000 7fa6fa800640 1 --2- 172.21.15.28:0/1537881800 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa6fc1638c0 0x7fa6fc150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.096+0000 7fa6f9400640 1 --2- 172.21.15.28:0/1537881800 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6fc15f350 0x7fa6fc0d59e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009870 tx=0x7fa6e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.097+0000 7fa6f2000640 1 -- 172.21.15.28:0/1537881800 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8041070 con 0x7fa6fc15f350 2024-04-26T20:57:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.097+0000 7fa6f2000640 1 -- 172.21.15.28:0/1537881800 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e803c680 con 0x7fa6fc15f350 2024-04-26T20:57:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.097+0000 7fa6f2000640 1 -- 172.21.15.28:0/1537881800 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8037040 con 0x7fa6fc15f350 2024-04-26T20:57:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.097+0000 7fa701000640 1 -- 172.21.15.28:0/1537881800 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6fc0ce880 con 0x7fa6fc15f350 2024-04-26T20:57:07.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.097+0000 7fa701000640 1 -- 172.21.15.28:0/1537881800 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6fc0c9260 con 0x7fa6fc15f350 2024-04-26T20:57:07.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.099+0000 7fa6f2000640 1 -- 172.21.15.28:0/1537881800 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa6e8044070 con 0x7fa6fc15f350 2024-04-26T20:57:07.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.100+0000 7fa6f2000640 1 --2- 172.21.15.28:0/1537881800 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa6dc06c8f0 0x7fa6dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.100+0000 7fa6f2000640 1 -- 172.21.15.28:0/1537881800 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa6e80ac440 con 0x7fa6fc15f350 2024-04-26T20:57:07.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.100+0000 7fa6f9e00640 1 --2- 172.21.15.28:0/1537881800 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa6dc06c8f0 0x7fa6dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.101+0000 7fa703e09740 1 Processor -- start 2024-04-26T20:57:07.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.101+0000 7fa703e09740 1 -- start start 2024-04-26T20:57:07.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.101+0000 7fa6f9e00640 1 --2- 172.21.15.28:0/1537881800 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa6dc06c8f0 0x7fa6dc06edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00d430 tx=0x7fa6ec007530 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:07.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.102+0000 7fa703e09740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 0x55e3105e8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.102+0000 7fa703e09740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e310622cb0 0x55e3105e69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.102+0000 7fa703e09740 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e3106110a0 0x55e310611520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.102+0000 7fa703e09740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e31054a600 con 0x55e3105e65e0 2024-04-26T20:57:07.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.102+0000 7fa703e09740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e3104e7100 con 0x55e310622cb0 2024-04-26T20:57:07.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.102+0000 7fa703e09740 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x55e3105465c0 con 0x55e3106110a0 2024-04-26T20:57:07.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.102+0000 7fa6f9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 0x55e3105e8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.102+0000 7fa6f9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 0x55e3105e8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35354/0 (socket says 172.21.15.28:35354) 2024-04-26T20:57:07.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.102+0000 7fa6f9e00640 1 -- 172.21.15.28:0/2802278828 learned_addr learned my addr 172.21.15.28:0/2802278828 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:07.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.102+0000 7fa6f9e00640 1 -- 172.21.15.28:0/2802278828 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e3106110a0 msgr2=0x55e310611520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:07.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.102+0000 7fa6f9e00640 1 --2- 172.21.15.28:0/2802278828 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e3106110a0 0x55e310611520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.102+0000 7fa6f9e00640 1 -- 172.21.15.28:0/2802278828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e310622cb0 msgr2=0x55e3105e69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:07.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.102+0000 7fa6f9e00640 1 --2- 172.21.15.28:0/2802278828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e310622cb0 0x55e3105e69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.102+0000 7fa6f9e00640 1 -- 172.21.15.28:0/2802278828 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e31025c2d0 con 0x55e3105e65e0 2024-04-26T20:57:07.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.103+0000 7fa6f9e00640 1 --2- 172.21.15.28:0/2802278828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 0x55e3105e8740 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00b6f0 tx=0x7fa6ec010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7799c78f4393e0e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:07.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.103+0000 7fa6e6a00640 1 -- 172.21.15.28:0/2802278828 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec018850 con 0x55e3105e65e0 2024-04-26T20:57:07.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.103+0000 7fa6e6a00640 1 -- 172.21.15.28:0/2802278828 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6ec0189f0 con 0x55e3105e65e0 2024-04-26T20:57:07.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.103+0000 7fa6e6a00640 1 -- 172.21.15.28:0/2802278828 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec018da0 con 0x55e3105e65e0 2024-04-26T20:57:07.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.103+0000 7fa703e09740 1 -- 172.21.15.28:0/2802278828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 msgr2=0x55e3105e8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:07.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.103+0000 7fa703e09740 1 --2- 172.21.15.28:0/2802278828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 0x55e3105e8740 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00b6f0 tx=0x7fa6ec010000 comp rx=0 tx=0).stop 2024-04-26T20:57:07.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.104+0000 7fa703e09740 1 -- 172.21.15.28:0/2802278828 shutdown_connections 2024-04-26T20:57:07.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.104+0000 7fa703e09740 1 --2- 172.21.15.28:0/2802278828 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e3106110a0 0x55e310611520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.104+0000 7fa703e09740 1 --2- 172.21.15.28:0/2802278828 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e310622cb0 0x55e3105e69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.104+0000 7fa703e09740 1 --2- 172.21.15.28:0/2802278828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 0x55e3105e8740 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.104+0000 7fa703e09740 1 -- 172.21.15.28:0/2802278828 >> 172.21.15.28:0/2802278828 conn(0x55e31057ad10 msgr2=0x55e31062c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:07.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.104+0000 7fa703e09740 1 -- 172.21.15.28:0/2802278828 shutdown_connections 2024-04-26T20:57:07.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.104+0000 7fa703e09740 1 -- 172.21.15.28:0/2802278828 wait complete. 2024-04-26T20:57:07.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.104+0000 7fa703e09740 1 -- start start 2024-04-26T20:57:07.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.105+0000 7fa703e09740 1 Processor -- start 2024-04-26T20:57:07.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.106+0000 7fa703e09740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 0x55e310627030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.106+0000 7fa703e09740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3106110a0 0x55e310624b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.106+0000 7fa703e09740 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e310622cb0 0x55e310625090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.106+0000 7fa703e09740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e3104d1f70 con 0x55e3105e65e0 2024-04-26T20:57:07.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.106+0000 7fa703e09740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e31060ebf0 con 0x55e3106110a0 2024-04-26T20:57:07.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.106+0000 7fa703e09740 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x55e3106014b0 con 0x55e310622cb0 2024-04-26T20:57:07.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.106+0000 7fa703e09740 10 client.0 _ll_register_callbacks cb 0x55e310540480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:57:07.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.106+0000 7fa6f9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 0x55e310627030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.106+0000 7fa6f9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 0x55e310627030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35356/0 (socket says 172.21.15.28:35356) 2024-04-26T20:57:07.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.106+0000 7fa6f9e00640 1 -- 172.21.15.28:0/3101920157 learned_addr learned my addr 172.21.15.28:0/3101920157 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:07.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.106+0000 7fa6f9e00640 1 -- 172.21.15.28:0/3101920157 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e310622cb0 msgr2=0x55e310625090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:07.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.106+0000 7fa6f9e00640 1 --2- 172.21.15.28:0/3101920157 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e310622cb0 0x55e310625090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.106+0000 7fa6f9e00640 1 -- 172.21.15.28:0/3101920157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3106110a0 msgr2=0x55e310624b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:07.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.106+0000 7fa6f9e00640 1 --2- 172.21.15.28:0/3101920157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3106110a0 0x55e310624b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.106+0000 7fa6f9e00640 1 -- 172.21.15.28:0/3101920157 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3103266b0 con 0x55e3105e65e0 2024-04-26T20:57:07.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.107+0000 7fa6f9e00640 1 --2- 172.21.15.28:0/3101920157 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 0x55e310627030 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00dfd0 tx=0x7fa6ec00f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b249550d1266ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:07.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.107+0000 7fa6db400640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-04-26T20:57:07.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.107+0000 7fa6db400640 1 -- 172.21.15.28:0/3101920157 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec022680 con 0x55e3105e65e0 2024-04-26T20:57:07.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.107+0000 7fa6db400640 1 -- 172.21.15.28:0/3101920157 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa6ec022820 con 0x55e3105e65e0 2024-04-26T20:57:07.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.107+0000 7fa703e09740 1 -- 172.21.15.28:0/3101920157 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6ec0189f0 con 0x55e3105e65e0 2024-04-26T20:57:07.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.107+0000 7fa6db400640 1 -- 172.21.15.28:0/3101920157 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fa6ec022aa0 con 0x55e3105e65e0 2024-04-26T20:57:07.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.107+0000 7fa703e09740 10 client.4620 fetch_fsmap learned FSMap version 8 2024-04-26T20:57:07.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.107+0000 7fa703e09740 1 -- 172.21.15.28:0/3101920157 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55e3106011f0 con 0x55e3105e65e0 2024-04-26T20:57:07.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.107+0000 7fa6db400640 1 -- 172.21.15.28:0/3101920157 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6ec0189f0 con 0x55e3105e65e0 2024-04-26T20:57:07.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.107+0000 7fa6db400640 1 -- 172.21.15.28:0/3101920157 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fa6ec022c40 con 0x55e3105e65e0 2024-04-26T20:57:07.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.107+0000 7fa703e09740 10 client.4620 fetch_fsmap finished waiting for FSMap version 8 2024-04-26T20:57:07.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.107+0000 7fa703e09740 10 client.4620 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T20:57:07.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.108+0000 7fa703e09740 20 client.4620 populate_metadata read hostname 'smithi028' 2024-04-26T20:57:07.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.108+0000 7fa703e09740 1 --2- 172.21.15.28:0/3101920157 >> [v2:172.21.15.103:6833/2361950169,v1:172.21.15.103:6835/2361950169] conn(0x55e3106116a0 0x55e310600370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.108+0000 7fa703e09740 4 client.4620 mds_command: new command op to 4272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:57:07.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.108+0000 7fa703e09740 1 -- 172.21.15.28:0/3101920157 --> [v2:172.21.15.103:6833/2361950169,v1:172.21.15.103:6835/2361950169] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e310558210 con 0x55e3106116a0 2024-04-26T20:57:07.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.108+0000 7fa6f9400640 1 --2- 172.21.15.28:0/3101920157 >> [v2:172.21.15.103:6833/2361950169,v1:172.21.15.103:6835/2361950169] conn(0x55e3106116a0 0x55e310600370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.109+0000 7fa6f9400640 1 --2- 172.21.15.28:0/3101920157 >> [v2:172.21.15.103:6833/2361950169,v1:172.21.15.103:6835/2361950169] conn(0x55e3106116a0 0x55e310600370 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1021afbde6ac1e2 server_cookie=452cf8ef8263d149 in_seq=0 out_seq=0 2024-04-26T20:57:07.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.109+0000 7fa6db400640 10 client.4620 ms_handle_connect on v2:172.21.15.103:6833/2361950169 2024-04-26T20:57:07.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.112+0000 7fa6db400640 1 -- 172.21.15.28:0/3101920157 <== mds.0 v2:172.21.15.103:6833/2361950169 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e310558210 con 0x55e3106116a0 2024-04-26T20:57:07.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.112+0000 7fa6db400640 10 client.4620 handle_command_reply: tid=0 2024-04-26T20:57:07.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.112+0000 7fa703e09740 1 client.4620 shutdown 2024-04-26T20:57:07.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.112+0000 7fa703e09740 1 -- 172.21.15.28:0/3101920157 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 msgr2=0x55e310627030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:07.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.112+0000 7fa703e09740 1 --2- 172.21.15.28:0/3101920157 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 0x55e310627030 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00dfd0 tx=0x7fa6ec00f040 comp rx=0 tx=0).stop 2024-04-26T20:57:07.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.112+0000 7fa703e09740 1 -- 172.21.15.28:0/3101920157 shutdown_connections 2024-04-26T20:57:07.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.112+0000 7fa703e09740 1 --2- 172.21.15.28:0/3101920157 >> [v2:172.21.15.103:6833/2361950169,v1:172.21.15.103:6835/2361950169] conn(0x55e3106116a0 0x55e310600370 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.112+0000 7fa703e09740 1 --2- 172.21.15.28:0/3101920157 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e310622cb0 0x55e310625090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.112+0000 7fa703e09740 1 --2- 172.21.15.28:0/3101920157 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3106110a0 0x55e310624b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.112+0000 7fa703e09740 1 --2- 172.21.15.28:0/3101920157 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 0x55e310627030 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.112+0000 7fa703e09740 1 -- 172.21.15.28:0/3101920157 >> 172.21.15.28:0/3101920157 conn(0x55e31057ad10 msgr2=0x55e31060e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:07.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.112+0000 7fa6db400640 0 client.4620 ms_handle_reset on v2:172.21.15.103:6833/2361950169 2024-04-26T20:57:07.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.112+0000 7fa703e09740 1 -- 172.21.15.28:0/3101920157 shutdown_connections 2024-04-26T20:57:07.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.112+0000 7fa703e09740 1 -- 172.21.15.28:0/3101920157 wait complete. 2024-04-26T20:57:07.115 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.112+0000 7fa703e09740 20 client.4620 trim_cache size 0 max 16384 2024-04-26T20:57:07.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.132+0000 7fa703e09740 1 Processor -- start 2024-04-26T20:57:07.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.132+0000 7fa703e09740 1 -- start start 2024-04-26T20:57:07.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.133+0000 7fa703e09740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 0x55e310612970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.133+0000 7fa703e09740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3106110a0 0x55e3105ff680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.133+0000 7fa703e09740 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e310622cb0 0x55e3105ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.133+0000 7fa703e09740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc15ced0 con 0x55e3105e65e0 2024-04-26T20:57:07.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.133+0000 7fa703e09740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc0cb200 con 0x55e3106110a0 2024-04-26T20:57:07.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.133+0000 7fa703e09740 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa6fc15c860 con 0x55e310622cb0 2024-04-26T20:57:07.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.133+0000 7fa6f9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 0x55e310612970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.134+0000 7fa6f9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 0x55e310612970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35366/0 (socket says 172.21.15.28:35366) 2024-04-26T20:57:07.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.134+0000 7fa6f9e00640 1 -- 172.21.15.28:0/257076369 learned_addr learned my addr 172.21.15.28:0/257076369 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:07.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.134+0000 7fa6f9400640 1 --2- 172.21.15.28:0/257076369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3106110a0 0x55e3105ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.134+0000 7fa6fa800640 1 --2- 172.21.15.28:0/257076369 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e310622cb0 0x55e3105ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.134+0000 7fa6f9e00640 1 -- 172.21.15.28:0/257076369 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e310622cb0 msgr2=0x55e3105ffbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:07.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.134+0000 7fa6f9e00640 1 --2- 172.21.15.28:0/257076369 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e310622cb0 0x55e3105ffbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.134+0000 7fa6f9e00640 1 -- 172.21.15.28:0/257076369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3106110a0 msgr2=0x55e3105ff680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:07.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.134+0000 7fa6f9e00640 1 --2- 172.21.15.28:0/257076369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3106110a0 0x55e3105ff680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.134+0000 7fa6f9e00640 1 -- 172.21.15.28:0/257076369 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e310612530 con 0x55e3105e65e0 2024-04-26T20:57:07.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.134+0000 7fa6f9400640 1 --2- 172.21.15.28:0/257076369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3106110a0 0x55e3105ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:57:07.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.134+0000 7fa6f9e00640 1 --2- 172.21.15.28:0/257076369 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 0x55e310612970 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00bfd0 tx=0x7fa6ec01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61369a520ac23daa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:07.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.135+0000 7fa6e6000640 1 -- 172.21.15.28:0/257076369 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec018960 con 0x55e3105e65e0 2024-04-26T20:57:07.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.135+0000 7fa6e6000640 1 -- 172.21.15.28:0/257076369 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6ec042070 con 0x55e3105e65e0 2024-04-26T20:57:07.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.135+0000 7fa6e6000640 1 -- 172.21.15.28:0/257076369 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec022680 con 0x55e3105e65e0 2024-04-26T20:57:07.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.135+0000 7fa703e09740 1 -- 172.21.15.28:0/257076369 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 msgr2=0x55e310612970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:07.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.135+0000 7fa703e09740 1 --2- 172.21.15.28:0/257076369 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 0x55e310612970 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00bfd0 tx=0x7fa6ec01a040 comp rx=0 tx=0).stop 2024-04-26T20:57:07.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.135+0000 7fa703e09740 1 -- 172.21.15.28:0/257076369 shutdown_connections 2024-04-26T20:57:07.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.135+0000 7fa703e09740 1 --2- 172.21.15.28:0/257076369 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e310622cb0 0x55e3105ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.135+0000 7fa703e09740 1 --2- 172.21.15.28:0/257076369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3106110a0 0x55e3105ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.135+0000 7fa703e09740 1 --2- 172.21.15.28:0/257076369 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e3105e65e0 0x55e310612970 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.135+0000 7fa703e09740 1 -- 172.21.15.28:0/257076369 >> 172.21.15.28:0/257076369 conn(0x55e31057ad10 msgr2=0x55e3105dcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:07.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.135+0000 7fa703e09740 1 -- 172.21.15.28:0/257076369 shutdown_connections 2024-04-26T20:57:07.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.135+0000 7fa703e09740 1 -- 172.21.15.28:0/257076369 wait complete. 2024-04-26T20:57:07.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.136+0000 7fa703e09740 1 -- start start 2024-04-26T20:57:07.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.136+0000 7fa703e09740 1 Processor -- start 2024-04-26T20:57:07.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.137+0000 7fa703e09740 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e3105e65e0 0x55e3106a5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.137+0000 7fa703e09740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3106110a0 0x55e31060ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.137+0000 7fa703e09740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e310622cb0 0x55e3105e4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.137+0000 7fa703e09740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x55e310611520 con 0x55e310622cb0 2024-04-26T20:57:07.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.137+0000 7fa703e09740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e310602e10 con 0x55e3106110a0 2024-04-26T20:57:07.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.137+0000 7fa703e09740 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x55e310611bb0 con 0x55e3105e65e0 2024-04-26T20:57:07.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.137+0000 7fa703e09740 10 client.0 _ll_register_callbacks cb 0x7fa6ec00b5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:57:07.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.138+0000 7fa6fa800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e310622cb0 0x55e3105e4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.138+0000 7fa6fa800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e310622cb0 0x55e3105e4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35368/0 (socket says 172.21.15.28:35368) 2024-04-26T20:57:07.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.138+0000 7fa6fa800640 1 -- 172.21.15.28:0/2347934760 learned_addr learned my addr 172.21.15.28:0/2347934760 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:07.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.138+0000 7fa6f9e00640 1 --2- 172.21.15.28:0/2347934760 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e3105e65e0 0x55e3106a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.138+0000 7fa6f9400640 1 --2- 172.21.15.28:0/2347934760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3106110a0 0x55e31060ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.138+0000 7fa6fa800640 1 -- 172.21.15.28:0/2347934760 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e3105e65e0 msgr2=0x55e3106a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:07.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.138+0000 7fa6fa800640 1 --2- 172.21.15.28:0/2347934760 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e3105e65e0 0x55e3106a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.138+0000 7fa6fa800640 1 -- 172.21.15.28:0/2347934760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3106110a0 msgr2=0x55e31060ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:07.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.138+0000 7fa6fa800640 1 --2- 172.21.15.28:0/2347934760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3106110a0 0x55e31060ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.138+0000 7fa6fa800640 1 -- 172.21.15.28:0/2347934760 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e310625680 con 0x55e310622cb0 2024-04-26T20:57:07.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.138+0000 7fa6fa800640 1 --2- 172.21.15.28:0/2347934760 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e310622cb0 0x55e3105e4c60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400cef0 tx=0x7fa6f400e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e29b51e2fbe6dea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:07.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.138+0000 7fa6db400640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-04-26T20:57:07.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.139+0000 7fa6db400640 1 -- 172.21.15.28:0/2347934760 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4018020 con 0x55e310622cb0 2024-04-26T20:57:07.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.139+0000 7fa6db400640 1 -- 172.21.15.28:0/2347934760 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa6f4013450 con 0x55e310622cb0 2024-04-26T20:57:07.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.139+0000 7fa703e09740 1 -- 172.21.15.28:0/2347934760 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6ec042070 con 0x55e310622cb0 2024-04-26T20:57:07.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.139+0000 7fa6db400640 1 -- 172.21.15.28:0/2347934760 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fa6f400eea0 con 0x55e310622cb0 2024-04-26T20:57:07.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.139+0000 7fa703e09740 10 client.4626 fetch_fsmap learned FSMap version 8 2024-04-26T20:57:07.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.139+0000 7fa703e09740 1 -- 172.21.15.28:0/2347934760 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55e31060e760 con 0x55e310622cb0 2024-04-26T20:57:07.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.139+0000 7fa6db400640 1 -- 172.21.15.28:0/2347934760 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6ec042070 con 0x55e310622cb0 2024-04-26T20:57:07.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.139+0000 7fa6db400640 1 -- 172.21.15.28:0/2347934760 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 8) ==== 3494+0+0 (secure 0 0 0) 0x7fa6f4013650 con 0x55e310622cb0 2024-04-26T20:57:07.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.139+0000 7fa703e09740 10 client.4626 fetch_fsmap finished waiting for FSMap version 8 2024-04-26T20:57:07.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.139+0000 7fa703e09740 10 client.4626 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T20:57:07.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.139+0000 7fa703e09740 20 client.4626 populate_metadata read hostname 'smithi028' 2024-04-26T20:57:07.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.139+0000 7fa703e09740 1 --2- 172.21.15.28:0/2347934760 >> [v2:172.21.15.103:6833/2361950169,v1:172.21.15.103:6835/2361950169] conn(0x55e31064f7e0 0x55e31060f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.139+0000 7fa703e09740 4 client.4626 mds_command: new command op to 4272 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:57:07.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.139+0000 7fa703e09740 1 -- 172.21.15.28:0/2347934760 --> [v2:172.21.15.103:6833/2361950169,v1:172.21.15.103:6835/2361950169] -- command(tid 0: {"prefix": "status"}) -- 0x55e310552080 con 0x55e31064f7e0 2024-04-26T20:57:07.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.140+0000 7fa6f9e00640 1 --2- 172.21.15.28:0/2347934760 >> [v2:172.21.15.103:6833/2361950169,v1:172.21.15.103:6835/2361950169] conn(0x55e31064f7e0 0x55e31060f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.141+0000 7fa6f9e00640 1 --2- 172.21.15.28:0/2347934760 >> [v2:172.21.15.103:6833/2361950169,v1:172.21.15.103:6835/2361950169] conn(0x55e31064f7e0 0x55e31060f8e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6040aa56ea1305b server_cookie=1348c51fcfe8b0af in_seq=0 out_seq=0 2024-04-26T20:57:07.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.141+0000 7fa6db400640 10 client.4626 ms_handle_connect on v2:172.21.15.103:6833/2361950169 2024-04-26T20:57:07.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.141+0000 7fa6db400640 1 -- 172.21.15.28:0/2347934760 <== mds.0 v2:172.21.15.103:6833/2361950169 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55e310552080 con 0x55e31064f7e0 2024-04-26T20:57:07.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.141+0000 7fa6db400640 10 client.4626 handle_command_reply: tid=0 2024-04-26T20:57:07.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.141+0000 7fa703e09740 1 client.4626 shutdown 2024-04-26T20:57:07.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.142+0000 7fa703e09740 1 -- 172.21.15.28:0/2347934760 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e310622cb0 msgr2=0x55e3105e4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:07.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.142+0000 7fa703e09740 1 --2- 172.21.15.28:0/2347934760 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e310622cb0 0x55e3105e4c60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400cef0 tx=0x7fa6f400e5a0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.142+0000 7fa703e09740 1 -- 172.21.15.28:0/2347934760 shutdown_connections 2024-04-26T20:57:07.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.142+0000 7fa703e09740 1 --2- 172.21.15.28:0/2347934760 >> [v2:172.21.15.103:6833/2361950169,v1:172.21.15.103:6835/2361950169] conn(0x55e31064f7e0 0x55e31060f8e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.142+0000 7fa703e09740 1 --2- 172.21.15.28:0/2347934760 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e310622cb0 0x55e3105e4c60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.142+0000 7fa703e09740 1 --2- 172.21.15.28:0/2347934760 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e3106110a0 0x55e31060ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.142+0000 7fa703e09740 1 --2- 172.21.15.28:0/2347934760 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x55e3105e65e0 0x55e3106a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.142+0000 7fa6db400640 0 client.4626 ms_handle_reset on v2:172.21.15.103:6833/2361950169 2024-04-26T20:57:07.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.142+0000 7fa703e09740 1 -- 172.21.15.28:0/2347934760 >> 172.21.15.28:0/2347934760 conn(0x55e31057ad10 msgr2=0x55e310612970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:07.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.142+0000 7fa703e09740 1 -- 172.21.15.28:0/2347934760 shutdown_connections 2024-04-26T20:57:07.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.142+0000 7fa703e09740 1 -- 172.21.15.28:0/2347934760 wait complete. 2024-04-26T20:57:07.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.142+0000 7fa703e09740 20 client.4626 trim_cache size 0 max 16384 2024-04-26T20:57:07.147 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T20:57:07.147 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "79845bd7-2c94-4fa6-8082-292285d530c9", 2024-04-26T20:57:07.147 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-04-26T20:57:07.147 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 4272, 2024-04-26T20:57:07.147 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-04-26T20:57:07.147 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-04-26T20:57:07.147 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-04-26T20:57:07.147 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 15.440276838999999, 2024-04-26T20:57:07.147 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 8, 2024-04-26T20:57:07.148 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 21, 2024-04-26T20:57:07.148 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 21, 2024-04-26T20:57:07.148 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 23.532235907 2024-04-26T20:57:07.148 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T20:57:07.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.143+0000 7fa701000640 1 -- 172.21.15.28:0/1537881800 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa6dc06c8f0 msgr2=0x7fa6dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:07.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.143+0000 7fa701000640 1 --2- 172.21.15.28:0/1537881800 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa6dc06c8f0 0x7fa6dc06edb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00d430 tx=0x7fa6ec007530 comp rx=0 tx=0).stop 2024-04-26T20:57:07.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.143+0000 7fa701000640 1 -- 172.21.15.28:0/1537881800 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6fc15f350 msgr2=0x7fa6fc0d59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:07.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.143+0000 7fa701000640 1 --2- 172.21.15.28:0/1537881800 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6fc15f350 0x7fa6fc0d59e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009870 tx=0x7fa6e80046c0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.143+0000 7fa701000640 1 -- 172.21.15.28:0/1537881800 shutdown_connections 2024-04-26T20:57:07.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.143+0000 7fa701000640 1 --2- 172.21.15.28:0/1537881800 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa6dc06c8f0 0x7fa6dc06edb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.143+0000 7fa701000640 1 --2- 172.21.15.28:0/1537881800 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa6fc1638c0 0x7fa6fc150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.143+0000 7fa701000640 1 --2- 172.21.15.28:0/1537881800 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6fc15f350 0x7fa6fc0d59e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.143+0000 7fa701000640 1 --2- 172.21.15.28:0/1537881800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6fc0cf8a0 0x7fa6fc0c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.143+0000 7fa701000640 1 -- 172.21.15.28:0/1537881800 >> 172.21.15.28:0/1537881800 conn(0x7fa6fc0b6d10 msgr2=0x7fa6fc0b9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:07.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.143+0000 7fa701000640 1 -- 172.21.15.28:0/1537881800 shutdown_connections 2024-04-26T20:57:07.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.143+0000 7fa701000640 1 -- 172.21.15.28:0/1537881800 wait complete. 2024-04-26T20:57:07.162 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T20:57:07.171 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T20:57:07.171 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T20:57:07.172 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T20:57:07.172 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-26T20:57:07.208 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-26T20:57:07.208 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-26T20:57:07.208 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-26T20:57:07.209 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:57:07.209 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:57:07.209 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-26T20:57:07.209 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:57:07.209 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T20:57:07.209 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-26T20:57:07.209 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:57:07.209 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T20:57:07.209 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-26T20:57:07.209 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T20:57:07.209 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T20:57:07.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:07.210 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-04-26T20:57:07.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:07.268 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link delete ceph-brx 2024-04-26T20:57:07.359 INFO:teuthology.orchestra.run.smithi028.stderr:Cannot find device "ceph-brx" 2024-04-26T20:57:07.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:57:07.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:07.365 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-04-26T20:57:07.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:07.424 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link delete ceph-brx 2024-04-26T20:57:07.518 INFO:teuthology.orchestra.run.smithi103.stderr:Cannot find device "ceph-brx" 2024-04-26T20:57:07.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T20:57:07.522 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-26T20:57:07.522 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:57:07.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.743+0000 7fab94800640 1 Processor -- start 2024-04-26T20:57:07.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.743+0000 7fab94800640 1 -- start start 2024-04-26T20:57:07.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.745+0000 7fab94800640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fab8c073870 0x7fab8c0b5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.745+0000 7fab94800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab8c074390 con 0x7fab8c0b5480 2024-04-26T20:57:07.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.745+0000 7fab94800640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab8c073dd0 con 0x7fab8c147c90 2024-04-26T20:57:07.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.745+0000 7fab94800640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fab8c0740b0 con 0x7fab8c073870 2024-04-26T20:57:07.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.745+0000 7fab91400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fab8c0b5480 0x7fab8c073160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:47798/0 (socket says 172.21.15.28:47798) 2024-04-26T20:57:07.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.746+0000 7fab91400640 1 -- 172.21.15.28:0/4094203136 learned_addr learned my addr 172.21.15.28:0/4094203136 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:07.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.746+0000 7fab90a00640 1 -- 172.21.15.28:0/4094203136 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2250031727 0 0) 0x7fab8c074390 con 0x7fab8c0b5480 2024-04-26T20:57:07.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.746+0000 7fab90a00640 1 -- 172.21.15.28:0/4094203136 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fab74003520 con 0x7fab8c0b5480 2024-04-26T20:57:07.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.746+0000 7fab92800640 1 --2- 172.21.15.28:0/4094203136 >> v2:172.21.15.103:3300/0 conn(0x7fab8c073870 0x7fab8c0b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.746+0000 7fab90a00640 1 -- 172.21.15.28:0/4094203136 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3848546772 0 0) 0x7fab74003520 con 0x7fab8c0b5480 2024-04-26T20:57:07.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.746+0000 7fab90a00640 1 -- 172.21.15.28:0/4094203136 >> v2:172.21.15.103:3300/0 conn(0x7fab8c073870 msgr2=0x7fab8c0b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:07.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.746+0000 7fab90a00640 1 --2- 172.21.15.28:0/4094203136 >> v2:172.21.15.103:3300/0 conn(0x7fab8c073870 0x7fab8c0b5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.746+0000 7fab90a00640 1 -- 172.21.15.28:0/4094203136 >> v1:172.21.15.103:6789/0 conn(0x7fab8c147c90 legacy=0x7fab8c06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:07.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.746+0000 7fab90a00640 1 -- 172.21.15.28:0/4094203136 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c0b3eb0 con 0x7fab8c0b5480 2024-04-26T20:57:07.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.746+0000 7fab90a00640 1 -- 172.21.15.28:0/4094203136 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fab7c001670 con 0x7fab8c0b5480 2024-04-26T20:57:07.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.747+0000 7fab90a00640 1 -- 172.21.15.28:0/4094203136 >> v1:172.21.15.28:6789/0 conn(0x7fab8c0b5480 legacy=0x7fab8c073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:07.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.747+0000 7fab90a00640 1 --2- 172.21.15.28:0/4094203136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab740039b0 0x7fab74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.747+0000 7fab90a00640 1 --2- 172.21.15.28:0/4094203136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab74004800 0x7fab74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.747+0000 7fab90a00640 1 --2- 172.21.15.28:0/4094203136 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fab740052e0 0x7fab7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.747+0000 7fab90a00640 1 -- 172.21.15.28:0/4094203136 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c0740b0 con 0x7fab74004800 2024-04-26T20:57:07.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.747+0000 7fab90a00640 1 -- 172.21.15.28:0/4094203136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fab7400c1b0 con 0x7fab740039b0 2024-04-26T20:57:07.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.747+0000 7fab90a00640 1 -- 172.21.15.28:0/4094203136 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fab7400c3b0 con 0x7fab740052e0 2024-04-26T20:57:07.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.747+0000 7fab91400640 1 --2- 172.21.15.28:0/4094203136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab74004800 0x7fab74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.747+0000 7fab91e00640 1 --2- 172.21.15.28:0/4094203136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab740039b0 0x7fab74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.747+0000 7fab92800640 1 --2- 172.21.15.28:0/4094203136 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fab740052e0 0x7fab7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.747+0000 7fab91400640 1 -- 172.21.15.28:0/4094203136 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fab740052e0 msgr2=0x7fab7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:07.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.747+0000 7fab91400640 1 --2- 172.21.15.28:0/4094203136 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fab740052e0 0x7fab7400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.747+0000 7fab91400640 1 -- 172.21.15.28:0/4094203136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab740039b0 msgr2=0x7fab74003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:07.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.747+0000 7fab91400640 1 --2- 172.21.15.28:0/4094203136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab740039b0 0x7fab74003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.747+0000 7fab91400640 1 -- 172.21.15.28:0/4094203136 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab74003520 con 0x7fab74004800 2024-04-26T20:57:07.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.748+0000 7fab92800640 1 -- 172.21.15.28:0/4094203136 reap_dead start 2024-04-26T20:57:07.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.748+0000 7fab91400640 1 --2- 172.21.15.28:0/4094203136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab74004800 0x7fab74004c80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00c210 tx=0x7fab74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf293136d61a302c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:07.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.749+0000 7fab90a00640 1 -- 172.21.15.28:0/4094203136 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c007cf0 con 0x7fab74004800 2024-04-26T20:57:07.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.749+0000 7fab90a00640 1 -- 172.21.15.28:0/4094203136 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab7c005af0 con 0x7fab74004800 2024-04-26T20:57:07.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.749+0000 7fab90a00640 1 -- 172.21.15.28:0/4094203136 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c007070 con 0x7fab74004800 2024-04-26T20:57:07.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.749+0000 7fab94800640 1 -- 172.21.15.28:0/4094203136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab74004800 msgr2=0x7fab74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:07.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.749+0000 7fab94800640 1 --2- 172.21.15.28:0/4094203136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab74004800 0x7fab74004c80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00c210 tx=0x7fab74005830 comp rx=0 tx=0).stop 2024-04-26T20:57:07.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.750+0000 7fab94800640 1 -- 172.21.15.28:0/4094203136 shutdown_connections 2024-04-26T20:57:07.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.750+0000 7fab94800640 1 --2- 172.21.15.28:0/4094203136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab74004800 0x7fab74004c80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.750+0000 7fab94800640 1 -- 172.21.15.28:0/4094203136 >> 172.21.15.28:0/4094203136 conn(0x7fab8c06c770 msgr2=0x7fab8c06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:07.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.750+0000 7fab94800640 1 -- 172.21.15.28:0/4094203136 shutdown_connections 2024-04-26T20:57:07.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.750+0000 7fab94800640 1 -- 172.21.15.28:0/4094203136 wait complete. 2024-04-26T20:57:07.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.751+0000 7fab94800640 1 Processor -- start 2024-04-26T20:57:07.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.752+0000 7fab94800640 1 -- start start 2024-04-26T20:57:07.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.753+0000 7fab94800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab74004800 0x7fab8c154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.753+0000 7fab94800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab8c15ea20 0x7fab8c13ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.753+0000 7fab94800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fab8c13e640 0x7fab8c13eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.753+0000 7fab94800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c0b5910 con 0x7fab74004800 2024-04-26T20:57:07.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.753+0000 7fab94800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c148070 con 0x7fab8c15ea20 2024-04-26T20:57:07.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.753+0000 7fab94800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fab8c0b5a90 con 0x7fab8c13e640 2024-04-26T20:57:07.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.753+0000 7fab91e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab74004800 0x7fab8c154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.753+0000 7fab91e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab74004800 0x7fab8c154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:35386/0 (socket says 172.21.15.28:35386) 2024-04-26T20:57:07.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.753+0000 7fab91e00640 1 -- 172.21.15.28:0/3049336993 learned_addr learned my addr 172.21.15.28:0/3049336993 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:07.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.753+0000 7fab91e00640 1 -- 172.21.15.28:0/3049336993 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fab8c13e640 msgr2=0x7fab8c13eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:07.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.753+0000 7fab91e00640 1 --2- 172.21.15.28:0/3049336993 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fab8c13e640 0x7fab8c13eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.753+0000 7fab91e00640 1 -- 172.21.15.28:0/3049336993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab8c15ea20 msgr2=0x7fab8c13ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:07.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.753+0000 7fab91e00640 1 --2- 172.21.15.28:0/3049336993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab8c15ea20 0x7fab8c13ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:07.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.753+0000 7fab91e00640 1 -- 172.21.15.28:0/3049336993 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c13cf80 con 0x7fab74004800 2024-04-26T20:57:07.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.754+0000 7fab91e00640 1 --2- 172.21.15.28:0/3049336993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab74004800 0x7fab8c154b10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fab8000e480 tx=0x7fab8000e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:07.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.754+0000 7fab86000640 1 -- 172.21.15.28:0/3049336993 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab800052e0 con 0x7fab74004800 2024-04-26T20:57:07.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.754+0000 7fab86000640 1 -- 172.21.15.28:0/3049336993 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab80018070 con 0x7fab74004800 2024-04-26T20:57:07.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.754+0000 7fab86000640 1 -- 172.21.15.28:0/3049336993 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab80003810 con 0x7fab74004800 2024-04-26T20:57:07.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.754+0000 7fab94800640 1 -- 172.21.15.28:0/3049336993 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8c1617b0 con 0x7fab74004800 2024-04-26T20:57:07.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.754+0000 7fab94800640 1 -- 172.21.15.28:0/3049336993 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8c161d70 con 0x7fab74004800 2024-04-26T20:57:07.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.755+0000 7fab94800640 1 -- 172.21.15.28:0/3049336993 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab8c06a400 con 0x7fab74004800 2024-04-26T20:57:07.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.756+0000 7fab86000640 1 -- 172.21.15.28:0/3049336993 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fab800039b0 con 0x7fab74004800 2024-04-26T20:57:07.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.757+0000 7fab86000640 1 --2- 172.21.15.28:0/3049336993 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fab7006c920 0x7fab7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:07.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.757+0000 7fab86000640 1 -- 172.21.15.28:0/3049336993 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fab8008b7c0 con 0x7fab74004800 2024-04-26T20:57:07.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.757+0000 7fab91400640 1 --2- 172.21.15.28:0/3049336993 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fab7006c920 0x7fab7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:07.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.758+0000 7fab91400640 1 --2- 172.21.15.28:0/3049336993 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fab7006c920 0x7fab7006ede0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fab7c032000 tx=0x7fab7c007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:07.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:07.762+0000 7fab86000640 1 -- 172.21.15.28:0/3049336993 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fab80059be0 con 0x7fab74004800 2024-04-26T20:57:08.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:08.016+0000 7fab94800640 1 -- 172.21.15.28:0/3049336993 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fab8c0cd4c0 con 0x7fab74004800 2024-04-26T20:57:08.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:08.016+0000 7fab86000640 1 -- 172.21.15.28:0/3049336993 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fab800594e0 con 0x7fab74004800 2024-04-26T20:57:08.018 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:57:08.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:08.019+0000 7fab94800640 1 -- 172.21.15.28:0/3049336993 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fab7006c920 msgr2=0x7fab7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:08.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:08.019+0000 7fab94800640 1 --2- 172.21.15.28:0/3049336993 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fab7006c920 0x7fab7006ede0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fab7c032000 tx=0x7fab7c007000 comp rx=0 tx=0).stop 2024-04-26T20:57:08.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:08.019+0000 7fab94800640 1 -- 172.21.15.28:0/3049336993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab74004800 msgr2=0x7fab8c154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:08.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:08.019+0000 7fab94800640 1 --2- 172.21.15.28:0/3049336993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab74004800 0x7fab8c154b10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fab8000e480 tx=0x7fab8000e940 comp rx=0 tx=0).stop 2024-04-26T20:57:08.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:08.019+0000 7fab94800640 1 -- 172.21.15.28:0/3049336993 shutdown_connections 2024-04-26T20:57:08.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:08.019+0000 7fab94800640 1 --2- 172.21.15.28:0/3049336993 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fab7006c920 0x7fab7006ede0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:08.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:08.019+0000 7fab94800640 1 --2- 172.21.15.28:0/3049336993 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fab8c13e640 0x7fab8c13eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:08.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:08.019+0000 7fab94800640 1 --2- 172.21.15.28:0/3049336993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fab8c15ea20 0x7fab8c13ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:08.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:08.019+0000 7fab94800640 1 --2- 172.21.15.28:0/3049336993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab74004800 0x7fab8c154b10 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:08.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:08.019+0000 7fab94800640 1 -- 172.21.15.28:0/3049336993 >> 172.21.15.28:0/3049336993 conn(0x7fab8c06c770 msgr2=0x7fab8c0b21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:08.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:08.020+0000 7fab94800640 1 -- 172.21.15.28:0/3049336993 shutdown_connections 2024-04-26T20:57:08.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:08.020+0000 7fab94800640 1 -- 172.21.15.28:0/3049336993 wait complete. 2024-04-26T20:57:08.040 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:57:08.040 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:57:08.040 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-04-26T20:57:08.040 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T20:57:08.040 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:57:08.040 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:57:08.040 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:57:08.040 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:57:08.040 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:57:08.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:08.040 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-04-26T20:57:08.094 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:57:08.094 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:57:08.094 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:57:08.094 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:08.094 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-04-26T20:57:08.095 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:08.095 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:57:08.095 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:57:08.095 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-04-26T20:57:08.095 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:57:08.095 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:57:08.095 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-04-26T20:57:08.095 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:57:08.095 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T20:57:08.095 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-04-26T20:57:08.095 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T20:57:08.095 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42608sec preferred_lft 42608sec 2024-04-26T20:57:08.095 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link noprefixroute 2024-04-26T20:57:08.095 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:08.095 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:57:08.096 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:57:08.096 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-04-26T20:57:08.097 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T20:57:08.097 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:57:08.097 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-04-26T20:57:08.097 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link add name ceph-brx type bridge 2024-04-26T20:57:08.097 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip addr flush dev ceph-brx 2024-04-26T20:57:08.097 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set ceph-brx up 2024-04-26T20:57:08.097 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T20:57:08.097 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-04-26T20:57:08.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:57:08.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:08.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:08.585 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:57:08.654 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-04-26T20:57:08.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:08.657 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-04-26T20:57:08.709 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-04-26T20:57:08.709 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:57:08.709 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T20:57:08.709 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T20:57:08.709 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:57:08.710 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:57:08.711 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-04-26T20:57:08.711 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T20:57:08.711 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T20:57:08.711 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T20:57:08.711 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-04-26T20:57:08.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:57:09.468 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:09.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:09.477 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-04-26T20:57:09.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:09.522 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list-id 2024-04-26T20:57:09.606 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:57:09.607 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-04-26T20:57:09.607 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:57:09.607 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T20:57:09.607 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-04-26T20:57:09.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:57:09.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:09.831 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T20:57:09.831 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:57:09.831 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-04-26T20:57:09.831 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T20:57:09.831 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T20:57:09.831 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T20:57:09.831 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T20:57:09.831 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T20:57:09.831 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-04-26T20:57:09.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:57:10.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:10.116 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:57:10.116 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-04-26T20:57:10.116 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-04-26T20:57:10.116 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T20:57:10.116 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-04-26T20:57:10.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:57:10.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:10.306 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-26T20:57:10.306 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:57:10.307 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T20:57:10.351 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:57:10.351 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:57:10.352 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:57:10.438 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2024-04-26T20:57:10.511 DEBUG:teuthology.orchestra.run.smithi028:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:57:10.557 INFO:teuthology.orchestra.run.smithi028.stdout:/proc 2024-04-26T20:57:10.557 INFO:teuthology.orchestra.run.smithi028.stdout:/sys 2024-04-26T20:57:10.557 INFO:teuthology.orchestra.run.smithi028.stdout:/dev 2024-04-26T20:57:10.557 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/security 2024-04-26T20:57:10.557 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/shm 2024-04-26T20:57:10.557 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/pts 2024-04-26T20:57:10.558 INFO:teuthology.orchestra.run.smithi028.stdout:/run 2024-04-26T20:57:10.558 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/cgroup 2024-04-26T20:57:10.558 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/pstore 2024-04-26T20:57:10.558 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/bpf 2024-04-26T20:57:10.558 INFO:teuthology.orchestra.run.smithi028.stdout:/ 2024-04-26T20:57:10.558 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/selinux 2024-04-26T20:57:10.558 INFO:teuthology.orchestra.run.smithi028.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:57:10.558 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/mqueue 2024-04-26T20:57:10.558 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/hugepages 2024-04-26T20:57:10.558 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/debug 2024-04-26T20:57:10.558 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/tracing 2024-04-26T20:57:10.558 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/fuse/connections 2024-04-26T20:57:10.558 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:57:10.558 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/config 2024-04-26T20:57:10.559 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:57:10.559 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:57:10.559 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:57:10.559 INFO:teuthology.orchestra.run.smithi028.stdout:/run/user/1000 2024-04-26T20:57:10.559 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph 2024-04-26T20:57:10.559 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:57:10.559 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:57:10.559 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:57:10.559 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:57:10.559 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns 2024-04-26T20:57:10.559 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:57:10.559 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:57:10.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:10.560 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2024-04-26T20:57:10.658 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T20:57:10.658 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-26T20:57:10.663 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2024-04-26T20:57:10.730 DEBUG:teuthology.orchestra.run.smithi028:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:57:10.772 INFO:teuthology.orchestra.run.smithi028.stdout:/proc 2024-04-26T20:57:10.772 INFO:teuthology.orchestra.run.smithi028.stdout:/sys 2024-04-26T20:57:10.772 INFO:teuthology.orchestra.run.smithi028.stdout:/dev 2024-04-26T20:57:10.772 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/security 2024-04-26T20:57:10.772 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/shm 2024-04-26T20:57:10.772 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/pts 2024-04-26T20:57:10.772 INFO:teuthology.orchestra.run.smithi028.stdout:/run 2024-04-26T20:57:10.772 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/cgroup 2024-04-26T20:57:10.772 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/pstore 2024-04-26T20:57:10.772 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/bpf 2024-04-26T20:57:10.772 INFO:teuthology.orchestra.run.smithi028.stdout:/ 2024-04-26T20:57:10.772 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/selinux 2024-04-26T20:57:10.773 INFO:teuthology.orchestra.run.smithi028.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:57:10.773 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/mqueue 2024-04-26T20:57:10.773 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/hugepages 2024-04-26T20:57:10.773 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/debug 2024-04-26T20:57:10.773 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/tracing 2024-04-26T20:57:10.773 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/fuse/connections 2024-04-26T20:57:10.773 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:57:10.773 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/config 2024-04-26T20:57:10.773 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:57:10.773 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:57:10.773 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:57:10.773 INFO:teuthology.orchestra.run.smithi028.stdout:/run/user/1000 2024-04-26T20:57:10.773 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph 2024-04-26T20:57:10.773 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:57:10.773 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:57:10.774 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:57:10.774 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:57:10.774 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns 2024-04-26T20:57:10.774 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:57:10.774 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:57:10.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:10.775 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2024-04-26T20:57:10.834 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:2024-04-26T20:57:10.834+0000 7f4e89462480 -1 init, newargv = 0x5611911570f0 newargc=15 2024-04-26T20:57:10.834 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:ceph-fuse[40181]: starting ceph client 2024-04-26T20:57:10.849 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:ceph-fuse[40181]: starting fuse 2024-04-26T20:57:10.859 INFO:teuthology.orchestra.run.smithi028.stdout:41 2024-04-26T20:57:10.860 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-26T20:57:10.860 DEBUG:teuthology.orchestra.run.smithi028:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:57:10.860 DEBUG:teuthology.orchestra.run.smithi028:> import glob 2024-04-26T20:57:10.860 DEBUG:teuthology.orchestra.run.smithi028:> import re 2024-04-26T20:57:10.860 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:57:10.860 DEBUG:teuthology.orchestra.run.smithi028:> import subprocess 2024-04-26T20:57:10.860 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:57:10.860 DEBUG:teuthology.orchestra.run.smithi028:> def _find_admin_socket(client_name): 2024-04-26T20:57:10.860 DEBUG:teuthology.orchestra.run.smithi028:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-26T20:57:10.860 DEBUG:teuthology.orchestra.run.smithi028:> files = glob.glob(asok_path) 2024-04-26T20:57:10.860 DEBUG:teuthology.orchestra.run.smithi028:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-26T20:57:10.860 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:57:10.861 DEBUG:teuthology.orchestra.run.smithi028:> # Given a non-glob path, it better be there 2024-04-26T20:57:10.861 DEBUG:teuthology.orchestra.run.smithi028:> if "*" not in asok_path: 2024-04-26T20:57:10.861 DEBUG:teuthology.orchestra.run.smithi028:> assert(len(files) == 1) 2024-04-26T20:57:10.861 DEBUG:teuthology.orchestra.run.smithi028:> return files[0] 2024-04-26T20:57:10.861 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:57:10.861 DEBUG:teuthology.orchestra.run.smithi028:> for f in files: 2024-04-26T20:57:10.861 DEBUG:teuthology.orchestra.run.smithi028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:57:10.861 DEBUG:teuthology.orchestra.run.smithi028:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:57:10.861 DEBUG:teuthology.orchestra.run.smithi028:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:57:10.861 DEBUG:teuthology.orchestra.run.smithi028:> contents = proc_f.read() 2024-04-26T20:57:10.861 DEBUG:teuthology.orchestra.run.smithi028:> if mountpoint in contents: 2024-04-26T20:57:10.861 DEBUG:teuthology.orchestra.run.smithi028:> return f 2024-04-26T20:57:10.861 DEBUG:teuthology.orchestra.run.smithi028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:57:10.861 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:57:10.861 DEBUG:teuthology.orchestra.run.smithi028:> print(_find_admin_socket("client.0")) 2024-04-26T20:57:10.862 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:57:11.042 INFO:teuthology.orchestra.run.smithi028.stdout:/var/run/ceph/ceph-client.0.40181.asok 2024-04-26T20:57:11.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:11.058 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.40181.asok 2024-04-26T20:57:11.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:11.058 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.40181.asok status 2024-04-26T20:57:11.286 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T20:57:11.286 INFO:teuthology.orchestra.run.smithi028.stdout: "metadata": { 2024-04-26T20:57:11.287 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:57:11.287 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:57:11.287 INFO:teuthology.orchestra.run.smithi028.stdout: "entity_id": "0", 2024-04-26T20:57:11.287 INFO:teuthology.orchestra.run.smithi028.stdout: "hostname": "smithi028", 2024-04-26T20:57:11.287 INFO:teuthology.orchestra.run.smithi028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-26T20:57:11.287 INFO:teuthology.orchestra.run.smithi028.stdout: "pid": "40181", 2024-04-26T20:57:11.287 INFO:teuthology.orchestra.run.smithi028.stdout: "root": "/" 2024-04-26T20:57:11.287 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:57:11.287 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_count": 0, 2024-04-26T20:57:11.287 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_pinned_count": 0, 2024-04-26T20:57:11.287 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 4638, 2024-04-26T20:57:11.287 INFO:teuthology.orchestra.run.smithi028.stdout: "inst": { 2024-04-26T20:57:11.287 INFO:teuthology.orchestra.run.smithi028.stdout: "name": { 2024-04-26T20:57:11.287 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "client", 2024-04-26T20:57:11.287 INFO:teuthology.orchestra.run.smithi028.stdout: "num": 4638 2024-04-26T20:57:11.288 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:57:11.288 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2024-04-26T20:57:11.288 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T20:57:11.288 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:57:11.288 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2744017993 2024-04-26T20:57:11.288 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:57:11.288 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:57:11.288 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2024-04-26T20:57:11.288 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T20:57:11.288 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:57:11.288 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2744017993 2024-04-26T20:57:11.288 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:57:11.288 INFO:teuthology.orchestra.run.smithi028.stdout: "inst_str": "client.4638 192.168.0.1:0/2744017993", 2024-04-26T20:57:11.288 INFO:teuthology.orchestra.run.smithi028.stdout: "addr_str": "192.168.0.1:0/2744017993", 2024-04-26T20:57:11.288 INFO:teuthology.orchestra.run.smithi028.stdout: "inode_count": 1, 2024-04-26T20:57:11.288 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_epoch": 8, 2024-04-26T20:57:11.289 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch": 21, 2024-04-26T20:57:11.289 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:57:11.289 INFO:teuthology.orchestra.run.smithi028.stdout: "blocklisted": false, 2024-04-26T20:57:11.289 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs" 2024-04-26T20:57:11.289 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T20:57:11.301 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:57:11.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.511+0000 7f4a2e200640 1 Processor -- start 2024-04-26T20:57:11.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.511+0000 7f4a2e200640 1 -- start start 2024-04-26T20:57:11.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.512+0000 7f4a2e200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4a2815f430 0x7f4a280b8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:11.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.513+0000 7f4a2e200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f4a280d1770 0x7f4a280d1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:11.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.513+0000 7f4a2e200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a280d0280 con 0x7f4a280b4570 2024-04-26T20:57:11.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.513+0000 7f4a2e200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f4a280d0040 con 0x7f4a280d1770 2024-04-26T20:57:11.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.513+0000 7f4a2e200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f4a280d0560 con 0x7f4a2815f430 2024-04-26T20:57:11.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.513+0000 7f4a27e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f4a280d1770 0x7f4a280d1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:11.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.513+0000 7f4a27e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f4a280d1770 0x7f4a280d1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.28:44602/0 (socket says 172.21.15.28:44602) 2024-04-26T20:57:11.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.513+0000 7f4a26a00640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f4a280b4570 0x7f4a280cf930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.28:34480/0 (socket says 172.21.15.28:34480) 2024-04-26T20:57:11.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.513+0000 7f4a27e00640 1 -- 172.21.15.28:0/2676986728 learned_addr learned my addr 172.21.15.28:0/2676986728 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:11.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.513+0000 7f4a27400640 1 --2- 172.21.15.28:0/2676986728 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4a2815f430 0x7f4a280b8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:11.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.514+0000 7f4a27e00640 1 -- 172.21.15.28:0/2676986728 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4a2815f430 msgr2=0x7f4a280b8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:11.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.514+0000 7f4a27e00640 1 --2- 172.21.15.28:0/2676986728 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4a2815f430 0x7f4a280b8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:11.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.514+0000 7f4a27e00640 1 -- 172.21.15.28:0/2676986728 >> v1:172.21.15.103:6789/0 conn(0x7f4a280b4570 legacy=0x7f4a280cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:11.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.514+0000 7f4a27e00640 1 -- 172.21.15.28:0/2676986728 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a280d0840 con 0x7f4a280d1770 2024-04-26T20:57:11.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.516+0000 7f4a27e00640 1 --2- 172.21.15.28:0/2676986728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a280d1770 0x7f4a280d1bb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4a20009870 tx=0x7f4a2002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=829fdb198f55b367 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:11.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.516+0000 7f4a26000640 1 -- 172.21.15.28:0/2676986728 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20037070 con 0x7f4a280d1770 2024-04-26T20:57:11.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.516+0000 7f4a26000640 1 -- 172.21.15.28:0/2676986728 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a20032440 con 0x7f4a280d1770 2024-04-26T20:57:11.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.516+0000 7f4a26000640 1 -- 172.21.15.28:0/2676986728 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20032810 con 0x7f4a280d1770 2024-04-26T20:57:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.517+0000 7f4a2e200640 1 -- 172.21.15.28:0/2676986728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a280d1770 msgr2=0x7f4a280d1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.517+0000 7f4a2e200640 1 --2- 172.21.15.28:0/2676986728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a280d1770 0x7f4a280d1bb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4a20009870 tx=0x7f4a2002a1c0 comp rx=0 tx=0).stop 2024-04-26T20:57:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.517+0000 7f4a2e200640 1 -- 172.21.15.28:0/2676986728 shutdown_connections 2024-04-26T20:57:11.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.517+0000 7f4a2e200640 1 --2- 172.21.15.28:0/2676986728 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a280d1770 0x7f4a280d1bb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:11.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.517+0000 7f4a2e200640 1 --2- 172.21.15.28:0/2676986728 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4a2815f430 0x7f4a280b8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:11.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.517+0000 7f4a2e200640 1 -- 172.21.15.28:0/2676986728 >> 172.21.15.28:0/2676986728 conn(0x7f4a2806c9e0 msgr2=0x7f4a28072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:11.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.517+0000 7f4a2e200640 1 -- 172.21.15.28:0/2676986728 shutdown_connections 2024-04-26T20:57:11.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.517+0000 7f4a2e200640 1 -- 172.21.15.28:0/2676986728 wait complete. 2024-04-26T20:57:11.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.520+0000 7f4a2e200640 1 Processor -- start 2024-04-26T20:57:11.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.520+0000 7f4a2e200640 1 -- start start 2024-04-26T20:57:11.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.521+0000 7f4a2e200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4a280b4570 0x7f4a281518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:11.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.521+0000 7f4a2e200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a280d1770 0x7f4a280b5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:11.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.521+0000 7f4a2e200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a2815f430 0x7f4a280cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:11.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.521+0000 7f4a2e200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2815f810 con 0x7f4a2815f430 2024-04-26T20:57:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.521+0000 7f4a2e200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4a280d20f0 con 0x7f4a280d1770 2024-04-26T20:57:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.521+0000 7f4a2e200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f4a280d38c0 con 0x7f4a280b4570 2024-04-26T20:57:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.522+0000 7f4a27e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a2815f430 0x7f4a280cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.522+0000 7f4a27e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a2815f430 0x7f4a280cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45400/0 (socket says 172.21.15.28:45400) 2024-04-26T20:57:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.522+0000 7f4a27e00640 1 -- 172.21.15.28:0/3888253732 learned_addr learned my addr 172.21.15.28:0/3888253732 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.522+0000 7f4a27400640 1 --2- 172.21.15.28:0/3888253732 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4a280b4570 0x7f4a281518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.522+0000 7f4a26a00640 1 --2- 172.21.15.28:0/3888253732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a280d1770 0x7f4a280b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.522+0000 7f4a27e00640 1 -- 172.21.15.28:0/3888253732 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4a280b4570 msgr2=0x7f4a281518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.522+0000 7f4a27e00640 1 --2- 172.21.15.28:0/3888253732 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4a280b4570 0x7f4a281518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.522+0000 7f4a27e00640 1 -- 172.21.15.28:0/3888253732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a280d1770 msgr2=0x7f4a280b5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.522+0000 7f4a27e00640 1 --2- 172.21.15.28:0/3888253732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a280d1770 0x7f4a280b5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:11.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.522+0000 7f4a27e00640 1 -- 172.21.15.28:0/3888253732 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a280b5fd0 con 0x7f4a2815f430 2024-04-26T20:57:11.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.523+0000 7f4a27e00640 1 --2- 172.21.15.28:0/3888253732 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a2815f430 0x7f4a280cc350 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4a20009870 tx=0x7f4a200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:11.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.523+0000 7f4a1f400640 1 -- 172.21.15.28:0/3888253732 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20037040 con 0x7f4a2815f430 2024-04-26T20:57:11.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.523+0000 7f4a1f400640 1 -- 172.21.15.28:0/3888253732 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a20042070 con 0x7f4a2815f430 2024-04-26T20:57:11.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.523+0000 7f4a2e200640 1 -- 172.21.15.28:0/3888253732 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a28161260 con 0x7f4a2815f430 2024-04-26T20:57:11.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.523+0000 7f4a1f400640 1 -- 172.21.15.28:0/3888253732 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2003b930 con 0x7f4a2815f430 2024-04-26T20:57:11.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.523+0000 7f4a2e200640 1 -- 172.21.15.28:0/3888253732 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a28161820 con 0x7f4a2815f430 2024-04-26T20:57:11.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.524+0000 7f4a2e200640 1 -- 172.21.15.28:0/3888253732 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a2806a400 con 0x7f4a2815f430 2024-04-26T20:57:11.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.526+0000 7f4a1f400640 1 -- 172.21.15.28:0/3888253732 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a2002ad10 con 0x7f4a2815f430 2024-04-26T20:57:11.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.527+0000 7f4a1f400640 1 --2- 172.21.15.28:0/3888253732 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4a0806c8f0 0x7f4a0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:11.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.527+0000 7f4a1f400640 1 -- 172.21.15.28:0/3888253732 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4a200aca50 con 0x7f4a2815f430 2024-04-26T20:57:11.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.528+0000 7f4a27400640 1 --2- 172.21.15.28:0/3888253732 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4a0806c8f0 0x7f4a0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:11.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.529+0000 7f4a27400640 1 --2- 172.21.15.28:0/3888253732 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4a0806c8f0 0x7f4a0806edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4a18011340 tx=0x7f4a1800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:11.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.531+0000 7f4a1f400640 1 -- 172.21.15.28:0/3888253732 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4a2007a8a0 con 0x7f4a2815f430 2024-04-26T20:57:11.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.775+0000 7f4a2e200640 1 -- 172.21.15.28:0/3888253732 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4a280d2f70 con 0x7f4a2815f430 2024-04-26T20:57:11.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.776+0000 7f4a1f400640 1 -- 172.21.15.28:0/3888253732 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f4a2007eba0 con 0x7f4a2815f430 2024-04-26T20:57:11.776 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:57:11.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.778+0000 7f4a2e200640 1 -- 172.21.15.28:0/3888253732 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4a0806c8f0 msgr2=0x7f4a0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:11.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.778+0000 7f4a2e200640 1 --2- 172.21.15.28:0/3888253732 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4a0806c8f0 0x7f4a0806edb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4a18011340 tx=0x7f4a1800b040 comp rx=0 tx=0).stop 2024-04-26T20:57:11.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.778+0000 7f4a2e200640 1 -- 172.21.15.28:0/3888253732 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a2815f430 msgr2=0x7f4a280cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:11.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.778+0000 7f4a2e200640 1 --2- 172.21.15.28:0/3888253732 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a2815f430 0x7f4a280cc350 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4a20009870 tx=0x7f4a200046c0 comp rx=0 tx=0).stop 2024-04-26T20:57:11.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.779+0000 7f4a2e200640 1 -- 172.21.15.28:0/3888253732 shutdown_connections 2024-04-26T20:57:11.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.779+0000 7f4a2e200640 1 --2- 172.21.15.28:0/3888253732 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4a0806c8f0 0x7f4a0806edb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:11.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.779+0000 7f4a2e200640 1 --2- 172.21.15.28:0/3888253732 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a2815f430 0x7f4a280cc350 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:11.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.779+0000 7f4a2e200640 1 --2- 172.21.15.28:0/3888253732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a280d1770 0x7f4a280b5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:11.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.779+0000 7f4a2e200640 1 --2- 172.21.15.28:0/3888253732 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4a280b4570 0x7f4a281518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:11.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.779+0000 7f4a2e200640 1 -- 172.21.15.28:0/3888253732 >> 172.21.15.28:0/3888253732 conn(0x7f4a2806c9e0 msgr2=0x7f4a28157c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:11.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.779+0000 7f4a2e200640 1 -- 172.21.15.28:0/3888253732 shutdown_connections 2024-04-26T20:57:11.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:11.779+0000 7f4a2e200640 1 -- 172.21.15.28:0/3888253732 wait complete. 2024-04-26T20:57:11.799 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:57:11.799 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:57:11.799 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-04-26T20:57:11.799 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-26T20:57:11.799 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T20:57:11.799 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:57:11.799 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:57:11.799 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:57:11.799 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-26T20:57:11.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:11.799 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-04-26T20:57:11.851 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:57:11.851 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:57:11.851 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:57:11.851 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:11.851 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-04-26T20:57:11.851 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:11.851 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:57:11.851 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:57:11.851 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-04-26T20:57:11.851 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:57:11.851 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:57:11.851 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-04-26T20:57:11.851 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:57:11.852 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T20:57:11.852 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-04-26T20:57:11.852 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T20:57:11.852 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42604sec preferred_lft 42604sec 2024-04-26T20:57:11.852 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link noprefixroute 2024-04-26T20:57:11.852 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:11.852 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:57:11.852 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:57:11.852 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-04-26T20:57:11.852 INFO:teuthology.orchestra.run.smithi028.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:57:11.852 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 4a:2a:43:26:cb:b2 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:57:11.852 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:57:11.852 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:11.852 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::608e:e5ff:fecb:cfa2/64 scope link 2024-04-26T20:57:11.853 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:11.853 INFO:teuthology.orchestra.run.smithi028.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:57:11.853 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 4a:2a:43:26:cb:b2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:57:11.853 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::482a:43ff:fe26:cbb2/64 scope link tentative 2024-04-26T20:57:11.853 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:11.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:11.854 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:57:11.931 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-04-26T20:57:11.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:11.935 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-04-26T20:57:11.990 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-04-26T20:57:11.990 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:57:11.990 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T20:57:11.990 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T20:57:11.991 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:57:11.992 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:57:11.992 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-04-26T20:57:11.992 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T20:57:11.992 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T20:57:11.992 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T20:57:11.992 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-04-26T20:57:12.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:57:12.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:12.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:12.220 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-04-26T20:57:12.271 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:57:12.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:12.273 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list-id 2024-04-26T20:57:12.329 INFO:teuthology.orchestra.run.smithi028.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-26T20:57:12.331 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:57:12.331 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-04-26T20:57:12.331 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:57:12.331 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-26T20:57:12.331 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-04-26T20:57:12.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:57:12.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:12.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:12.528 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T20:57:12.615 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:57:12.615 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:57:12.615 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:57:12.615 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:12.615 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-04-26T20:57:12.615 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:12.615 INFO:teuthology.orchestra.run.smithi028.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:57:12.616 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether fe:e9:df:23:33:c3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T20:57:12.616 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T20:57:12.616 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:12.616 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::fce9:dfff:fe23:33c3/64 scope link 2024-04-26T20:57:12.616 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:12.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:12.620 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T20:57:12.706 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:57:12.706 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:57:12.707 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:57:12.707 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:12.707 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-04-26T20:57:12.707 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:12.707 INFO:teuthology.orchestra.run.smithi028.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:57:12.707 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether fe:e9:df:23:33:c3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T20:57:12.707 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T20:57:12.707 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:12.707 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::fce9:dfff:fe23:33c3/64 scope link 2024-04-26T20:57:12.707 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:12.711 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-26T20:57:12.711 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:57:12.711 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-04-26T20:57:12.711 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-26T20:57:12.712 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T20:57:12.712 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-26T20:57:12.712 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-26T20:57:12.712 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-26T20:57:12.712 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-04-26T20:57:12.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:57:12.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:13.005 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:57:13.006 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-04-26T20:57:13.006 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.1 up 2024-04-26T20:57:13.006 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T20:57:13.006 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-04-26T20:57:13.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:57:13.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:13.219 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-26T20:57:13.219 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:57:13.219 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-26T20:57:13.272 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-26T20:57:13.272 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:57:13.272 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-26T20:57:13.322 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2024-04-26T20:57:13.409 DEBUG:teuthology.orchestra.run.smithi028:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:57:13.467 INFO:teuthology.orchestra.run.smithi028.stdout:/proc 2024-04-26T20:57:13.467 INFO:teuthology.orchestra.run.smithi028.stdout:/sys 2024-04-26T20:57:13.467 INFO:teuthology.orchestra.run.smithi028.stdout:/dev 2024-04-26T20:57:13.467 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/security 2024-04-26T20:57:13.467 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/shm 2024-04-26T20:57:13.467 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/pts 2024-04-26T20:57:13.468 INFO:teuthology.orchestra.run.smithi028.stdout:/run 2024-04-26T20:57:13.468 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/cgroup 2024-04-26T20:57:13.468 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/pstore 2024-04-26T20:57:13.468 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/bpf 2024-04-26T20:57:13.468 INFO:teuthology.orchestra.run.smithi028.stdout:/ 2024-04-26T20:57:13.468 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/selinux 2024-04-26T20:57:13.468 INFO:teuthology.orchestra.run.smithi028.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:57:13.468 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/mqueue 2024-04-26T20:57:13.468 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/hugepages 2024-04-26T20:57:13.468 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/debug 2024-04-26T20:57:13.468 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/tracing 2024-04-26T20:57:13.468 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/fuse/connections 2024-04-26T20:57:13.468 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:57:13.469 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/config 2024-04-26T20:57:13.469 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:57:13.469 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:57:13.469 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:57:13.469 INFO:teuthology.orchestra.run.smithi028.stdout:/run/user/1000 2024-04-26T20:57:13.469 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph 2024-04-26T20:57:13.469 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:57:13.469 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:57:13.469 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:57:13.469 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:57:13.469 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns 2024-04-26T20:57:13.469 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:57:13.469 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:57:13.469 INFO:teuthology.orchestra.run.smithi028.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-26T20:57:13.470 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:57:13.470 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:57:13.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:13.471 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2024-04-26T20:57:13.525 INFO:teuthology.orchestra.run.smithi028.stdout:41 2024-04-26T20:57:13.525 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-04-26T20:57:13.526 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-26T20:57:13.533 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2024-04-26T20:57:13.607 DEBUG:teuthology.orchestra.run.smithi028:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:57:13.650 INFO:teuthology.orchestra.run.smithi028.stdout:/proc 2024-04-26T20:57:13.650 INFO:teuthology.orchestra.run.smithi028.stdout:/sys 2024-04-26T20:57:13.650 INFO:teuthology.orchestra.run.smithi028.stdout:/dev 2024-04-26T20:57:13.650 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/security 2024-04-26T20:57:13.650 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/shm 2024-04-26T20:57:13.650 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/pts 2024-04-26T20:57:13.651 INFO:teuthology.orchestra.run.smithi028.stdout:/run 2024-04-26T20:57:13.651 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/cgroup 2024-04-26T20:57:13.651 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/pstore 2024-04-26T20:57:13.651 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/bpf 2024-04-26T20:57:13.651 INFO:teuthology.orchestra.run.smithi028.stdout:/ 2024-04-26T20:57:13.651 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/selinux 2024-04-26T20:57:13.651 INFO:teuthology.orchestra.run.smithi028.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:57:13.651 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/mqueue 2024-04-26T20:57:13.651 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/hugepages 2024-04-26T20:57:13.651 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/debug 2024-04-26T20:57:13.651 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/tracing 2024-04-26T20:57:13.651 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/fuse/connections 2024-04-26T20:57:13.651 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:57:13.651 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/config 2024-04-26T20:57:13.652 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:57:13.652 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:57:13.652 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:57:13.652 INFO:teuthology.orchestra.run.smithi028.stdout:/run/user/1000 2024-04-26T20:57:13.652 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph 2024-04-26T20:57:13.652 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:57:13.652 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:57:13.652 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:57:13.652 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:57:13.652 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns 2024-04-26T20:57:13.652 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:57:13.652 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:57:13.652 INFO:teuthology.orchestra.run.smithi028.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-26T20:57:13.652 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:57:13.653 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:57:13.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:13.654 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2024-04-26T20:57:13.707 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi028.stderr:2024-04-26T20:57:13.708+0000 7fd08b40b480 -1 init, newargv = 0x55feea3b0d10 newargc=15 2024-04-26T20:57:13.708 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi028.stderr:ceph-fuse[40679]: starting ceph client 2024-04-26T20:57:13.719 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi028.stderr:ceph-fuse[40679]: starting fuse 2024-04-26T20:57:13.739 INFO:teuthology.orchestra.run.smithi028.stdout:41 2024-04-26T20:57:13.739 INFO:teuthology.orchestra.run.smithi028.stdout:43 2024-04-26T20:57:13.740 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 43] 2024-04-26T20:57:13.740 DEBUG:teuthology.orchestra.run.smithi028:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:57:13.740 DEBUG:teuthology.orchestra.run.smithi028:> import glob 2024-04-26T20:57:13.740 DEBUG:teuthology.orchestra.run.smithi028:> import re 2024-04-26T20:57:13.740 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:57:13.740 DEBUG:teuthology.orchestra.run.smithi028:> import subprocess 2024-04-26T20:57:13.740 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:57:13.740 DEBUG:teuthology.orchestra.run.smithi028:> def _find_admin_socket(client_name): 2024-04-26T20:57:13.740 DEBUG:teuthology.orchestra.run.smithi028:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-26T20:57:13.740 DEBUG:teuthology.orchestra.run.smithi028:> files = glob.glob(asok_path) 2024-04-26T20:57:13.740 DEBUG:teuthology.orchestra.run.smithi028:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-26T20:57:13.741 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:57:13.741 DEBUG:teuthology.orchestra.run.smithi028:> # Given a non-glob path, it better be there 2024-04-26T20:57:13.741 DEBUG:teuthology.orchestra.run.smithi028:> if "*" not in asok_path: 2024-04-26T20:57:13.741 DEBUG:teuthology.orchestra.run.smithi028:> assert(len(files) == 1) 2024-04-26T20:57:13.741 DEBUG:teuthology.orchestra.run.smithi028:> return files[0] 2024-04-26T20:57:13.741 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:57:13.741 DEBUG:teuthology.orchestra.run.smithi028:> for f in files: 2024-04-26T20:57:13.741 DEBUG:teuthology.orchestra.run.smithi028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:57:13.741 DEBUG:teuthology.orchestra.run.smithi028:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:57:13.741 DEBUG:teuthology.orchestra.run.smithi028:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:57:13.741 DEBUG:teuthology.orchestra.run.smithi028:> contents = proc_f.read() 2024-04-26T20:57:13.741 DEBUG:teuthology.orchestra.run.smithi028:> if mountpoint in contents: 2024-04-26T20:57:13.741 DEBUG:teuthology.orchestra.run.smithi028:> return f 2024-04-26T20:57:13.741 DEBUG:teuthology.orchestra.run.smithi028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:57:13.742 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:57:13.742 DEBUG:teuthology.orchestra.run.smithi028:> print(_find_admin_socket("client.2")) 2024-04-26T20:57:13.742 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:57:13.937 INFO:teuthology.orchestra.run.smithi028.stdout:/var/run/ceph/ceph-client.2.40679.asok 2024-04-26T20:57:13.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:13.954 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.40679.asok 2024-04-26T20:57:13.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:13.955 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.40679.asok status 2024-04-26T20:57:14.192 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T20:57:14.192 INFO:teuthology.orchestra.run.smithi028.stdout: "metadata": { 2024-04-26T20:57:14.192 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:57:14.192 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:57:14.192 INFO:teuthology.orchestra.run.smithi028.stdout: "entity_id": "2", 2024-04-26T20:57:14.192 INFO:teuthology.orchestra.run.smithi028.stdout: "hostname": "smithi028", 2024-04-26T20:57:14.192 INFO:teuthology.orchestra.run.smithi028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-26T20:57:14.192 INFO:teuthology.orchestra.run.smithi028.stdout: "pid": "40679", 2024-04-26T20:57:14.192 INFO:teuthology.orchestra.run.smithi028.stdout: "root": "/" 2024-04-26T20:57:14.192 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:57:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_count": 0, 2024-04-26T20:57:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_pinned_count": 0, 2024-04-26T20:57:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 4647, 2024-04-26T20:57:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "inst": { 2024-04-26T20:57:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "name": { 2024-04-26T20:57:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "client", 2024-04-26T20:57:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "num": 4647 2024-04-26T20:57:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:57:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2024-04-26T20:57:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T20:57:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.2:0", 2024-04-26T20:57:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4246610957 2024-04-26T20:57:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:57:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:57:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2024-04-26T20:57:14.193 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T20:57:14.194 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.2:0", 2024-04-26T20:57:14.194 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 4246610957 2024-04-26T20:57:14.194 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:57:14.194 INFO:teuthology.orchestra.run.smithi028.stdout: "inst_str": "client.4647 192.168.0.2:0/4246610957", 2024-04-26T20:57:14.194 INFO:teuthology.orchestra.run.smithi028.stdout: "addr_str": "192.168.0.2:0/4246610957", 2024-04-26T20:57:14.194 INFO:teuthology.orchestra.run.smithi028.stdout: "inode_count": 1, 2024-04-26T20:57:14.194 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_epoch": 8, 2024-04-26T20:57:14.194 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch": 21, 2024-04-26T20:57:14.194 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:57:14.194 INFO:teuthology.orchestra.run.smithi028.stdout: "blocklisted": false, 2024-04-26T20:57:14.194 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs" 2024-04-26T20:57:14.194 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T20:57:14.207 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:57:14.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.412+0000 7f19cf200640 1 Processor -- start 2024-04-26T20:57:14.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.413+0000 7f19cf200640 1 -- start start 2024-04-26T20:57:14.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.414+0000 7f19cf200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f19d015f2e0 0x7f19d006c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:14.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.414+0000 7f19cf200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f19d00d2a00 0x7f19d006c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:14.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.414+0000 7f19cf200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19d00d3db0 con 0x7f19d00d38f0 2024-04-26T20:57:14.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.414+0000 7f19cf200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f19d00d4090 con 0x7f19d015f2e0 2024-04-26T20:57:14.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.414+0000 7f19cf200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f19d00d4370 con 0x7f19d00d2a00 2024-04-26T20:57:14.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.414+0000 7f19cde00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f19d015f2e0 0x7f19d006c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:14.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.415+0000 7f19cd400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f19d00d38f0 0x7f19d00b38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:56100/0 (socket says 172.21.15.28:56100) 2024-04-26T20:57:14.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.415+0000 7f19cd400640 1 -- 172.21.15.28:0/293380773 learned_addr learned my addr 172.21.15.28:0/293380773 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:14.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.415+0000 7f19ce800640 1 --2- 172.21.15.28:0/293380773 >> v2:172.21.15.103:3300/0 conn(0x7f19d00d2a00 0x7f19d006c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:14.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.415+0000 7f19cde00640 1 -- 172.21.15.28:0/293380773 >> v2:172.21.15.103:3300/0 conn(0x7f19d00d2a00 msgr2=0x7f19d006c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:14.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.415+0000 7f19cde00640 1 --2- 172.21.15.28:0/293380773 >> v2:172.21.15.103:3300/0 conn(0x7f19d00d2a00 0x7f19d006c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:14.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.415+0000 7f19cde00640 1 -- 172.21.15.28:0/293380773 >> v1:172.21.15.28:6789/0 conn(0x7f19d00d38f0 legacy=0x7f19d00b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:14.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.415+0000 7f19cde00640 1 -- 172.21.15.28:0/293380773 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d00d4650 con 0x7f19d015f2e0 2024-04-26T20:57:14.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.415+0000 7f19cca00640 1 -- 172.21.15.28:0/293380773 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 224466841 0 0) 0x7f19d00d3db0 con 0x7f19d00d38f0 2024-04-26T20:57:14.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.416+0000 7f19cde00640 1 --2- 172.21.15.28:0/293380773 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d015f2e0 0x7f19d006c350 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f19d00b7930 tx=0x7f19c002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a802e2aa0e1314 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:14.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.416+0000 7f19cca00640 1 -- 172.21.15.28:0/293380773 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c0037070 con 0x7f19d015f2e0 2024-04-26T20:57:14.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.417+0000 7f19cca00640 1 -- 172.21.15.28:0/293380773 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19c0032440 con 0x7f19d015f2e0 2024-04-26T20:57:14.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.417+0000 7f19cca00640 1 -- 172.21.15.28:0/293380773 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c002ae60 con 0x7f19d015f2e0 2024-04-26T20:57:14.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.417+0000 7f19cf200640 1 -- 172.21.15.28:0/293380773 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d015f2e0 msgr2=0x7f19d006c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:14.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.417+0000 7f19cf200640 1 --2- 172.21.15.28:0/293380773 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d015f2e0 0x7f19d006c350 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f19d00b7930 tx=0x7f19c002a110 comp rx=0 tx=0).stop 2024-04-26T20:57:14.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.417+0000 7f19cf200640 1 -- 172.21.15.28:0/293380773 shutdown_connections 2024-04-26T20:57:14.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.417+0000 7f19cf200640 1 --2- 172.21.15.28:0/293380773 >> v2:172.21.15.103:3300/0 conn(0x7f19d00d2a00 0x7f19d006c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:14.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.417+0000 7f19cf200640 1 --2- 172.21.15.28:0/293380773 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d015f2e0 0x7f19d006c350 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:14.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.417+0000 7f19cf200640 1 -- 172.21.15.28:0/293380773 >> 172.21.15.28:0/293380773 conn(0x7f19d00d4e30 msgr2=0x7f19d00d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:14.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.417+0000 7f19cf200640 1 -- 172.21.15.28:0/293380773 shutdown_connections 2024-04-26T20:57:14.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.417+0000 7f19cf200640 1 -- 172.21.15.28:0/293380773 wait complete. 2024-04-26T20:57:14.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.419+0000 7f19cf200640 1 Processor -- start 2024-04-26T20:57:14.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.419+0000 7f19cf200640 1 -- start start 2024-04-26T20:57:14.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.420+0000 7f19cf200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19d00d2a00 0x7f19d00ce360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:14.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.420+0000 7f19cf200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19d00d38f0 0x7f19d01f8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:14.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.420+0000 7f19cf200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d015f2e0 0x7f19d01e7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:14.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.420+0000 7f19cf200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f19d006d430 con 0x7f19d015f2e0 2024-04-26T20:57:14.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.420+0000 7f19cf200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19d006d2b0 con 0x7f19d00d38f0 2024-04-26T20:57:14.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.421+0000 7f19cf200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f19d00b3fb0 con 0x7f19d00d2a00 2024-04-26T20:57:14.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.421+0000 7f19ce800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d015f2e0 0x7f19d01e7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:14.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.421+0000 7f19ce800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d015f2e0 0x7f19d01e7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45420/0 (socket says 172.21.15.28:45420) 2024-04-26T20:57:14.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.421+0000 7f19ce800640 1 -- 172.21.15.28:0/3764297431 learned_addr learned my addr 172.21.15.28:0/3764297431 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:14.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.421+0000 7f19cde00640 1 --2- 172.21.15.28:0/3764297431 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19d00d2a00 0x7f19d00ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:14.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.421+0000 7f19cd400640 1 --2- 172.21.15.28:0/3764297431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19d00d38f0 0x7f19d01f8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:14.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.421+0000 7f19ce800640 1 -- 172.21.15.28:0/3764297431 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19d00d2a00 msgr2=0x7f19d00ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:14.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.421+0000 7f19ce800640 1 --2- 172.21.15.28:0/3764297431 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19d00d2a00 0x7f19d00ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:14.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.421+0000 7f19ce800640 1 -- 172.21.15.28:0/3764297431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19d00d38f0 msgr2=0x7f19d01f8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:14.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.421+0000 7f19ce800640 1 --2- 172.21.15.28:0/3764297431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19d00d38f0 0x7f19d01f8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:14.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.421+0000 7f19ce800640 1 -- 172.21.15.28:0/3764297431 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d00ce8a0 con 0x7f19d015f2e0 2024-04-26T20:57:14.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.421+0000 7f19cde00640 1 --2- 172.21.15.28:0/3764297431 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19d00d2a00 0x7f19d00ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:14.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.421+0000 7f19cd400640 1 --2- 172.21.15.28:0/3764297431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19d00d38f0 0x7f19d01f8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:14.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.422+0000 7f19ce800640 1 --2- 172.21.15.28:0/3764297431 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d015f2e0 0x7f19d01e7960 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f19c8007e70 tx=0x7f19c800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:14.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.422+0000 7f19c6000640 1 -- 172.21.15.28:0/3764297431 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8014620 con 0x7f19d015f2e0 2024-04-26T20:57:14.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.422+0000 7f19c6000640 1 -- 172.21.15.28:0/3764297431 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19c8019070 con 0x7f19d015f2e0 2024-04-26T20:57:14.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.422+0000 7f19c6000640 1 -- 172.21.15.28:0/3764297431 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8014a20 con 0x7f19d015f2e0 2024-04-26T20:57:14.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.422+0000 7f19cf200640 1 -- 172.21.15.28:0/3764297431 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d01f9110 con 0x7f19d015f2e0 2024-04-26T20:57:14.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.422+0000 7f19cf200640 1 -- 172.21.15.28:0/3764297431 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d00cd880 con 0x7f19d015f2e0 2024-04-26T20:57:14.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.423+0000 7f19cf200640 1 -- 172.21.15.28:0/3764297431 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19d006a360 con 0x7f19d015f2e0 2024-04-26T20:57:14.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.424+0000 7f19c6000640 1 -- 172.21.15.28:0/3764297431 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19c8014bc0 con 0x7f19d015f2e0 2024-04-26T20:57:14.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.425+0000 7f19c6000640 1 --2- 172.21.15.28:0/3764297431 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f19b006c8f0 0x7f19b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:14.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.425+0000 7f19c6000640 1 -- 172.21.15.28:0/3764297431 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f19c808c540 con 0x7f19d015f2e0 2024-04-26T20:57:14.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.426+0000 7f19cde00640 1 --2- 172.21.15.28:0/3764297431 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f19b006c8f0 0x7f19b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:14.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.426+0000 7f19cde00640 1 --2- 172.21.15.28:0/3764297431 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f19b006c8f0 0x7f19b006edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f19c0034000 tx=0x7f19c002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:14.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.430+0000 7f19c6000640 1 -- 172.21.15.28:0/3764297431 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f19c8011260 con 0x7f19d015f2e0 2024-04-26T20:57:14.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.676+0000 7f19cf200640 1 -- 172.21.15.28:0/3764297431 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f19d006ce80 con 0x7f19d015f2e0 2024-04-26T20:57:14.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.677+0000 7f19c6000640 1 -- 172.21.15.28:0/3764297431 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f19c805a8d0 con 0x7f19d015f2e0 2024-04-26T20:57:14.678 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:57:14.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.680+0000 7f19cf200640 1 -- 172.21.15.28:0/3764297431 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f19b006c8f0 msgr2=0x7f19b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:14.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.680+0000 7f19cf200640 1 --2- 172.21.15.28:0/3764297431 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f19b006c8f0 0x7f19b006edb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f19c0034000 tx=0x7f19c002e000 comp rx=0 tx=0).stop 2024-04-26T20:57:14.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.680+0000 7f19cf200640 1 -- 172.21.15.28:0/3764297431 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d015f2e0 msgr2=0x7f19d01e7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:14.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.680+0000 7f19cf200640 1 --2- 172.21.15.28:0/3764297431 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d015f2e0 0x7f19d01e7960 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f19c8007e70 tx=0x7f19c800e4b0 comp rx=0 tx=0).stop 2024-04-26T20:57:14.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.680+0000 7f19cf200640 1 -- 172.21.15.28:0/3764297431 shutdown_connections 2024-04-26T20:57:14.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.680+0000 7f19cf200640 1 --2- 172.21.15.28:0/3764297431 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f19b006c8f0 0x7f19b006edb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:14.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.680+0000 7f19cf200640 1 --2- 172.21.15.28:0/3764297431 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d015f2e0 0x7f19d01e7960 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:14.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.680+0000 7f19cf200640 1 --2- 172.21.15.28:0/3764297431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19d00d38f0 0x7f19d01f8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:14.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.680+0000 7f19cf200640 1 --2- 172.21.15.28:0/3764297431 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19d00d2a00 0x7f19d00ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:14.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.680+0000 7f19cf200640 1 -- 172.21.15.28:0/3764297431 >> 172.21.15.28:0/3764297431 conn(0x7f19d00d4e30 msgr2=0x7f19d00d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:14.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.680+0000 7f19cf200640 1 -- 172.21.15.28:0/3764297431 shutdown_connections 2024-04-26T20:57:14.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:14.681+0000 7f19cf200640 1 -- 172.21.15.28:0/3764297431 wait complete. 2024-04-26T20:57:14.702 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:57:14.702 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:57:14.703 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi103.front.sepia.ceph.com 2024-04-26T20:57:14.703 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T20:57:14.703 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T20:57:14.703 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:57:14.703 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:57:14.703 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:57:14.703 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:57:14.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:14.703 DEBUG:teuthology.orchestra.run.smithi103:> ip addr 2024-04-26T20:57:14.759 INFO:teuthology.orchestra.run.smithi103.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:57:14.760 INFO:teuthology.orchestra.run.smithi103.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:57:14.760 INFO:teuthology.orchestra.run.smithi103.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:57:14.760 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:14.760 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 ::1/128 scope host 2024-04-26T20:57:14.760 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:14.760 INFO:teuthology.orchestra.run.smithi103.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:57:14.760 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c4 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:57:14.760 INFO:teuthology.orchestra.run.smithi103.stdout: altname enp7s0f0 2024-04-26T20:57:14.760 INFO:teuthology.orchestra.run.smithi103.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:57:14.760 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c5 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:57:14.760 INFO:teuthology.orchestra.run.smithi103.stdout: altname enp7s0f1 2024-04-26T20:57:14.760 INFO:teuthology.orchestra.run.smithi103.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:57:14.760 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0a brd ff:ff:ff:ff:ff:ff 2024-04-26T20:57:14.760 INFO:teuthology.orchestra.run.smithi103.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:57:14.761 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0b brd ff:ff:ff:ff:ff:ff 2024-04-26T20:57:14.761 INFO:teuthology.orchestra.run.smithi103.stdout: inet 172.21.15.103/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T20:57:14.761 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft 42635sec preferred_lft 42635sec 2024-04-26T20:57:14.761 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 fe80::ec4:7aff:fe8f:cc0b/64 scope link 2024-04-26T20:57:14.761 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:14.762 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T20:57:14.762 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:57:14.762 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-04-26T20:57:14.762 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add name ceph-brx type bridge 2024-04-26T20:57:14.762 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip addr flush dev ceph-brx 2024-04-26T20:57:14.762 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set ceph-brx up 2024-04-26T20:57:14.763 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T20:57:14.763 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-04-26T20:57:14.870 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:57:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:57:15.222 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:57:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:15.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:15.232 DEBUG:teuthology.orchestra.run.smithi103:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:57:15.298 INFO:teuthology.orchestra.run.smithi103.stdout:1 2024-04-26T20:57:15.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:15.301 DEBUG:teuthology.orchestra.run.smithi103:> route 2024-04-26T20:57:15.364 INFO:teuthology.orchestra.run.smithi103.stdout:Kernel IP routing table 2024-04-26T20:57:15.364 INFO:teuthology.orchestra.run.smithi103.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:57:15.364 INFO:teuthology.orchestra.run.smithi103.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:57:15.365 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:57:15.365 INFO:teuthology.orchestra.run.smithi103.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:57:15.365 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:57:15.365 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-04-26T20:57:15.365 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:57:15.365 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:57:15.365 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:57:15.365 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-04-26T20:57:15.460 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:57:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:57:15.584 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:57:15.592 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:57:16.079 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:57:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:16.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:16.090 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-04-26T20:57:16.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:16.144 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list-id 2024-04-26T20:57:16.197 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:57:16.198 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-04-26T20:57:16.198 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:57:16.198 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-26T20:57:16.198 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-04-26T20:57:16.305 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:57:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:57:16.377 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:57:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:16.387 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-26T20:57:16.388 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:57:16.388 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-04-26T20:57:16.388 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-26T20:57:16.388 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T20:57:16.388 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-26T20:57:16.388 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-26T20:57:16.388 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-26T20:57:16.388 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-04-26T20:57:16.492 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:57:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:57:16.679 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:57:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:16.689 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:57:16.689 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-04-26T20:57:16.689 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set brx.0 up 2024-04-26T20:57:16.689 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T20:57:16.689 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-04-26T20:57:16.784 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:57:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:57:16.882 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:57:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:16.891 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-26T20:57:16.892 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:57:16.892 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T20:57:16.938 INFO:teuthology.orchestra.run.smithi103.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T20:57:16.938 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:57:16.938 DEBUG:teuthology.orchestra.run.smithi103:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T20:57:17.034 DEBUG:teuthology.orchestra.run.smithi103:> sudo modprobe fuse 2024-04-26T20:57:17.111 DEBUG:teuthology.orchestra.run.smithi103:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:57:17.163 INFO:teuthology.orchestra.run.smithi103.stdout:/proc 2024-04-26T20:57:17.163 INFO:teuthology.orchestra.run.smithi103.stdout:/sys 2024-04-26T20:57:17.163 INFO:teuthology.orchestra.run.smithi103.stdout:/dev 2024-04-26T20:57:17.163 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/security 2024-04-26T20:57:17.163 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/shm 2024-04-26T20:57:17.163 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/pts 2024-04-26T20:57:17.163 INFO:teuthology.orchestra.run.smithi103.stdout:/run 2024-04-26T20:57:17.163 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/cgroup 2024-04-26T20:57:17.163 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/pstore 2024-04-26T20:57:17.163 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/bpf 2024-04-26T20:57:17.163 INFO:teuthology.orchestra.run.smithi103.stdout:/ 2024-04-26T20:57:17.163 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/selinux 2024-04-26T20:57:17.164 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:57:17.164 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/hugepages 2024-04-26T20:57:17.164 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/mqueue 2024-04-26T20:57:17.164 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/debug 2024-04-26T20:57:17.164 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/tracing 2024-04-26T20:57:17.164 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/fuse/connections 2024-04-26T20:57:17.164 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/config 2024-04-26T20:57:17.164 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:57:17.164 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:57:17.164 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:57:17.164 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:57:17.164 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/1000 2024-04-26T20:57:17.164 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph 2024-04-26T20:57:17.164 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T20:57:17.164 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T20:57:17.165 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T20:57:17.165 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T20:57:17.165 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns 2024-04-26T20:57:17.165 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:57:17.165 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:57:17.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:17.166 DEBUG:teuthology.orchestra.run.smithi103:> ls /sys/fs/fuse/connections 2024-04-26T20:57:17.213 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T20:57:17.213 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-26T20:57:17.217 DEBUG:teuthology.orchestra.run.smithi103:> sudo modprobe fuse 2024-04-26T20:57:17.285 DEBUG:teuthology.orchestra.run.smithi103:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:57:17.326 INFO:teuthology.orchestra.run.smithi103.stdout:/proc 2024-04-26T20:57:17.326 INFO:teuthology.orchestra.run.smithi103.stdout:/sys 2024-04-26T20:57:17.326 INFO:teuthology.orchestra.run.smithi103.stdout:/dev 2024-04-26T20:57:17.326 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/security 2024-04-26T20:57:17.327 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/shm 2024-04-26T20:57:17.327 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/pts 2024-04-26T20:57:17.327 INFO:teuthology.orchestra.run.smithi103.stdout:/run 2024-04-26T20:57:17.327 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/cgroup 2024-04-26T20:57:17.327 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/pstore 2024-04-26T20:57:17.327 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/bpf 2024-04-26T20:57:17.327 INFO:teuthology.orchestra.run.smithi103.stdout:/ 2024-04-26T20:57:17.327 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/selinux 2024-04-26T20:57:17.327 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:57:17.327 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/hugepages 2024-04-26T20:57:17.327 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/mqueue 2024-04-26T20:57:17.327 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/debug 2024-04-26T20:57:17.327 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/tracing 2024-04-26T20:57:17.327 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/fuse/connections 2024-04-26T20:57:17.327 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/config 2024-04-26T20:57:17.328 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:57:17.328 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:57:17.328 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:57:17.328 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:57:17.328 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/1000 2024-04-26T20:57:17.328 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph 2024-04-26T20:57:17.328 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T20:57:17.328 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T20:57:17.328 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T20:57:17.328 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T20:57:17.328 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns 2024-04-26T20:57:17.328 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:57:17.328 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:57:17.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:17.329 DEBUG:teuthology.orchestra.run.smithi103:> ls /sys/fs/fuse/connections 2024-04-26T20:57:17.394 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi103.stderr:2024-04-26T20:57:17.393+0000 7fcae3179480 -1 init, newargv = 0x55ff1d9dbca0 newargc=15 2024-04-26T20:57:17.394 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi103.stderr:ceph-fuse[38812]: starting ceph client 2024-04-26T20:57:17.406 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi103.stderr:ceph-fuse[38812]: starting fuse 2024-04-26T20:57:17.413 INFO:teuthology.orchestra.run.smithi103.stdout:42 2024-04-26T20:57:17.413 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-26T20:57:17.413 DEBUG:teuthology.orchestra.run.smithi103:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:57:17.414 DEBUG:teuthology.orchestra.run.smithi103:> import glob 2024-04-26T20:57:17.414 DEBUG:teuthology.orchestra.run.smithi103:> import re 2024-04-26T20:57:17.414 DEBUG:teuthology.orchestra.run.smithi103:> import os 2024-04-26T20:57:17.414 DEBUG:teuthology.orchestra.run.smithi103:> import subprocess 2024-04-26T20:57:17.414 DEBUG:teuthology.orchestra.run.smithi103:> 2024-04-26T20:57:17.414 DEBUG:teuthology.orchestra.run.smithi103:> def _find_admin_socket(client_name): 2024-04-26T20:57:17.414 DEBUG:teuthology.orchestra.run.smithi103:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-26T20:57:17.414 DEBUG:teuthology.orchestra.run.smithi103:> files = glob.glob(asok_path) 2024-04-26T20:57:17.414 DEBUG:teuthology.orchestra.run.smithi103:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-26T20:57:17.414 DEBUG:teuthology.orchestra.run.smithi103:> 2024-04-26T20:57:17.414 DEBUG:teuthology.orchestra.run.smithi103:> # Given a non-glob path, it better be there 2024-04-26T20:57:17.414 DEBUG:teuthology.orchestra.run.smithi103:> if "*" not in asok_path: 2024-04-26T20:57:17.414 DEBUG:teuthology.orchestra.run.smithi103:> assert(len(files) == 1) 2024-04-26T20:57:17.414 DEBUG:teuthology.orchestra.run.smithi103:> return files[0] 2024-04-26T20:57:17.414 DEBUG:teuthology.orchestra.run.smithi103:> 2024-04-26T20:57:17.415 DEBUG:teuthology.orchestra.run.smithi103:> for f in files: 2024-04-26T20:57:17.415 DEBUG:teuthology.orchestra.run.smithi103:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:57:17.415 DEBUG:teuthology.orchestra.run.smithi103:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:57:17.415 DEBUG:teuthology.orchestra.run.smithi103:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:57:17.415 DEBUG:teuthology.orchestra.run.smithi103:> contents = proc_f.read() 2024-04-26T20:57:17.415 DEBUG:teuthology.orchestra.run.smithi103:> if mountpoint in contents: 2024-04-26T20:57:17.415 DEBUG:teuthology.orchestra.run.smithi103:> return f 2024-04-26T20:57:17.415 DEBUG:teuthology.orchestra.run.smithi103:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:57:17.415 DEBUG:teuthology.orchestra.run.smithi103:> 2024-04-26T20:57:17.415 DEBUG:teuthology.orchestra.run.smithi103:> print(_find_admin_socket("client.1")) 2024-04-26T20:57:17.415 DEBUG:teuthology.orchestra.run.smithi103:> ' 2024-04-26T20:57:17.605 INFO:teuthology.orchestra.run.smithi103.stdout:/var/run/ceph/ceph-client.1.38812.asok 2024-04-26T20:57:17.609 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:57:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:17.621 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.38812.asok 2024-04-26T20:57:17.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:17.621 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.38812.asok status 2024-04-26T20:57:17.853 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-04-26T20:57:17.853 INFO:teuthology.orchestra.run.smithi103.stdout: "metadata": { 2024-04-26T20:57:17.853 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:57:17.853 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:57:17.853 INFO:teuthology.orchestra.run.smithi103.stdout: "entity_id": "1", 2024-04-26T20:57:17.853 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": "smithi103", 2024-04-26T20:57:17.853 INFO:teuthology.orchestra.run.smithi103.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-26T20:57:17.853 INFO:teuthology.orchestra.run.smithi103.stdout: "pid": "38812", 2024-04-26T20:57:17.853 INFO:teuthology.orchestra.run.smithi103.stdout: "root": "/" 2024-04-26T20:57:17.853 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T20:57:17.853 INFO:teuthology.orchestra.run.smithi103.stdout: "dentry_count": 0, 2024-04-26T20:57:17.854 INFO:teuthology.orchestra.run.smithi103.stdout: "dentry_pinned_count": 0, 2024-04-26T20:57:17.854 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 4454, 2024-04-26T20:57:17.854 INFO:teuthology.orchestra.run.smithi103.stdout: "inst": { 2024-04-26T20:57:17.854 INFO:teuthology.orchestra.run.smithi103.stdout: "name": { 2024-04-26T20:57:17.854 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client", 2024-04-26T20:57:17.854 INFO:teuthology.orchestra.run.smithi103.stdout: "num": 4454 2024-04-26T20:57:17.854 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T20:57:17.854 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": { 2024-04-26T20:57:17.854 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v1", 2024-04-26T20:57:17.854 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:57:17.854 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 81779872 2024-04-26T20:57:17.854 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T20:57:17.854 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T20:57:17.854 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": { 2024-04-26T20:57:17.855 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v1", 2024-04-26T20:57:17.855 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:57:17.855 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 81779872 2024-04-26T20:57:17.855 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T20:57:17.855 INFO:teuthology.orchestra.run.smithi103.stdout: "inst_str": "client.4454 192.168.0.1:0/81779872", 2024-04-26T20:57:17.855 INFO:teuthology.orchestra.run.smithi103.stdout: "addr_str": "192.168.0.1:0/81779872", 2024-04-26T20:57:17.855 INFO:teuthology.orchestra.run.smithi103.stdout: "inode_count": 1, 2024-04-26T20:57:17.855 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_epoch": 8, 2024-04-26T20:57:17.855 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_epoch": 21, 2024-04-26T20:57:17.855 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:57:17.855 INFO:teuthology.orchestra.run.smithi103.stdout: "blocklisted": false, 2024-04-26T20:57:17.855 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs" 2024-04-26T20:57:17.855 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-04-26T20:57:17.867 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:57:18.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.086+0000 7f183f200640 1 Processor -- start 2024-04-26T20:57:18.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.086+0000 7f183f200640 1 -- start start 2024-04-26T20:57:18.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.088+0000 7f183f200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f184015aab0 0x7f18400cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:18.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.088+0000 7f183f200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f18401611f0 0x7f18400724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:18.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.088+0000 7f183f200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f184015ae90 con 0x7f1840073d10 2024-04-26T20:57:18.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.088+0000 7f183f200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f18400cfa40 con 0x7f184015aab0 2024-04-26T20:57:18.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.088+0000 7f183f200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f1840072040 con 0x7f18401611f0 2024-04-26T20:57:18.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.089+0000 7f183de00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f184015aab0 0x7f18400cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:18.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.089+0000 7f183de00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f184015aab0 0x7f18400cd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45430/0 (socket says 172.21.15.28:45430) 2024-04-26T20:57:18.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.089+0000 7f183de00640 1 -- 172.21.15.28:0/3616536145 learned_addr learned my addr 172.21.15.28:0/3616536145 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:18.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.089+0000 7f183e800640 1 --2- 172.21.15.28:0/3616536145 >> v2:172.21.15.103:3300/0 conn(0x7f18401611f0 0x7f18400724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:18.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.089+0000 7f183de00640 1 -- 172.21.15.28:0/3616536145 >> v2:172.21.15.103:3300/0 conn(0x7f18401611f0 msgr2=0x7f18400724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:18.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.089+0000 7f183de00640 1 --2- 172.21.15.28:0/3616536145 >> v2:172.21.15.103:3300/0 conn(0x7f18401611f0 0x7f18400724d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:18.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.089+0000 7f183de00640 1 -- 172.21.15.28:0/3616536145 >> v1:172.21.15.103:6789/0 conn(0x7f1840073d10 legacy=0x7f1840160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:18.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.089+0000 7f183de00640 1 -- 172.21.15.28:0/3616536145 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18400cfc90 con 0x7f184015aab0 2024-04-26T20:57:18.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.089+0000 7f183e800640 1 --2- 172.21.15.28:0/3616536145 >> v2:172.21.15.103:3300/0 conn(0x7f18401611f0 0x7f18400724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:57:18.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.091+0000 7f183de00640 1 --2- 172.21.15.28:0/3616536145 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f184015aab0 0x7f18400cd450 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f184006fc30 tx=0x7f183002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b073d1224581dda9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:18.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.091+0000 7f183d400640 1 -- 172.21.15.28:0/3616536145 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830037070 con 0x7f184015aab0 2024-04-26T20:57:18.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.091+0000 7f183d400640 1 -- 172.21.15.28:0/3616536145 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1830032440 con 0x7f184015aab0 2024-04-26T20:57:18.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.091+0000 7f183d400640 1 -- 172.21.15.28:0/3616536145 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830032810 con 0x7f184015aab0 2024-04-26T20:57:18.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.092+0000 7f183f200640 1 -- 172.21.15.28:0/3616536145 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f184015aab0 msgr2=0x7f18400cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:18.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.092+0000 7f183f200640 1 --2- 172.21.15.28:0/3616536145 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f184015aab0 0x7f18400cd450 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f184006fc30 tx=0x7f183002a1a0 comp rx=0 tx=0).stop 2024-04-26T20:57:18.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.092+0000 7f183f200640 1 -- 172.21.15.28:0/3616536145 shutdown_connections 2024-04-26T20:57:18.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.092+0000 7f183f200640 1 --2- 172.21.15.28:0/3616536145 >> v2:172.21.15.103:3300/0 conn(0x7f18401611f0 0x7f18400724d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:18.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.092+0000 7f183f200640 1 --2- 172.21.15.28:0/3616536145 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f184015aab0 0x7f18400cd450 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:18.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.092+0000 7f183f200640 1 -- 172.21.15.28:0/3616536145 >> 172.21.15.28:0/3616536145 conn(0x7f18400b2720 msgr2=0x7f18400b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:18.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.092+0000 7f183f200640 1 -- 172.21.15.28:0/3616536145 shutdown_connections 2024-04-26T20:57:18.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.092+0000 7f183f200640 1 -- 172.21.15.28:0/3616536145 wait complete. 2024-04-26T20:57:18.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.094+0000 7f183f200640 1 Processor -- start 2024-04-26T20:57:18.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.094+0000 7f183f200640 1 -- start start 2024-04-26T20:57:18.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.095+0000 7f183f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1840073d10 0x7f1840138610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:18.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.095+0000 7f183f200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f184015aab0 0x7f1840158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:18.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.095+0000 7f183f200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f18401611f0 0x7f18400b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:18.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.095+0000 7f183f200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f184006df00 con 0x7f184015aab0 2024-04-26T20:57:18.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.095+0000 7f183f200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1840074180 con 0x7f1840073d10 2024-04-26T20:57:18.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.095+0000 7f183f200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f18400d0250 con 0x7f18401611f0 2024-04-26T20:57:18.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.096+0000 7f1835400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f184015aab0 0x7f1840158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:18.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.096+0000 7f1835400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f184015aab0 0x7f1840158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45438/0 (socket says 172.21.15.28:45438) 2024-04-26T20:57:18.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.096+0000 7f1835400640 1 -- 172.21.15.28:0/3733676431 learned_addr learned my addr 172.21.15.28:0/3733676431 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:18.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.096+0000 7f183de00640 1 --2- 172.21.15.28:0/3733676431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1840073d10 0x7f1840138610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:18.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.096+0000 7f183e800640 1 --2- 172.21.15.28:0/3733676431 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f18401611f0 0x7f18400b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:18.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.096+0000 7f1835400640 1 -- 172.21.15.28:0/3733676431 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f18401611f0 msgr2=0x7f18400b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:18.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.096+0000 7f1835400640 1 --2- 172.21.15.28:0/3733676431 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f18401611f0 0x7f18400b09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:18.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.096+0000 7f1835400640 1 -- 172.21.15.28:0/3733676431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1840073d10 msgr2=0x7f1840138610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:18.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.096+0000 7f1835400640 1 --2- 172.21.15.28:0/3733676431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1840073d10 0x7f1840138610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:18.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.096+0000 7f1835400640 1 -- 172.21.15.28:0/3733676431 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1840159350 con 0x7f184015aab0 2024-04-26T20:57:18.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.097+0000 7f1835400640 1 --2- 172.21.15.28:0/3733676431 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f184015aab0 0x7f1840158e10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f182800b420 tx=0x7f182800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:18.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.097+0000 7f1836a00640 1 -- 172.21.15.28:0/3733676431 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828011830 con 0x7f184015aab0 2024-04-26T20:57:18.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.097+0000 7f1836a00640 1 -- 172.21.15.28:0/3733676431 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1828016070 con 0x7f184015aab0 2024-04-26T20:57:18.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.097+0000 7f183f200640 1 -- 172.21.15.28:0/3733676431 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1840159620 con 0x7f184015aab0 2024-04-26T20:57:18.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.097+0000 7f1836a00640 1 -- 172.21.15.28:0/3733676431 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1828011c10 con 0x7f184015aab0 2024-04-26T20:57:18.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.097+0000 7f183f200640 1 -- 172.21.15.28:0/3733676431 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1840155b90 con 0x7f184015aab0 2024-04-26T20:57:18.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.098+0000 7f183f200640 1 -- 172.21.15.28:0/3733676431 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f184006a360 con 0x7f184015aab0 2024-04-26T20:57:18.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.099+0000 7f1836a00640 1 -- 172.21.15.28:0/3733676431 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f182801b460 con 0x7f184015aab0 2024-04-26T20:57:18.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.100+0000 7f1836a00640 1 --2- 172.21.15.28:0/3733676431 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f181c06c8f0 0x7f181c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:18.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.101+0000 7f1836a00640 1 -- 172.21.15.28:0/3733676431 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f182808b660 con 0x7f184015aab0 2024-04-26T20:57:18.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.101+0000 7f183de00640 1 --2- 172.21.15.28:0/3733676431 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f181c06c8f0 0x7f181c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:18.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.101+0000 7f183de00640 1 --2- 172.21.15.28:0/3733676431 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f181c06c8f0 0x7f181c06edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1830034000 tx=0x7f183002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:18.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.105+0000 7f1836a00640 1 -- 172.21.15.28:0/3733676431 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1828020020 con 0x7f184015aab0 2024-04-26T20:57:18.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.352+0000 7f183f200640 1 -- 172.21.15.28:0/3733676431 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f184006fc80 con 0x7f184015aab0 2024-04-26T20:57:18.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.352+0000 7f1836a00640 1 -- 172.21.15.28:0/3733676431 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f18280599f0 con 0x7f184015aab0 2024-04-26T20:57:18.354 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:57:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.355+0000 7f183f200640 1 -- 172.21.15.28:0/3733676431 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f181c06c8f0 msgr2=0x7f181c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.355+0000 7f183f200640 1 --2- 172.21.15.28:0/3733676431 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f181c06c8f0 0x7f181c06edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1830034000 tx=0x7f183002e000 comp rx=0 tx=0).stop 2024-04-26T20:57:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.355+0000 7f183f200640 1 -- 172.21.15.28:0/3733676431 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f184015aab0 msgr2=0x7f1840158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.355+0000 7f183f200640 1 --2- 172.21.15.28:0/3733676431 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f184015aab0 0x7f1840158e10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f182800b420 tx=0x7f182800b8e0 comp rx=0 tx=0).stop 2024-04-26T20:57:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.355+0000 7f183f200640 1 -- 172.21.15.28:0/3733676431 shutdown_connections 2024-04-26T20:57:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.355+0000 7f183f200640 1 --2- 172.21.15.28:0/3733676431 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f181c06c8f0 0x7f181c06edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:18.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.355+0000 7f183f200640 1 --2- 172.21.15.28:0/3733676431 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f18401611f0 0x7f18400b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:18.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.355+0000 7f183f200640 1 --2- 172.21.15.28:0/3733676431 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f184015aab0 0x7f1840158e10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:18.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.355+0000 7f183f200640 1 --2- 172.21.15.28:0/3733676431 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1840073d10 0x7f1840138610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:18.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.355+0000 7f183f200640 1 -- 172.21.15.28:0/3733676431 >> 172.21.15.28:0/3733676431 conn(0x7f18400b2720 msgr2=0x7f1840072960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:18.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.355+0000 7f183f200640 1 -- 172.21.15.28:0/3733676431 shutdown_connections 2024-04-26T20:57:18.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:18.355+0000 7f183f200640 1 -- 172.21.15.28:0/3733676431 wait complete. 2024-04-26T20:57:18.376 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:57:18.376 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:57:18.376 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi103.front.sepia.ceph.com 2024-04-26T20:57:18.376 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-26T20:57:18.376 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T20:57:18.376 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:57:18.376 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:57:18.376 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:57:18.377 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-26T20:57:18.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:18.377 DEBUG:teuthology.orchestra.run.smithi103:> ip addr 2024-04-26T20:57:18.433 INFO:teuthology.orchestra.run.smithi103.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:57:18.434 INFO:teuthology.orchestra.run.smithi103.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:57:18.434 INFO:teuthology.orchestra.run.smithi103.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:57:18.434 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:18.434 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 ::1/128 scope host 2024-04-26T20:57:18.434 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:18.434 INFO:teuthology.orchestra.run.smithi103.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:57:18.434 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c4 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:57:18.434 INFO:teuthology.orchestra.run.smithi103.stdout: altname enp7s0f0 2024-04-26T20:57:18.434 INFO:teuthology.orchestra.run.smithi103.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:57:18.434 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:d9:b5:c5 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:57:18.434 INFO:teuthology.orchestra.run.smithi103.stdout: altname enp7s0f1 2024-04-26T20:57:18.434 INFO:teuthology.orchestra.run.smithi103.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:57:18.434 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0a brd ff:ff:ff:ff:ff:ff 2024-04-26T20:57:18.435 INFO:teuthology.orchestra.run.smithi103.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:57:18.435 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0c:c4:7a:8f:cc:0b brd ff:ff:ff:ff:ff:ff 2024-04-26T20:57:18.435 INFO:teuthology.orchestra.run.smithi103.stdout: inet 172.21.15.103/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-26T20:57:18.435 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft 42632sec preferred_lft 42632sec 2024-04-26T20:57:18.435 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 fe80::ec4:7aff:fe8f:cc0b/64 scope link 2024-04-26T20:57:18.435 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:18.435 INFO:teuthology.orchestra.run.smithi103.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:57:18.435 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0e:57:a3:77:52:da brd ff:ff:ff:ff:ff:ff 2024-04-26T20:57:18.435 INFO:teuthology.orchestra.run.smithi103.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:57:18.435 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:18.435 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 fe80::fc50:dff:fe79:d6ed/64 scope link 2024-04-26T20:57:18.435 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:18.435 INFO:teuthology.orchestra.run.smithi103.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:57:18.435 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 0e:57:a3:77:52:da brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:57:18.436 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 fe80::c57:a3ff:fe77:52da/64 scope link 2024-04-26T20:57:18.436 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:18.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:18.437 DEBUG:teuthology.orchestra.run.smithi103:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:57:18.519 INFO:teuthology.orchestra.run.smithi103.stdout:1 2024-04-26T20:57:18.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:18.523 DEBUG:teuthology.orchestra.run.smithi103:> route 2024-04-26T20:57:18.575 INFO:teuthology.orchestra.run.smithi103.stdout:Kernel IP routing table 2024-04-26T20:57:18.575 INFO:teuthology.orchestra.run.smithi103.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:57:18.575 INFO:teuthology.orchestra.run.smithi103.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T20:57:18.575 INFO:teuthology.orchestra.run.smithi103.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T20:57:18.575 INFO:teuthology.orchestra.run.smithi103.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:57:18.576 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:57:18.576 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-04-26T20:57:18.576 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T20:57:18.576 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T20:57:18.576 DEBUG:teuthology.orchestra.run.smithi103:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T20:57:18.576 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-04-26T20:57:18.682 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:57:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:57:18.781 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:57:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:18.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:18.790 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-04-26T20:57:18.836 INFO:teuthology.orchestra.run.smithi103.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T20:57:18.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:18.837 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list-id 2024-04-26T20:57:18.928 INFO:teuthology.orchestra.run.smithi103.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-26T20:57:18.928 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:57:18.928 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-04-26T20:57:18.928 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:57:18.928 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-26T20:57:18.928 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-04-26T20:57:19.031 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:57:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:57:19.100 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:57:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:19.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:19.110 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T20:57:19.189 INFO:teuthology.orchestra.run.smithi103.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:57:19.198 INFO:teuthology.orchestra.run.smithi103.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:57:19.198 INFO:teuthology.orchestra.run.smithi103.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:57:19.198 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:19.198 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 ::1/128 scope host 2024-04-26T20:57:19.198 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:19.198 INFO:teuthology.orchestra.run.smithi103.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:57:19.198 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 22:2d:7e:47:1a:1a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T20:57:19.198 INFO:teuthology.orchestra.run.smithi103.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T20:57:19.198 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:19.198 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 fe80::202d:7eff:fe47:1a1a/64 scope link 2024-04-26T20:57:19.199 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:19.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:19.200 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T20:57:19.285 INFO:teuthology.orchestra.run.smithi103.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:57:19.285 INFO:teuthology.orchestra.run.smithi103.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:57:19.285 INFO:teuthology.orchestra.run.smithi103.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:57:19.285 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:19.285 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 ::1/128 scope host 2024-04-26T20:57:19.285 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:19.286 INFO:teuthology.orchestra.run.smithi103.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:57:19.286 INFO:teuthology.orchestra.run.smithi103.stdout: link/ether 22:2d:7e:47:1a:1a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T20:57:19.286 INFO:teuthology.orchestra.run.smithi103.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T20:57:19.286 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:19.286 INFO:teuthology.orchestra.run.smithi103.stdout: inet6 fe80::202d:7eff:fe47:1a1a/64 scope link 2024-04-26T20:57:19.286 INFO:teuthology.orchestra.run.smithi103.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:57:19.289 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-26T20:57:19.290 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:57:19.290 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-04-26T20:57:19.290 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-26T20:57:19.290 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T20:57:19.290 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-26T20:57:19.290 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-26T20:57:19.290 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-26T20:57:19.290 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-04-26T20:57:19.396 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:57:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:57:19.550 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:57:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:19.560 DEBUG:teuthology.orchestra.run.smithi103:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:57:19.560 DEBUG:teuthology.orchestra.run.smithi103:> set -e 2024-04-26T20:57:19.560 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set brx.1 up 2024-04-26T20:57:19.561 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T20:57:19.561 DEBUG:teuthology.orchestra.run.smithi103:> ') 2024-04-26T20:57:19.660 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:57:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:57:19.756 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:57:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:19.766 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-26T20:57:19.766 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:57:19.766 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-26T20:57:19.807 INFO:teuthology.orchestra.run.smithi103.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-26T20:57:19.808 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:57:19.808 DEBUG:teuthology.orchestra.run.smithi103:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-26T20:57:19.902 DEBUG:teuthology.orchestra.run.smithi103:> sudo modprobe fuse 2024-04-26T20:57:19.988 DEBUG:teuthology.orchestra.run.smithi103:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:57:20.045 INFO:teuthology.orchestra.run.smithi103.stdout:/proc 2024-04-26T20:57:20.045 INFO:teuthology.orchestra.run.smithi103.stdout:/sys 2024-04-26T20:57:20.045 INFO:teuthology.orchestra.run.smithi103.stdout:/dev 2024-04-26T20:57:20.045 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/security 2024-04-26T20:57:20.045 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/shm 2024-04-26T20:57:20.045 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/pts 2024-04-26T20:57:20.045 INFO:teuthology.orchestra.run.smithi103.stdout:/run 2024-04-26T20:57:20.045 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/cgroup 2024-04-26T20:57:20.045 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/pstore 2024-04-26T20:57:20.045 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/bpf 2024-04-26T20:57:20.045 INFO:teuthology.orchestra.run.smithi103.stdout:/ 2024-04-26T20:57:20.045 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/selinux 2024-04-26T20:57:20.045 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:57:20.046 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/hugepages 2024-04-26T20:57:20.046 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/mqueue 2024-04-26T20:57:20.046 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/debug 2024-04-26T20:57:20.046 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/tracing 2024-04-26T20:57:20.046 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/fuse/connections 2024-04-26T20:57:20.046 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/config 2024-04-26T20:57:20.046 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:57:20.046 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:57:20.046 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:57:20.046 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:57:20.046 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/1000 2024-04-26T20:57:20.046 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph 2024-04-26T20:57:20.047 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T20:57:20.047 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T20:57:20.047 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T20:57:20.047 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T20:57:20.047 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns 2024-04-26T20:57:20.047 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:57:20.047 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:57:20.047 INFO:teuthology.orchestra.run.smithi103.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-26T20:57:20.047 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:57:20.047 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:57:20.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:20.048 DEBUG:teuthology.orchestra.run.smithi103:> ls /sys/fs/fuse/connections 2024-04-26T20:57:20.099 INFO:teuthology.orchestra.run.smithi103.stdout:42 2024-04-26T20:57:20.099 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-04-26T20:57:20.099 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-26T20:57:20.104 DEBUG:teuthology.orchestra.run.smithi103:> sudo modprobe fuse 2024-04-26T20:57:20.173 DEBUG:teuthology.orchestra.run.smithi103:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:57:20.215 INFO:teuthology.orchestra.run.smithi103.stdout:/proc 2024-04-26T20:57:20.215 INFO:teuthology.orchestra.run.smithi103.stdout:/sys 2024-04-26T20:57:20.215 INFO:teuthology.orchestra.run.smithi103.stdout:/dev 2024-04-26T20:57:20.215 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/security 2024-04-26T20:57:20.215 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/shm 2024-04-26T20:57:20.215 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/pts 2024-04-26T20:57:20.215 INFO:teuthology.orchestra.run.smithi103.stdout:/run 2024-04-26T20:57:20.215 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/cgroup 2024-04-26T20:57:20.216 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/pstore 2024-04-26T20:57:20.216 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/bpf 2024-04-26T20:57:20.216 INFO:teuthology.orchestra.run.smithi103.stdout:/ 2024-04-26T20:57:20.216 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/selinux 2024-04-26T20:57:20.216 INFO:teuthology.orchestra.run.smithi103.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:57:20.216 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/hugepages 2024-04-26T20:57:20.216 INFO:teuthology.orchestra.run.smithi103.stdout:/dev/mqueue 2024-04-26T20:57:20.216 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/debug 2024-04-26T20:57:20.217 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/tracing 2024-04-26T20:57:20.217 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/fs/fuse/connections 2024-04-26T20:57:20.217 INFO:teuthology.orchestra.run.smithi103.stdout:/sys/kernel/config 2024-04-26T20:57:20.217 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:57:20.217 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:57:20.217 INFO:teuthology.orchestra.run.smithi103.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:57:20.217 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:57:20.217 INFO:teuthology.orchestra.run.smithi103.stdout:/run/user/1000 2024-04-26T20:57:20.218 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph 2024-04-26T20:57:20.218 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph/osd/ceph-4 2024-04-26T20:57:20.218 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph/osd/ceph-5 2024-04-26T20:57:20.218 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph/osd/ceph-6 2024-04-26T20:57:20.218 INFO:teuthology.orchestra.run.smithi103.stdout:/var/lib/ceph/osd/ceph-7 2024-04-26T20:57:20.218 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns 2024-04-26T20:57:20.218 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:57:20.218 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T20:57:20.219 INFO:teuthology.orchestra.run.smithi103.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-26T20:57:20.219 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:57:20.219 INFO:teuthology.orchestra.run.smithi103.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T20:57:20.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:20.220 DEBUG:teuthology.orchestra.run.smithi103:> ls /sys/fs/fuse/connections 2024-04-26T20:57:20.273 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi103.stderr:2024-04-26T20:57:20.272+0000 7f0e87a3a480 -1 init, newargv = 0x55b52a4f92c0 newargc=15 2024-04-26T20:57:20.273 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi103.stderr:ceph-fuse[39268]: starting ceph client 2024-04-26T20:57:20.282 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi103.stderr:ceph-fuse[39268]: starting fuse 2024-04-26T20:57:20.303 INFO:teuthology.orchestra.run.smithi103.stdout:42 2024-04-26T20:57:20.303 INFO:teuthology.orchestra.run.smithi103.stdout:43 2024-04-26T20:57:20.303 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-04-26T20:57:20.304 DEBUG:teuthology.orchestra.run.smithi103:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:57:20.304 DEBUG:teuthology.orchestra.run.smithi103:> import glob 2024-04-26T20:57:20.304 DEBUG:teuthology.orchestra.run.smithi103:> import re 2024-04-26T20:57:20.304 DEBUG:teuthology.orchestra.run.smithi103:> import os 2024-04-26T20:57:20.304 DEBUG:teuthology.orchestra.run.smithi103:> import subprocess 2024-04-26T20:57:20.304 DEBUG:teuthology.orchestra.run.smithi103:> 2024-04-26T20:57:20.304 DEBUG:teuthology.orchestra.run.smithi103:> def _find_admin_socket(client_name): 2024-04-26T20:57:20.304 DEBUG:teuthology.orchestra.run.smithi103:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-26T20:57:20.304 DEBUG:teuthology.orchestra.run.smithi103:> files = glob.glob(asok_path) 2024-04-26T20:57:20.304 DEBUG:teuthology.orchestra.run.smithi103:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-26T20:57:20.304 DEBUG:teuthology.orchestra.run.smithi103:> 2024-04-26T20:57:20.304 DEBUG:teuthology.orchestra.run.smithi103:> # Given a non-glob path, it better be there 2024-04-26T20:57:20.304 DEBUG:teuthology.orchestra.run.smithi103:> if "*" not in asok_path: 2024-04-26T20:57:20.304 DEBUG:teuthology.orchestra.run.smithi103:> assert(len(files) == 1) 2024-04-26T20:57:20.304 DEBUG:teuthology.orchestra.run.smithi103:> return files[0] 2024-04-26T20:57:20.305 DEBUG:teuthology.orchestra.run.smithi103:> 2024-04-26T20:57:20.305 DEBUG:teuthology.orchestra.run.smithi103:> for f in files: 2024-04-26T20:57:20.305 DEBUG:teuthology.orchestra.run.smithi103:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:57:20.305 DEBUG:teuthology.orchestra.run.smithi103:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:57:20.305 DEBUG:teuthology.orchestra.run.smithi103:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:57:20.305 DEBUG:teuthology.orchestra.run.smithi103:> contents = proc_f.read() 2024-04-26T20:57:20.305 DEBUG:teuthology.orchestra.run.smithi103:> if mountpoint in contents: 2024-04-26T20:57:20.305 DEBUG:teuthology.orchestra.run.smithi103:> return f 2024-04-26T20:57:20.305 DEBUG:teuthology.orchestra.run.smithi103:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:57:20.305 DEBUG:teuthology.orchestra.run.smithi103:> 2024-04-26T20:57:20.305 DEBUG:teuthology.orchestra.run.smithi103:> print(_find_admin_socket("client.3")) 2024-04-26T20:57:20.305 DEBUG:teuthology.orchestra.run.smithi103:> ' 2024-04-26T20:57:20.501 INFO:teuthology.orchestra.run.smithi103.stdout:/var/run/ceph/ceph-client.3.39268.asok 2024-04-26T20:57:20.505 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T20:57:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:57:20.518 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.39268.asok 2024-04-26T20:57:20.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:20.518 DEBUG:teuthology.orchestra.run.smithi103:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.39268.asok status 2024-04-26T20:57:20.746 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-04-26T20:57:20.746 INFO:teuthology.orchestra.run.smithi103.stdout: "metadata": { 2024-04-26T20:57:20.746 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:57:20.746 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:57:20.746 INFO:teuthology.orchestra.run.smithi103.stdout: "entity_id": "3", 2024-04-26T20:57:20.746 INFO:teuthology.orchestra.run.smithi103.stdout: "hostname": "smithi103", 2024-04-26T20:57:20.746 INFO:teuthology.orchestra.run.smithi103.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-26T20:57:20.746 INFO:teuthology.orchestra.run.smithi103.stdout: "pid": "39268", 2024-04-26T20:57:20.747 INFO:teuthology.orchestra.run.smithi103.stdout: "root": "/" 2024-04-26T20:57:20.747 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T20:57:20.747 INFO:teuthology.orchestra.run.smithi103.stdout: "dentry_count": 0, 2024-04-26T20:57:20.747 INFO:teuthology.orchestra.run.smithi103.stdout: "dentry_pinned_count": 0, 2024-04-26T20:57:20.747 INFO:teuthology.orchestra.run.smithi103.stdout: "id": 4457, 2024-04-26T20:57:20.747 INFO:teuthology.orchestra.run.smithi103.stdout: "inst": { 2024-04-26T20:57:20.747 INFO:teuthology.orchestra.run.smithi103.stdout: "name": { 2024-04-26T20:57:20.747 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "client", 2024-04-26T20:57:20.747 INFO:teuthology.orchestra.run.smithi103.stdout: "num": 4457 2024-04-26T20:57:20.747 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T20:57:20.747 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": { 2024-04-26T20:57:20.747 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v1", 2024-04-26T20:57:20.747 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "192.168.0.2:0", 2024-04-26T20:57:20.747 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 1956586705 2024-04-26T20:57:20.747 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T20:57:20.748 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T20:57:20.748 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": { 2024-04-26T20:57:20.748 INFO:teuthology.orchestra.run.smithi103.stdout: "type": "v1", 2024-04-26T20:57:20.748 INFO:teuthology.orchestra.run.smithi103.stdout: "addr": "192.168.0.2:0", 2024-04-26T20:57:20.748 INFO:teuthology.orchestra.run.smithi103.stdout: "nonce": 1956586705 2024-04-26T20:57:20.748 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T20:57:20.748 INFO:teuthology.orchestra.run.smithi103.stdout: "inst_str": "client.4457 192.168.0.2:0/1956586705", 2024-04-26T20:57:20.748 INFO:teuthology.orchestra.run.smithi103.stdout: "addr_str": "192.168.0.2:0/1956586705", 2024-04-26T20:57:20.748 INFO:teuthology.orchestra.run.smithi103.stdout: "inode_count": 1, 2024-04-26T20:57:20.748 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_epoch": 8, 2024-04-26T20:57:20.748 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_epoch": 21, 2024-04-26T20:57:20.748 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:57:20.748 INFO:teuthology.orchestra.run.smithi103.stdout: "blocklisted": false, 2024-04-26T20:57:20.748 INFO:teuthology.orchestra.run.smithi103.stdout: "fs_name": "cephfs" 2024-04-26T20:57:20.749 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-04-26T20:57:20.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:20.761 DEBUG:teuthology.orchestra.run.smithi028:> stat --file-system '--printf=%T 2024-04-26T20:57:20.761 DEBUG:teuthology.orchestra.run.smithi028:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-26T20:57:20.822 INFO:teuthology.orchestra.run.smithi028.stdout:fuseblk 2024-04-26T20:57:20.823 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-26T20:57:20.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:20.823 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:57:20.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:20.921 DEBUG:teuthology.orchestra.run.smithi028:> stat --file-system '--printf=%T 2024-04-26T20:57:20.921 DEBUG:teuthology.orchestra.run.smithi028:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-26T20:57:20.980 INFO:teuthology.orchestra.run.smithi028.stdout:fuseblk 2024-04-26T20:57:20.981 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-26T20:57:20.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:20.982 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-26T20:57:21.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:21.073 DEBUG:teuthology.orchestra.run.smithi103:> stat --file-system '--printf=%T 2024-04-26T20:57:21.073 DEBUG:teuthology.orchestra.run.smithi103:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-26T20:57:21.132 INFO:teuthology.orchestra.run.smithi103.stdout:fuseblk 2024-04-26T20:57:21.133 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-26T20:57:21.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:21.133 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T20:57:21.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:21.217 DEBUG:teuthology.orchestra.run.smithi103:> stat --file-system '--printf=%T 2024-04-26T20:57:21.217 DEBUG:teuthology.orchestra.run.smithi103:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-26T20:57:21.269 INFO:teuthology.orchestra.run.smithi103.stdout:fuseblk 2024-04-26T20:57:21.270 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-26T20:57:21.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:21.270 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-26T20:57:21.351 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-26T20:57:21.368 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:57:21.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.589+0000 7f7d84a00640 1 Processor -- start 2024-04-26T20:57:21.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.590+0000 7f7d84a00640 1 -- start start 2024-04-26T20:57:21.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.591+0000 7f7d84a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7d80146060 0x7f7d80071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:21.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.591+0000 7f7d84a00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f7d801610c0 0x7f7d800cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:21.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.591+0000 7f7d84a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d80146440 con 0x7f7d80146670 2024-04-26T20:57:21.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.591+0000 7f7d84a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f7d8006f510 con 0x7f7d80146060 2024-04-26T20:57:21.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.591+0000 7f7d84a00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f7d8006f010 con 0x7f7d801610c0 2024-04-26T20:57:21.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.592+0000 7f7d7f400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7d80146060 0x7f7d80071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:21.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.592+0000 7f7d7ea00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7d80146670 0x7f7d801609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33028/0 (socket says 172.21.15.28:33028) 2024-04-26T20:57:21.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.592+0000 7f7d7ea00640 1 -- 172.21.15.28:0/3558258177 learned_addr learned my addr 172.21.15.28:0/3558258177 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:21.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.592+0000 7f7d7fe00640 1 --2- 172.21.15.28:0/3558258177 >> v2:172.21.15.103:3300/0 conn(0x7f7d801610c0 0x7f7d800cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:21.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.592+0000 7f7d7f400640 1 -- 172.21.15.28:0/3558258177 >> v2:172.21.15.103:3300/0 conn(0x7f7d801610c0 msgr2=0x7f7d800cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:21.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.592+0000 7f7d7f400640 1 --2- 172.21.15.28:0/3558258177 >> v2:172.21.15.103:3300/0 conn(0x7f7d801610c0 0x7f7d800cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:21.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.592+0000 7f7d7f400640 1 -- 172.21.15.28:0/3558258177 >> v1:172.21.15.28:6789/0 conn(0x7f7d80146670 legacy=0x7f7d801609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:21.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.592+0000 7f7d7f400640 1 -- 172.21.15.28:0/3558258177 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d801614d0 con 0x7f7d80146060 2024-04-26T20:57:21.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.592+0000 7f7d7e000640 1 -- 172.21.15.28:0/3558258177 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3483013830 0 0) 0x7f7d80146440 con 0x7f7d80146670 2024-04-26T20:57:21.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.593+0000 7f7d7f400640 1 --2- 172.21.15.28:0/3558258177 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d80146060 0x7f7d80071e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7d80072360 tx=0x7f7d7002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce57175236406e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:21.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.594+0000 7f7d7e000640 1 -- 172.21.15.28:0/3558258177 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70007740 con 0x7f7d80146060 2024-04-26T20:57:21.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.594+0000 7f7d7e000640 1 -- 172.21.15.28:0/3558258177 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d700078e0 con 0x7f7d80146060 2024-04-26T20:57:21.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.594+0000 7f7d7e000640 1 -- 172.21.15.28:0/3558258177 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70007cb0 con 0x7f7d80146060 2024-04-26T20:57:21.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.594+0000 7f7d84a00640 1 -- 172.21.15.28:0/3558258177 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d80146060 msgr2=0x7f7d80071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:21.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.594+0000 7f7d84a00640 1 --2- 172.21.15.28:0/3558258177 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d80146060 0x7f7d80071e20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7d80072360 tx=0x7f7d7002a5f0 comp rx=0 tx=0).stop 2024-04-26T20:57:21.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.594+0000 7f7d84a00640 1 -- 172.21.15.28:0/3558258177 shutdown_connections 2024-04-26T20:57:21.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.594+0000 7f7d84a00640 1 --2- 172.21.15.28:0/3558258177 >> v2:172.21.15.103:3300/0 conn(0x7f7d801610c0 0x7f7d800cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:21.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.594+0000 7f7d84a00640 1 --2- 172.21.15.28:0/3558258177 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d80146060 0x7f7d80071e20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:21.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.594+0000 7f7d84a00640 1 -- 172.21.15.28:0/3558258177 >> 172.21.15.28:0/3558258177 conn(0x7f7d800d0a50 msgr2=0x7f7d800d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:21.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.594+0000 7f7d84a00640 1 -- 172.21.15.28:0/3558258177 shutdown_connections 2024-04-26T20:57:21.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.594+0000 7f7d84a00640 1 -- 172.21.15.28:0/3558258177 wait complete. 2024-04-26T20:57:21.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.596+0000 7f7d84a00640 1 Processor -- start 2024-04-26T20:57:21.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.596+0000 7f7d84a00640 1 -- start start 2024-04-26T20:57:21.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.597+0000 7f7d84a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7d80146060 0x7f7d8015adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:21.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.597+0000 7f7d84a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d80146670 0x7f7d8013fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:21.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.597+0000 7f7d84a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d801610c0 0x7f7d80157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:21.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.597+0000 7f7d84a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8006ff60 con 0x7f7d801610c0 2024-04-26T20:57:21.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.597+0000 7f7d84a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8006fc80 con 0x7f7d80146670 2024-04-26T20:57:21.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.597+0000 7f7d84a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7d800cb5e0 con 0x7f7d80146060 2024-04-26T20:57:21.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.598+0000 7f7d7fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d801610c0 0x7f7d80157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:21.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.598+0000 7f7d7fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d801610c0 0x7f7d80157e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42228/0 (socket says 172.21.15.28:42228) 2024-04-26T20:57:21.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.598+0000 7f7d7fe00640 1 -- 172.21.15.28:0/1565823417 learned_addr learned my addr 172.21.15.28:0/1565823417 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.598+0000 7f7d7ea00640 1 --2- 172.21.15.28:0/1565823417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d80146670 0x7f7d8013fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.598+0000 7f7d7f400640 1 --2- 172.21.15.28:0/1565823417 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7d80146060 0x7f7d8015adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.598+0000 7f7d7fe00640 1 -- 172.21.15.28:0/1565823417 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7d80146060 msgr2=0x7f7d8015adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.598+0000 7f7d7fe00640 1 --2- 172.21.15.28:0/1565823417 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7d80146060 0x7f7d8015adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.598+0000 7f7d7fe00640 1 -- 172.21.15.28:0/1565823417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d80146670 msgr2=0x7f7d8013fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.598+0000 7f7d7fe00640 1 --2- 172.21.15.28:0/1565823417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d80146670 0x7f7d8013fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.598+0000 7f7d7fe00640 1 -- 172.21.15.28:0/1565823417 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8015e260 con 0x7f7d801610c0 2024-04-26T20:57:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.598+0000 7f7d7ea00640 1 --2- 172.21.15.28:0/1565823417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d80146670 0x7f7d8013fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.599+0000 7f7d7fe00640 1 --2- 172.21.15.28:0/1565823417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d801610c0 0x7f7d80157e70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800d5d0 tx=0x7f7d7800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.599+0000 7f7d77400640 1 -- 172.21.15.28:0/1565823417 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d780125f0 con 0x7f7d801610c0 2024-04-26T20:57:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.599+0000 7f7d77400640 1 -- 172.21.15.28:0/1565823417 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d78019070 con 0x7f7d801610c0 2024-04-26T20:57:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.599+0000 7f7d77400640 1 -- 172.21.15.28:0/1565823417 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d780129f0 con 0x7f7d801610c0 2024-04-26T20:57:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.599+0000 7f7d84a00640 1 -- 172.21.15.28:0/1565823417 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8015e540 con 0x7f7d801610c0 2024-04-26T20:57:21.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.599+0000 7f7d84a00640 1 -- 172.21.15.28:0/1565823417 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d801583b0 con 0x7f7d801610c0 2024-04-26T20:57:21.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.600+0000 7f7d84a00640 1 -- 172.21.15.28:0/1565823417 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d8015a420 con 0x7f7d801610c0 2024-04-26T20:57:21.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.601+0000 7f7d77400640 1 -- 172.21.15.28:0/1565823417 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d78012b90 con 0x7f7d801610c0 2024-04-26T20:57:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.602+0000 7f7d77400640 1 --2- 172.21.15.28:0/1565823417 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7d5c06c8f0 0x7f7d5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:21.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.602+0000 7f7d77400640 1 -- 172.21.15.28:0/1565823417 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7d7808bfc0 con 0x7f7d801610c0 2024-04-26T20:57:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.603+0000 7f7d7f400640 1 --2- 172.21.15.28:0/1565823417 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7d5c06c8f0 0x7f7d5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:21.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.603+0000 7f7d7f400640 1 --2- 172.21.15.28:0/1565823417 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7d5c06c8f0 0x7f7d5c06edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d70034000 tx=0x7f7d7002f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:21.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.607+0000 7f7d77400640 1 -- 172.21.15.28:0/1565823417 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7d7805a360 con 0x7f7d801610c0 2024-04-26T20:57:21.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.815+0000 7f7d84a00640 1 -- 172.21.15.28:0/1565823417 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d8006a360 con 0x7f7d801610c0 2024-04-26T20:57:21.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.818+0000 7f7d77400640 1 -- 172.21.15.28:0/1565823417 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7f7d78059c60 con 0x7f7d801610c0 2024-04-26T20:57:21.820 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:21.820 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":21,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:56:52.699663+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:57:21.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.821+0000 7f7d84a00640 1 -- 172.21.15.28:0/1565823417 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7d5c06c8f0 msgr2=0x7f7d5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:21.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.821+0000 7f7d84a00640 1 --2- 172.21.15.28:0/1565823417 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7d5c06c8f0 0x7f7d5c06edb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d70034000 tx=0x7f7d7002f000 comp rx=0 tx=0).stop 2024-04-26T20:57:21.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.821+0000 7f7d84a00640 1 -- 172.21.15.28:0/1565823417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d801610c0 msgr2=0x7f7d80157e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:21.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.821+0000 7f7d84a00640 1 --2- 172.21.15.28:0/1565823417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d801610c0 0x7f7d80157e70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800d5d0 tx=0x7f7d7800da90 comp rx=0 tx=0).stop 2024-04-26T20:57:21.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.821+0000 7f7d84a00640 1 -- 172.21.15.28:0/1565823417 shutdown_connections 2024-04-26T20:57:21.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.821+0000 7f7d84a00640 1 --2- 172.21.15.28:0/1565823417 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7d5c06c8f0 0x7f7d5c06edb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:21.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.821+0000 7f7d84a00640 1 --2- 172.21.15.28:0/1565823417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d801610c0 0x7f7d80157e70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:21.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.821+0000 7f7d84a00640 1 --2- 172.21.15.28:0/1565823417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7d80146670 0x7f7d8013fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:21.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.821+0000 7f7d84a00640 1 --2- 172.21.15.28:0/1565823417 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7d80146060 0x7f7d8015adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:21.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.821+0000 7f7d84a00640 1 -- 172.21.15.28:0/1565823417 >> 172.21.15.28:0/1565823417 conn(0x7f7d800d0a50 msgr2=0x7f7d800cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:21.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.821+0000 7f7d84a00640 1 -- 172.21.15.28:0/1565823417 shutdown_connections 2024-04-26T20:57:21.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:21.821+0000 7f7d84a00640 1 -- 172.21.15.28:0/1565823417 wait complete. 2024-04-26T20:57:21.842 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:56:48.688498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:56:50.117541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:57:21.843 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:57:22.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.047+0000 7fe23f000640 1 Processor -- start 2024-04-26T20:57:22.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.047+0000 7fe23f000640 1 -- start start 2024-04-26T20:57:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.049+0000 7fe23f000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe23806fc10 0x7fe238073f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.049+0000 7fe23f000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe23815e310 con 0x7fe2380ceaa0 2024-04-26T20:57:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.049+0000 7fe23f000640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe238072aa0 con 0x7fe238073580 2024-04-26T20:57:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.049+0000 7fe23f000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe238074490 con 0x7fe23806fc10 2024-04-26T20:57:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.050+0000 7fe23d000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe2380ceaa0 0x7fe23806e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33034/0 (socket says 172.21.15.28:33034) 2024-04-26T20:57:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.050+0000 7fe23d000640 1 -- 172.21.15.28:0/2241592594 learned_addr learned my addr 172.21.15.28:0/2241592594 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.050+0000 7fe237e00640 1 --2- 172.21.15.28:0/2241592594 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe23806fc10 0x7fe238073f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.050+0000 7fe236a00640 1 -- 172.21.15.28:0/2241592594 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 999346062 0 0) 0x7fe23815e310 con 0x7fe2380ceaa0 2024-04-26T20:57:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.050+0000 7fe236a00640 1 -- 172.21.15.28:0/2241592594 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe21c003520 con 0x7fe2380ceaa0 2024-04-26T20:57:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.051+0000 7fe236a00640 1 -- 172.21.15.28:0/2241592594 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3416440461 0 0) 0x7fe21c003520 con 0x7fe2380ceaa0 2024-04-26T20:57:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.051+0000 7fe236a00640 1 -- 172.21.15.28:0/2241592594 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe23806fc10 msgr2=0x7fe238073f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.051+0000 7fe236a00640 1 --2- 172.21.15.28:0/2241592594 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe23806fc10 0x7fe238073f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.051+0000 7fe236a00640 1 -- 172.21.15.28:0/2241592594 >> v1:172.21.15.103:6789/0 conn(0x7fe238073580 legacy=0x7fe2380ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:22.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.051+0000 7fe236a00640 1 -- 172.21.15.28:0/2241592594 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe238072d80 con 0x7fe2380ceaa0 2024-04-26T20:57:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.051+0000 7fe236a00640 1 -- 172.21.15.28:0/2241592594 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fe2300019f0 con 0x7fe2380ceaa0 2024-04-26T20:57:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.051+0000 7fe236a00640 1 -- 172.21.15.28:0/2241592594 >> v1:172.21.15.28:6789/0 conn(0x7fe2380ceaa0 legacy=0x7fe23806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.051+0000 7fe236a00640 1 --2- 172.21.15.28:0/2241592594 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe21c0039e0 0x7fe21c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.051+0000 7fe236a00640 1 --2- 172.21.15.28:0/2241592594 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe23806fc10 0x7fe21c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.051+0000 7fe236a00640 1 --2- 172.21.15.28:0/2241592594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe21c004d40 0x7fe21c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.051+0000 7fe236a00640 1 -- 172.21.15.28:0/2241592594 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c00ba70 con 0x7fe21c0039e0 2024-04-26T20:57:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.051+0000 7fe236a00640 1 -- 172.21.15.28:0/2241592594 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe21c00bbf0 con 0x7fe21c004d40 2024-04-26T20:57:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.051+0000 7fe236a00640 1 -- 172.21.15.28:0/2241592594 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe21c00bdf0 con 0x7fe23806fc10 2024-04-26T20:57:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.051+0000 7fe237e00640 1 --2- 172.21.15.28:0/2241592594 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe21c0039e0 0x7fe21c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.051+0000 7fe237e00640 1 -- 172.21.15.28:0/2241592594 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe23806fc10 msgr2=0x7fe21c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:22.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.051+0000 7fe237e00640 1 --2- 172.21.15.28:0/2241592594 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe23806fc10 0x7fe21c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.051+0000 7fe23d000640 1 --2- 172.21.15.28:0/2241592594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe21c004d40 0x7fe21c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.051+0000 7fe237e00640 1 -- 172.21.15.28:0/2241592594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe21c004d40 msgr2=0x7fe21c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.052+0000 7fe237e00640 1 --2- 172.21.15.28:0/2241592594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe21c004d40 0x7fe21c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.052+0000 7fe237e00640 1 -- 172.21.15.28:0/2241592594 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe21c003520 con 0x7fe21c0039e0 2024-04-26T20:57:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.052+0000 7fe23d000640 1 --2- 172.21.15.28:0/2241592594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe21c004d40 0x7fe21c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.053+0000 7fe237e00640 1 --2- 172.21.15.28:0/2241592594 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe21c0039e0 0x7fe21c003dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe228009ac0 tx=0x7fe21c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36c852765857fbf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.053+0000 7fe236a00640 1 -- 172.21.15.28:0/2241592594 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228036070 con 0x7fe21c0039e0 2024-04-26T20:57:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.053+0000 7fe236a00640 1 -- 172.21.15.28:0/2241592594 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe228031400 con 0x7fe21c0039e0 2024-04-26T20:57:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.053+0000 7fe236a00640 1 -- 172.21.15.28:0/2241592594 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228009690 con 0x7fe21c0039e0 2024-04-26T20:57:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.053+0000 7fe23f000640 1 -- 172.21.15.28:0/2241592594 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe21c0039e0 msgr2=0x7fe21c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.053+0000 7fe23f000640 1 --2- 172.21.15.28:0/2241592594 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe21c0039e0 0x7fe21c003dc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe228009ac0 tx=0x7fe21c005830 comp rx=0 tx=0).stop 2024-04-26T20:57:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.053+0000 7fe23d000640 1 -- 172.21.15.28:0/2241592594 reap_dead start 2024-04-26T20:57:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.054+0000 7fe23f000640 1 -- 172.21.15.28:0/2241592594 shutdown_connections 2024-04-26T20:57:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.054+0000 7fe23f000640 1 -- 172.21.15.28:0/2241592594 >> 172.21.15.28:0/2241592594 conn(0x7fe23806c770 msgr2=0x7fe23806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.054+0000 7fe23f000640 1 -- 172.21.15.28:0/2241592594 shutdown_connections 2024-04-26T20:57:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.054+0000 7fe23f000640 1 -- 172.21.15.28:0/2241592594 wait complete. 2024-04-26T20:57:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.055+0000 7fe23f000640 1 Processor -- start 2024-04-26T20:57:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.056+0000 7fe23f000640 1 -- start start 2024-04-26T20:57:22.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.056+0000 7fe23f000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe21c0039e0 0x7fe2380ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.056+0000 7fe23f000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe2381566a0 0x7fe2381f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.056+0000 7fe23f000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2381f8120 0x7fe2381f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.056+0000 7fe23f000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe238073a00 con 0x7fe21c0039e0 2024-04-26T20:57:22.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.056+0000 7fe23f000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe23815dfa0 con 0x7fe2381f8120 2024-04-26T20:57:22.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.056+0000 7fe23f000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe238073b80 con 0x7fe2381566a0 2024-04-26T20:57:22.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.057+0000 7fe237e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe21c0039e0 0x7fe2380ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:22.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.057+0000 7fe237e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe21c0039e0 0x7fe2380ae240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42244/0 (socket says 172.21.15.28:42244) 2024-04-26T20:57:22.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.057+0000 7fe237e00640 1 -- 172.21.15.28:0/893952369 learned_addr learned my addr 172.21.15.28:0/893952369 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:22.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.057+0000 7fe237e00640 1 -- 172.21.15.28:0/893952369 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe2381566a0 msgr2=0x7fe2381f4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:22.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.057+0000 7fe237e00640 1 --2- 172.21.15.28:0/893952369 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe2381566a0 0x7fe2381f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.057+0000 7fe237e00640 1 -- 172.21.15.28:0/893952369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2381f8120 msgr2=0x7fe2381f85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:22.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.057+0000 7fe237e00640 1 --2- 172.21.15.28:0/893952369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2381f8120 0x7fe2381f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.057+0000 7fe237e00640 1 -- 172.21.15.28:0/893952369 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2380ab5a0 con 0x7fe21c0039e0 2024-04-26T20:57:22.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.058+0000 7fe237e00640 1 --2- 172.21.15.28:0/893952369 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe21c0039e0 0x7fe2380ae240 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe228034040 tx=0x7fe228004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:22.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.058+0000 7fe22fe00640 1 -- 172.21.15.28:0/893952369 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228040020 con 0x7fe21c0039e0 2024-04-26T20:57:22.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.058+0000 7fe22fe00640 1 -- 172.21.15.28:0/893952369 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe228003bc0 con 0x7fe21c0039e0 2024-04-26T20:57:22.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.058+0000 7fe22fe00640 1 -- 172.21.15.28:0/893952369 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228036040 con 0x7fe21c0039e0 2024-04-26T20:57:22.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.058+0000 7fe23f000640 1 -- 172.21.15.28:0/893952369 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2380ab860 con 0x7fe21c0039e0 2024-04-26T20:57:22.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.058+0000 7fe23f000640 1 -- 172.21.15.28:0/893952369 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2380abe20 con 0x7fe21c0039e0 2024-04-26T20:57:22.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.059+0000 7fe23f000640 1 -- 172.21.15.28:0/893952369 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2380d07d0 con 0x7fe21c0039e0 2024-04-26T20:57:22.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.060+0000 7fe22fe00640 1 -- 172.21.15.28:0/893952369 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe228043020 con 0x7fe21c0039e0 2024-04-26T20:57:22.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.061+0000 7fe22fe00640 1 --2- 172.21.15.28:0/893952369 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe21806c940 0x7fe21806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.061+0000 7fe22fe00640 1 -- 172.21.15.28:0/893952369 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe2280abb60 con 0x7fe21c0039e0 2024-04-26T20:57:22.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.062+0000 7fe237400640 1 --2- 172.21.15.28:0/893952369 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe21806c940 0x7fe21806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:22.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.062+0000 7fe237400640 1 --2- 172.21.15.28:0/893952369 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe21806c940 0x7fe21806ee00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe2380ad240 tx=0x7fe224007950 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:22.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.066+0000 7fe22fe00640 1 -- 172.21.15.28:0/893952369 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe2380d07d0 con 0x7fe21c0039e0 2024-04-26T20:57:22.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.268+0000 7fe23f000640 1 -- 172.21.15.28:0/893952369 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe23806a360 con 0x7fe21c0039e0 2024-04-26T20:57:22.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.269+0000 7fe22fe00640 1 -- 172.21.15.28:0/893952369 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fe22804a070 con 0x7fe21c0039e0 2024-04-26T20:57:22.270 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:57:22.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.271+0000 7fe23f000640 1 -- 172.21.15.28:0/893952369 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe21806c940 msgr2=0x7fe21806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:22.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.271+0000 7fe23f000640 1 --2- 172.21.15.28:0/893952369 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe21806c940 0x7fe21806ee00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe2380ad240 tx=0x7fe224007950 comp rx=0 tx=0).stop 2024-04-26T20:57:22.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.271+0000 7fe23f000640 1 -- 172.21.15.28:0/893952369 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe21c0039e0 msgr2=0x7fe2380ae240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:22.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.272+0000 7fe23f000640 1 --2- 172.21.15.28:0/893952369 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe21c0039e0 0x7fe2380ae240 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe228034040 tx=0x7fe228004730 comp rx=0 tx=0).stop 2024-04-26T20:57:22.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.272+0000 7fe23f000640 1 -- 172.21.15.28:0/893952369 shutdown_connections 2024-04-26T20:57:22.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.272+0000 7fe23f000640 1 --2- 172.21.15.28:0/893952369 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe21806c940 0x7fe21806ee00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.272+0000 7fe23f000640 1 --2- 172.21.15.28:0/893952369 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2381f8120 0x7fe2381f85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.272+0000 7fe23f000640 1 --2- 172.21.15.28:0/893952369 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe2381566a0 0x7fe2381f4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.272+0000 7fe23f000640 1 --2- 172.21.15.28:0/893952369 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe21c0039e0 0x7fe2380ae240 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.272+0000 7fe23f000640 1 -- 172.21.15.28:0/893952369 >> 172.21.15.28:0/893952369 conn(0x7fe23806c770 msgr2=0x7fe23806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:22.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.272+0000 7fe23f000640 1 -- 172.21.15.28:0/893952369 shutdown_connections 2024-04-26T20:57:22.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.272+0000 7fe23f000640 1 -- 172.21.15.28:0/893952369 wait complete. 2024-04-26T20:57:22.292 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:57:22.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.495+0000 7fce13400640 1 Processor -- start 2024-04-26T20:57:22.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.496+0000 7fce13400640 1 -- start start 2024-04-26T20:57:22.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.497+0000 7fce13400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fce0c145770 0x7fce0c0b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.497+0000 7fce13400640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fce0c0b3d80 0x7fce0c15c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.497+0000 7fce13400640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce0c145b50 con 0x7fce0c0b41e0 2024-04-26T20:57:22.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.497+0000 7fce13400640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fce0c0cf910 con 0x7fce0c0b3d80 2024-04-26T20:57:22.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.498+0000 7fce13400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fce0c073660 con 0x7fce0c145770 2024-04-26T20:57:22.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.498+0000 7fce0be00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fce0c0b3d80 0x7fce0c15c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:22.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.498+0000 7fce10a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fce0c145770 0x7fce0c0b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:22.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.498+0000 7fce0be00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fce0c0b3d80 0x7fce0c15c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.28:35124/0 (socket says 172.21.15.28:35124) 2024-04-26T20:57:22.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.498+0000 7fce0be00640 1 -- 172.21.15.28:0/3695052585 learned_addr learned my addr 172.21.15.28:0/3695052585 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:22.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.499+0000 7fce0be00640 1 -- 172.21.15.28:0/3695052585 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fce0c145770 msgr2=0x7fce0c0b5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:22.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.499+0000 7fce0be00640 1 --2- 172.21.15.28:0/3695052585 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fce0c145770 0x7fce0c0b5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.499+0000 7fce0be00640 1 -- 172.21.15.28:0/3695052585 >> v1:172.21.15.103:6789/0 conn(0x7fce0c0b41e0 legacy=0x7fce0c0cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:22.500 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.499+0000 7fce0be00640 1 -- 172.21.15.28:0/3695052585 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0c0730a0 con 0x7fce0c0b3d80 2024-04-26T20:57:22.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.500+0000 7fce0be00640 1 --2- 172.21.15.28:0/3695052585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce0c0b3d80 0x7fce0c15c340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400bb40 tx=0x7fcdf402c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8f00a00ca8538c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:22.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.501+0000 7fce0b400640 1 -- 172.21.15.28:0/3695052585 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4036020 con 0x7fce0c0b3d80 2024-04-26T20:57:22.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.501+0000 7fce0b400640 1 -- 172.21.15.28:0/3695052585 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf4032810 con 0x7fce0c0b3d80 2024-04-26T20:57:22.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.501+0000 7fce0b400640 1 -- 172.21.15.28:0/3695052585 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4032be0 con 0x7fce0c0b3d80 2024-04-26T20:57:22.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.502+0000 7fce13400640 1 -- 172.21.15.28:0/3695052585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce0c0b3d80 msgr2=0x7fce0c15c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:22.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.502+0000 7fce13400640 1 --2- 172.21.15.28:0/3695052585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce0c0b3d80 0x7fce0c15c340 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400bb40 tx=0x7fcdf402c950 comp rx=0 tx=0).stop 2024-04-26T20:57:22.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.502+0000 7fce13400640 1 -- 172.21.15.28:0/3695052585 shutdown_connections 2024-04-26T20:57:22.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.502+0000 7fce13400640 1 --2- 172.21.15.28:0/3695052585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce0c0b3d80 0x7fce0c15c340 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.502+0000 7fce13400640 1 --2- 172.21.15.28:0/3695052585 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fce0c145770 0x7fce0c0b5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.502+0000 7fce13400640 1 -- 172.21.15.28:0/3695052585 >> 172.21.15.28:0/3695052585 conn(0x7fce0c06c770 msgr2=0x7fce0c06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:22.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.502+0000 7fce13400640 1 -- 172.21.15.28:0/3695052585 shutdown_connections 2024-04-26T20:57:22.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.502+0000 7fce13400640 1 -- 172.21.15.28:0/3695052585 wait complete. 2024-04-26T20:57:22.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.504+0000 7fce13400640 1 Processor -- start 2024-04-26T20:57:22.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.504+0000 7fce13400640 1 -- start start 2024-04-26T20:57:22.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.505+0000 7fce13400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce0c0b3d80 0x7fce0c0affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.505+0000 7fce13400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce0c0b41e0 0x7fce0c1f8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.505+0000 7fce13400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fce0c145770 0x7fce0c15b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.505+0000 7fce13400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c15d0b0 con 0x7fce0c0b41e0 2024-04-26T20:57:22.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.505+0000 7fce13400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c0cf5a0 con 0x7fce0c0b3d80 2024-04-26T20:57:22.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.505+0000 7fce13400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fce0c15d230 con 0x7fce0c145770 2024-04-26T20:57:22.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.505+0000 7fce0be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce0c0b41e0 0x7fce0c1f8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:22.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.505+0000 7fce0be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce0c0b41e0 0x7fce0c1f8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42250/0 (socket says 172.21.15.28:42250) 2024-04-26T20:57:22.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.505+0000 7fce0be00640 1 -- 172.21.15.28:0/1465519227 learned_addr learned my addr 172.21.15.28:0/1465519227 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:22.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.506+0000 7fce10a00640 1 --2- 172.21.15.28:0/1465519227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce0c0b3d80 0x7fce0c0affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:22.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.506+0000 7fce11400640 1 --2- 172.21.15.28:0/1465519227 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fce0c145770 0x7fce0c15b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:22.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.506+0000 7fce0be00640 1 -- 172.21.15.28:0/1465519227 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fce0c145770 msgr2=0x7fce0c15b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:22.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.506+0000 7fce0be00640 1 --2- 172.21.15.28:0/1465519227 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fce0c145770 0x7fce0c15b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.506+0000 7fce0be00640 1 -- 172.21.15.28:0/1465519227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce0c0b3d80 msgr2=0x7fce0c0affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:22.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.506+0000 7fce0be00640 1 --2- 172.21.15.28:0/1465519227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce0c0b3d80 0x7fce0c0affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.506+0000 7fce0be00640 1 -- 172.21.15.28:0/1465519227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0c143800 con 0x7fce0c0b41e0 2024-04-26T20:57:22.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.506+0000 7fce0be00640 1 --2- 172.21.15.28:0/1465519227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce0c0b41e0 0x7fce0c1f8c70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400bc60 tx=0x7fcdf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:22.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.507+0000 7fce08c00640 1 -- 172.21.15.28:0/1465519227 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4036040 con 0x7fce0c0b41e0 2024-04-26T20:57:22.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.507+0000 7fce08c00640 1 -- 172.21.15.28:0/1465519227 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf4031c20 con 0x7fce0c0b41e0 2024-04-26T20:57:22.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.507+0000 7fce08c00640 1 -- 172.21.15.28:0/1465519227 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4032e60 con 0x7fce0c0b41e0 2024-04-26T20:57:22.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.507+0000 7fce13400640 1 -- 172.21.15.28:0/1465519227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0c1f91b0 con 0x7fce0c0b41e0 2024-04-26T20:57:22.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.507+0000 7fce13400640 1 -- 172.21.15.28:0/1465519227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce0c144030 con 0x7fce0c0b41e0 2024-04-26T20:57:22.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.508+0000 7fce13400640 1 -- 172.21.15.28:0/1465519227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce0c06a360 con 0x7fce0c0b41e0 2024-04-26T20:57:22.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.509+0000 7fce08c00640 1 -- 172.21.15.28:0/1465519227 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdf4039020 con 0x7fce0c0b41e0 2024-04-26T20:57:22.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.510+0000 7fce08c00640 1 --2- 172.21.15.28:0/1465519227 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcde806c8f0 0x7fcde806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.510+0000 7fce08c00640 1 -- 172.21.15.28:0/1465519227 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcdf40ac9b0 con 0x7fce0c0b41e0 2024-04-26T20:57:22.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.510+0000 7fce10a00640 1 --2- 172.21.15.28:0/1465519227 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcde806c8f0 0x7fcde806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:22.512 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.511+0000 7fce10a00640 1 --2- 172.21.15.28:0/1465519227 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcde806c8f0 0x7fcde806edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009a30 tx=0x7fcdfc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:22.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.515+0000 7fce08c00640 1 -- 172.21.15.28:0/1465519227 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcdf407ad40 con 0x7fce0c0b41e0 2024-04-26T20:57:22.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.725+0000 7fce13400640 1 -- 172.21.15.28:0/1465519227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fce0c0b25e0 con 0x7fce0c0b41e0 2024-04-26T20:57:22.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.726+0000 7fce08c00640 1 -- 172.21.15.28:0/1465519227 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fcdf407a640 con 0x7fce0c0b41e0 2024-04-26T20:57:22.727 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-04-26T20:57:22.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.729+0000 7fce13400640 1 -- 172.21.15.28:0/1465519227 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcde806c8f0 msgr2=0x7fcde806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:22.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.729+0000 7fce13400640 1 --2- 172.21.15.28:0/1465519227 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcde806c8f0 0x7fcde806edb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009a30 tx=0x7fcdfc008040 comp rx=0 tx=0).stop 2024-04-26T20:57:22.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.729+0000 7fce13400640 1 -- 172.21.15.28:0/1465519227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce0c0b41e0 msgr2=0x7fce0c1f8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:22.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.729+0000 7fce13400640 1 --2- 172.21.15.28:0/1465519227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce0c0b41e0 0x7fce0c1f8c70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400bc60 tx=0x7fcdf40046c0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.729+0000 7fce13400640 1 -- 172.21.15.28:0/1465519227 shutdown_connections 2024-04-26T20:57:22.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.729+0000 7fce13400640 1 --2- 172.21.15.28:0/1465519227 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcde806c8f0 0x7fcde806edb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.729+0000 7fce13400640 1 --2- 172.21.15.28:0/1465519227 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fce0c145770 0x7fce0c15b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.729+0000 7fce13400640 1 --2- 172.21.15.28:0/1465519227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fce0c0b41e0 0x7fce0c1f8c70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.729+0000 7fce13400640 1 --2- 172.21.15.28:0/1465519227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fce0c0b3d80 0x7fce0c0affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.729+0000 7fce13400640 1 -- 172.21.15.28:0/1465519227 >> 172.21.15.28:0/1465519227 conn(0x7fce0c06c770 msgr2=0x7fce0c15d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:22.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.729+0000 7fce13400640 1 -- 172.21.15.28:0/1465519227 shutdown_connections 2024-04-26T20:57:22.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.729+0000 7fce13400640 1 -- 172.21.15.28:0/1465519227 wait complete. 2024-04-26T20:57:22.749 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:57:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.964+0000 7fb9af200640 1 Processor -- start 2024-04-26T20:57:22.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.965+0000 7fb9af200640 1 -- start start 2024-04-26T20:57:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.966+0000 7fb9af200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fb9a80d0c70 0x7fb9a8161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.966+0000 7fb9af200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a806ed40 con 0x7fb9a80d1050 2024-04-26T20:57:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.966+0000 7fb9af200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a8161860 con 0x7fb9a80b4a30 2024-04-26T20:57:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.966+0000 7fb9af200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a80cf2f0 con 0x7fb9a80d0c70 2024-04-26T20:57:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.967+0000 7fb9ad400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb9a80d1050 0x7fb9a806df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33042/0 (socket says 172.21.15.28:33042) 2024-04-26T20:57:22.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.967+0000 7fb9ad400640 1 -- 172.21.15.28:0/1195356952 learned_addr learned my addr 172.21.15.28:0/1195356952 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:22.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.967+0000 7fb9ade00640 1 --2- 172.21.15.28:0/1195356952 >> v2:172.21.15.103:3300/0 conn(0x7fb9a80d0c70 0x7fb9a8161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:22.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.967+0000 7fb9aca00640 1 -- 172.21.15.28:0/1195356952 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 170574696 0 0) 0x7fb9a806ed40 con 0x7fb9a80d1050 2024-04-26T20:57:22.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.967+0000 7fb9aca00640 1 -- 172.21.15.28:0/1195356952 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb990003520 con 0x7fb9a80d1050 2024-04-26T20:57:22.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.968+0000 7fb9aca00640 1 -- 172.21.15.28:0/1195356952 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2899442413 0 0) 0x7fb990003520 con 0x7fb9a80d1050 2024-04-26T20:57:22.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.968+0000 7fb9aca00640 1 -- 172.21.15.28:0/1195356952 >> v2:172.21.15.103:3300/0 conn(0x7fb9a80d0c70 msgr2=0x7fb9a8161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:22.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.968+0000 7fb9aca00640 1 --2- 172.21.15.28:0/1195356952 >> v2:172.21.15.103:3300/0 conn(0x7fb9a80d0c70 0x7fb9a8161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.968+0000 7fb9aca00640 1 -- 172.21.15.28:0/1195356952 >> v1:172.21.15.103:6789/0 conn(0x7fb9a80b4a30 legacy=0x7fb9a806e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.968+0000 7fb9aca00640 1 -- 172.21.15.28:0/1195356952 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a80cf5d0 con 0x7fb9a80d1050 2024-04-26T20:57:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.968+0000 7fb9aca00640 1 -- 172.21.15.28:0/1195356952 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fb998001670 con 0x7fb9a80d1050 2024-04-26T20:57:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.968+0000 7fb9aca00640 1 -- 172.21.15.28:0/1195356952 >> v1:172.21.15.28:6789/0 conn(0x7fb9a80d1050 legacy=0x7fb9a806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.968+0000 7fb9aca00640 1 --2- 172.21.15.28:0/1195356952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9900039b0 0x7fb990003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.968+0000 7fb9aca00640 1 --2- 172.21.15.28:0/1195356952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb990004800 0x7fb990004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.968+0000 7fb9aca00640 1 --2- 172.21.15.28:0/1195356952 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb9900051c0 0x7fb99000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.968+0000 7fb9aca00640 1 -- 172.21.15.28:0/1195356952 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a80cf2f0 con 0x7fb9900039b0 2024-04-26T20:57:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.968+0000 7fb9aca00640 1 -- 172.21.15.28:0/1195356952 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb99000c090 con 0x7fb990004800 2024-04-26T20:57:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.968+0000 7fb9aca00640 1 -- 172.21.15.28:0/1195356952 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb99000c290 con 0x7fb9900051c0 2024-04-26T20:57:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.968+0000 7fb9ade00640 1 --2- 172.21.15.28:0/1195356952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9900039b0 0x7fb990003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.969+0000 7fb9ade00640 1 -- 172.21.15.28:0/1195356952 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb9900051c0 msgr2=0x7fb99000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.969+0000 7fb9ade00640 1 --2- 172.21.15.28:0/1195356952 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb9900051c0 0x7fb99000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.969+0000 7fb9ad400640 1 --2- 172.21.15.28:0/1195356952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb990004800 0x7fb990004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.969+0000 7fb9ade00640 1 -- 172.21.15.28:0/1195356952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb990004800 msgr2=0x7fb990004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.969+0000 7fb9ade00640 1 --2- 172.21.15.28:0/1195356952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb990004800 0x7fb990004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.969+0000 7fb9ade00640 1 -- 172.21.15.28:0/1195356952 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb990003520 con 0x7fb9900039b0 2024-04-26T20:57:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.969+0000 7fb9ae800640 1 -- 172.21.15.28:0/1195356952 reap_dead start 2024-04-26T20:57:22.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.969+0000 7fb9ad400640 1 --2- 172.21.15.28:0/1195356952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb990004800 0x7fb990004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.970+0000 7fb9ade00640 1 --2- 172.21.15.28:0/1195356952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9900039b0 0x7fb990003df0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009ac0 tx=0x7fb990005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38cf366874e947e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.970+0000 7fb9aca00640 1 -- 172.21.15.28:0/1195356952 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c036070 con 0x7fb9900039b0 2024-04-26T20:57:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.970+0000 7fb9aca00640 1 -- 172.21.15.28:0/1195356952 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb99c031400 con 0x7fb9900039b0 2024-04-26T20:57:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.970+0000 7fb9aca00640 1 -- 172.21.15.28:0/1195356952 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c009690 con 0x7fb9900039b0 2024-04-26T20:57:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.971+0000 7fb9af200640 1 -- 172.21.15.28:0/1195356952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9900039b0 msgr2=0x7fb990003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.971+0000 7fb9af200640 1 --2- 172.21.15.28:0/1195356952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9900039b0 0x7fb990003df0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009ac0 tx=0x7fb990005830 comp rx=0 tx=0).stop 2024-04-26T20:57:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.971+0000 7fb9af200640 1 -- 172.21.15.28:0/1195356952 shutdown_connections 2024-04-26T20:57:22.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.971+0000 7fb9af200640 1 --2- 172.21.15.28:0/1195356952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9900039b0 0x7fb990003df0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.971+0000 7fb9af200640 1 -- 172.21.15.28:0/1195356952 >> 172.21.15.28:0/1195356952 conn(0x7fb9a806c530 msgr2=0x7fb9a806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.971+0000 7fb9af200640 1 -- 172.21.15.28:0/1195356952 shutdown_connections 2024-04-26T20:57:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.971+0000 7fb9af200640 1 -- 172.21.15.28:0/1195356952 wait complete. 2024-04-26T20:57:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.973+0000 7fb9af200640 1 Processor -- start 2024-04-26T20:57:22.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.973+0000 7fb9af200640 1 -- start start 2024-04-26T20:57:22.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.974+0000 7fb9af200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9900039b0 0x7fb9a815b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.974+0000 7fb9af200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9a81423e0 0x7fb9a81e7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.974+0000 7fb9af200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb9a81e8470 0x7fb9a81e6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.974+0000 7fb9af200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a80cfe70 con 0x7fb9900039b0 2024-04-26T20:57:22.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.974+0000 7fb9af200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a80cfb90 con 0x7fb9a81423e0 2024-04-26T20:57:22.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.974+0000 7fb9af200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a8146840 con 0x7fb9a81e8470 2024-04-26T20:57:22.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.974+0000 7fb9ade00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9900039b0 0x7fb9a815b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:22.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.974+0000 7fb9ade00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9900039b0 0x7fb9a815b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42270/0 (socket says 172.21.15.28:42270) 2024-04-26T20:57:22.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.974+0000 7fb9ade00640 1 -- 172.21.15.28:0/663787856 learned_addr learned my addr 172.21.15.28:0/663787856 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:22.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.975+0000 7fb9ad400640 1 --2- 172.21.15.28:0/663787856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9a81423e0 0x7fb9a81e7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:22.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.975+0000 7fb9ade00640 1 -- 172.21.15.28:0/663787856 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb9a81e8470 msgr2=0x7fb9a81e6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:22.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.975+0000 7fb9ade00640 1 --2- 172.21.15.28:0/663787856 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb9a81e8470 0x7fb9a81e6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.975+0000 7fb9ade00640 1 -- 172.21.15.28:0/663787856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9a81423e0 msgr2=0x7fb9a81e7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:22.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.975+0000 7fb9ade00640 1 --2- 172.21.15.28:0/663787856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9a81423e0 0x7fb9a81e7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:22.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.975+0000 7fb9ade00640 1 -- 172.21.15.28:0/663787856 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a8159ec0 con 0x7fb9900039b0 2024-04-26T20:57:22.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.975+0000 7fb9ade00640 1 --2- 172.21.15.28:0/663787856 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9900039b0 0x7fb9a815b0e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009bb0 tx=0x7fb99c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:22.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.975+0000 7fb9a2000640 1 -- 172.21.15.28:0/663787856 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c036050 con 0x7fb9900039b0 2024-04-26T20:57:22.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.975+0000 7fb9a2000640 1 -- 172.21.15.28:0/663787856 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb99c042070 con 0x7fb9900039b0 2024-04-26T20:57:22.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.975+0000 7fb9af200640 1 -- 172.21.15.28:0/663787856 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a815a1a0 con 0x7fb9900039b0 2024-04-26T20:57:22.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.975+0000 7fb9a2000640 1 -- 172.21.15.28:0/663787856 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c003e20 con 0x7fb9900039b0 2024-04-26T20:57:22.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.975+0000 7fb9af200640 1 -- 172.21.15.28:0/663787856 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a815a760 con 0x7fb9900039b0 2024-04-26T20:57:22.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.977+0000 7fb9af200640 1 -- 172.21.15.28:0/663787856 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a81e8890 con 0x7fb9900039b0 2024-04-26T20:57:22.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.978+0000 7fb9a2000640 1 -- 172.21.15.28:0/663787856 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb99c047460 con 0x7fb9900039b0 2024-04-26T20:57:22.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.978+0000 7fb9a2000640 1 --2- 172.21.15.28:0/663787856 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb98c06c8f0 0x7fb98c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:22.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.979+0000 7fb9a2000640 1 -- 172.21.15.28:0/663787856 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb99c0ac690 con 0x7fb9900039b0 2024-04-26T20:57:22.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.979+0000 7fb9ad400640 1 --2- 172.21.15.28:0/663787856 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb98c06c8f0 0x7fb98c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:22.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.980+0000 7fb9ad400640 1 --2- 172.21.15.28:0/663787856 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb98c06c8f0 0x7fb98c06edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb998000ff0 tx=0x7fb998009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:22.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:22.985+0000 7fb9a2000640 1 -- 172.21.15.28:0/663787856 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb99c07aa20 con 0x7fb9900039b0 2024-04-26T20:57:23.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.189+0000 7fb9af200640 1 -- 172.21.15.28:0/663787856 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb9a806a2c0 con 0x7fb9900039b0 2024-04-26T20:57:23.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.190+0000 7fb9a2000640 1 -- 172.21.15.28:0/663787856 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fb99c07a320 con 0x7fb9900039b0 2024-04-26T20:57:23.192 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-04-26T20:57:23.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.193+0000 7fb9af200640 1 -- 172.21.15.28:0/663787856 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb98c06c8f0 msgr2=0x7fb98c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:23.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.193+0000 7fb9af200640 1 --2- 172.21.15.28:0/663787856 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb98c06c8f0 0x7fb98c06edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb998000ff0 tx=0x7fb998009040 comp rx=0 tx=0).stop 2024-04-26T20:57:23.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.193+0000 7fb9af200640 1 -- 172.21.15.28:0/663787856 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9900039b0 msgr2=0x7fb9a815b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:23.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.193+0000 7fb9af200640 1 --2- 172.21.15.28:0/663787856 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9900039b0 0x7fb9a815b0e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009bb0 tx=0x7fb99c004730 comp rx=0 tx=0).stop 2024-04-26T20:57:23.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.193+0000 7fb9af200640 1 -- 172.21.15.28:0/663787856 shutdown_connections 2024-04-26T20:57:23.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.193+0000 7fb9af200640 1 --2- 172.21.15.28:0/663787856 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb98c06c8f0 0x7fb98c06edb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:23.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.193+0000 7fb9af200640 1 --2- 172.21.15.28:0/663787856 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb9a81e8470 0x7fb9a81e6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:23.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.193+0000 7fb9af200640 1 --2- 172.21.15.28:0/663787856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9a81423e0 0x7fb9a81e7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:23.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.193+0000 7fb9af200640 1 --2- 172.21.15.28:0/663787856 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9900039b0 0x7fb9a815b0e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:23.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.193+0000 7fb9af200640 1 -- 172.21.15.28:0/663787856 >> 172.21.15.28:0/663787856 conn(0x7fb9a806c530 msgr2=0x7fb9a80b1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:23.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.193+0000 7fb9af200640 1 -- 172.21.15.28:0/663787856 shutdown_connections 2024-04-26T20:57:23.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.193+0000 7fb9af200640 1 -- 172.21.15.28:0/663787856 wait complete. 2024-04-26T20:57:23.215 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:57:23.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.423+0000 7f9b1ae00640 1 Processor -- start 2024-04-26T20:57:23.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.424+0000 7f9b1ae00640 1 -- start start 2024-04-26T20:57:23.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.425+0000 7f9b1ae00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9b14146670 0x7f9b14071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:23.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.426+0000 7f9b1ae00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b14146440 con 0x7f9b14146060 2024-04-26T20:57:23.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.426+0000 7f9b1ae00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b1406f010 con 0x7f9b141610c0 2024-04-26T20:57:23.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.426+0000 7f9b1ae00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f9b1406f510 con 0x7f9b14146670 2024-04-26T20:57:23.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.426+0000 7f9b19a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f9b14146060 0x7f9b141609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33054/0 (socket says 172.21.15.28:33054) 2024-04-26T20:57:23.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.426+0000 7f9b19a00640 1 -- 172.21.15.28:0/3192395803 learned_addr learned my addr 172.21.15.28:0/3192395803 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:23.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.426+0000 7f9b19000640 1 --2- 172.21.15.28:0/3192395803 >> v2:172.21.15.28:3300/0 conn(0x7f9b14146670 0x7f9b14071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:23.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.427+0000 7f9b13e00640 1 -- 172.21.15.28:0/3192395803 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3792616350 0 0) 0x7f9b14146440 con 0x7f9b14146060 2024-04-26T20:57:23.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.427+0000 7f9b19000640 1 -- 172.21.15.28:0/3192395803 >> v1:172.21.15.103:6789/0 conn(0x7f9b141610c0 legacy=0x7f9b140cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:23.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.427+0000 7f9b19000640 1 -- 172.21.15.28:0/3192395803 >> v1:172.21.15.28:6789/0 conn(0x7f9b14146060 legacy=0x7f9b141609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:23.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.427+0000 7f9b19000640 1 -- 172.21.15.28:0/3192395803 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b141614d0 con 0x7f9b14146670 2024-04-26T20:57:23.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.428+0000 7f9b19000640 1 --2- 172.21.15.28:0/3192395803 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b14146670 0x7f9b14071e20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9b000097a0 tx=0x7f9b0002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b47fea5ba4083a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:23.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.428+0000 7f9b13e00640 1 -- 172.21.15.28:0/3192395803 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00037070 con 0x7f9b14146670 2024-04-26T20:57:23.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.428+0000 7f9b13e00640 1 -- 172.21.15.28:0/3192395803 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b00032440 con 0x7f9b14146670 2024-04-26T20:57:23.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.428+0000 7f9b13e00640 1 -- 172.21.15.28:0/3192395803 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00032810 con 0x7f9b14146670 2024-04-26T20:57:23.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.429+0000 7f9b1ae00640 1 -- 172.21.15.28:0/3192395803 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b14146670 msgr2=0x7f9b14071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:23.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.429+0000 7f9b1ae00640 1 --2- 172.21.15.28:0/3192395803 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b14146670 0x7f9b14071e20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f9b000097a0 tx=0x7f9b0002a1a0 comp rx=0 tx=0).stop 2024-04-26T20:57:23.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.429+0000 7f9b1ae00640 1 -- 172.21.15.28:0/3192395803 shutdown_connections 2024-04-26T20:57:23.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.429+0000 7f9b1ae00640 1 --2- 172.21.15.28:0/3192395803 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b14146670 0x7f9b14071e20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:23.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.429+0000 7f9b1ae00640 1 -- 172.21.15.28:0/3192395803 >> 172.21.15.28:0/3192395803 conn(0x7f9b140d0a50 msgr2=0x7f9b140d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:23.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.429+0000 7f9b1ae00640 1 -- 172.21.15.28:0/3192395803 shutdown_connections 2024-04-26T20:57:23.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.429+0000 7f9b1ae00640 1 -- 172.21.15.28:0/3192395803 wait complete. 2024-04-26T20:57:23.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.431+0000 7f9b1ae00640 1 Processor -- start 2024-04-26T20:57:23.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.431+0000 7f9b1ae00640 1 -- start start 2024-04-26T20:57:23.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.432+0000 7f9b1ae00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b14146060 0x7f9b1415d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:23.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.432+0000 7f9b1ae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b14146670 0x7f9b140cff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:23.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.432+0000 7f9b1ae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b141610c0 0x7f9b140b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:23.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.432+0000 7f9b1ae00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1406ff60 con 0x7f9b141610c0 2024-04-26T20:57:23.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.432+0000 7f9b1ae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1406fc80 con 0x7f9b14146670 2024-04-26T20:57:23.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.432+0000 7f9b1ae00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9b140cb7b0 con 0x7f9b14146060 2024-04-26T20:57:23.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.433+0000 7f9b1a400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b141610c0 0x7f9b140b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:23.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.433+0000 7f9b1a400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b141610c0 0x7f9b140b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42282/0 (socket says 172.21.15.28:42282) 2024-04-26T20:57:23.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.433+0000 7f9b1a400640 1 -- 172.21.15.28:0/3793003493 learned_addr learned my addr 172.21.15.28:0/3793003493 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:23.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.433+0000 7f9b19a00640 1 --2- 172.21.15.28:0/3793003493 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b14146060 0x7f9b1415d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:23.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.433+0000 7f9b19000640 1 --2- 172.21.15.28:0/3793003493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b14146670 0x7f9b140cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:23.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.433+0000 7f9b1a400640 1 -- 172.21.15.28:0/3793003493 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b14146060 msgr2=0x7f9b1415d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:23.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.433+0000 7f9b1a400640 1 --2- 172.21.15.28:0/3793003493 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b14146060 0x7f9b1415d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:23.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.433+0000 7f9b1a400640 1 -- 172.21.15.28:0/3793003493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b14146670 msgr2=0x7f9b140cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:23.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.433+0000 7f9b1a400640 1 --2- 172.21.15.28:0/3793003493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b14146670 0x7f9b140cff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:23.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.433+0000 7f9b1a400640 1 -- 172.21.15.28:0/3793003493 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b14143cf0 con 0x7f9b141610c0 2024-04-26T20:57:23.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.434+0000 7f9b1a400640 1 --2- 172.21.15.28:0/3793003493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b141610c0 0x7f9b140b4860 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00c6b0 tx=0x7f9b0c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:23.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.434+0000 7f9b11600640 1 -- 172.21.15.28:0/3793003493 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c003b70 con 0x7f9b141610c0 2024-04-26T20:57:23.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.434+0000 7f9b11600640 1 -- 172.21.15.28:0/3793003493 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b0c003d30 con 0x7f9b141610c0 2024-04-26T20:57:23.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.434+0000 7f9b11600640 1 -- 172.21.15.28:0/3793003493 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c005310 con 0x7f9b141610c0 2024-04-26T20:57:23.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.434+0000 7f9b1ae00640 1 -- 172.21.15.28:0/3793003493 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b14143fd0 con 0x7f9b141610c0 2024-04-26T20:57:23.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.434+0000 7f9b1ae00640 1 -- 172.21.15.28:0/3793003493 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b14144870 con 0x7f9b141610c0 2024-04-26T20:57:23.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.435+0000 7f9b1ae00640 1 -- 172.21.15.28:0/3793003493 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b141441a0 con 0x7f9b141610c0 2024-04-26T20:57:23.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.436+0000 7f9b11600640 1 -- 172.21.15.28:0/3793003493 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b0c0054b0 con 0x7f9b141610c0 2024-04-26T20:57:23.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.437+0000 7f9b11600640 1 --2- 172.21.15.28:0/3793003493 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9af406c8f0 0x7f9af406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:23.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.437+0000 7f9b11600640 1 -- 172.21.15.28:0/3793003493 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9b0c08bab0 con 0x7f9b141610c0 2024-04-26T20:57:23.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.438+0000 7f9b19a00640 1 --2- 172.21.15.28:0/3793003493 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9af406c8f0 0x7f9af406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:23.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.438+0000 7f9b19a00640 1 --2- 172.21.15.28:0/3793003493 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9af406c8f0 0x7f9af406edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9b08000f60 tx=0x7f9b08007bb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:23.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.442+0000 7f9b11600640 1 -- 172.21.15.28:0/3793003493 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9b0c059e40 con 0x7f9b141610c0 2024-04-26T20:57:23.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.649+0000 7f9b1ae00640 1 -- 172.21.15.28:0/3793003493 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b14144a70 con 0x7f9b141610c0 2024-04-26T20:57:23.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.652+0000 7f9b11600640 1 -- 172.21.15.28:0/3793003493 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7f9b14144a70 con 0x7f9b141610c0 2024-04-26T20:57:23.654 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:23.654 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":21,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:56:52.699663+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:57:23.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.655+0000 7f9b1ae00640 1 -- 172.21.15.28:0/3793003493 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9af406c8f0 msgr2=0x7f9af406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:23.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.655+0000 7f9b1ae00640 1 --2- 172.21.15.28:0/3793003493 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9af406c8f0 0x7f9af406edb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9b08000f60 tx=0x7f9b08007bb0 comp rx=0 tx=0).stop 2024-04-26T20:57:23.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.655+0000 7f9b1ae00640 1 -- 172.21.15.28:0/3793003493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b141610c0 msgr2=0x7f9b140b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:23.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.655+0000 7f9b1ae00640 1 --2- 172.21.15.28:0/3793003493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b141610c0 0x7f9b140b4860 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00c6b0 tx=0x7f9b0c00cb70 comp rx=0 tx=0).stop 2024-04-26T20:57:23.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.655+0000 7f9b1ae00640 1 -- 172.21.15.28:0/3793003493 shutdown_connections 2024-04-26T20:57:23.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.655+0000 7f9b1ae00640 1 --2- 172.21.15.28:0/3793003493 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9af406c8f0 0x7f9af406edb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:23.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.655+0000 7f9b1ae00640 1 --2- 172.21.15.28:0/3793003493 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b141610c0 0x7f9b140b4860 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:23.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.655+0000 7f9b1ae00640 1 --2- 172.21.15.28:0/3793003493 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b14146670 0x7f9b140cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:23.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.655+0000 7f9b1ae00640 1 --2- 172.21.15.28:0/3793003493 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b14146060 0x7f9b1415d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:23.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.655+0000 7f9b1ae00640 1 -- 172.21.15.28:0/3793003493 >> 172.21.15.28:0/3793003493 conn(0x7f9b140d0a50 msgr2=0x7f9b140d0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:23.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.655+0000 7f9b1ae00640 1 -- 172.21.15.28:0/3793003493 shutdown_connections 2024-04-26T20:57:23.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.655+0000 7f9b1ae00640 1 -- 172.21.15.28:0/3793003493 wait complete. 2024-04-26T20:57:23.677 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:56:48.688498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:56:50.117541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:57:23.677 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:57:23.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.888+0000 7f665e000640 1 Processor -- start 2024-04-26T20:57:23.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.888+0000 7f665e000640 1 -- start start 2024-04-26T20:57:23.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.889+0000 7f665e000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f665806fc10 0x7f6658073f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:23.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.889+0000 7f665e000640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f6658073580 0x7f66580ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:23.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.889+0000 7f665e000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f66580ce8d0 0x7f665806e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:23.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.889+0000 7f665e000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f6658074490 con 0x7f66580ce8d0 2024-04-26T20:57:23.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.889+0000 7f665e000640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f66580ced20 con 0x7f6658073580 2024-04-26T20:57:23.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.889+0000 7f665e000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f665815e310 con 0x7f665806fc10 2024-04-26T20:57:23.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.890+0000 7f6657e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f66580ce8d0 0x7f665806e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:23.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.890+0000 7f6657e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f66580ce8d0 0x7f665806e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42294/0 (socket says 172.21.15.28:42294) 2024-04-26T20:57:23.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.890+0000 7f6657e00640 1 -- 172.21.15.28:0/4071124204 learned_addr learned my addr 172.21.15.28:0/4071124204 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:23.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.890+0000 7f6657e00640 1 -- 172.21.15.28:0/4071124204 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f665806fc10 msgr2=0x7f6658073f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:23.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.890+0000 7f6656a00640 1 --2- 172.21.15.28:0/4071124204 >> v2:172.21.15.103:3300/0 conn(0x7f6658073580 0x7f66580ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:23.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.890+0000 7f6657400640 1 --2- 172.21.15.28:0/4071124204 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f665806fc10 0x7f6658073f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:23.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.890+0000 7f6657e00640 1 --2- 172.21.15.28:0/4071124204 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f665806fc10 0x7f6658073f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:23.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.890+0000 7f6657e00640 1 -- 172.21.15.28:0/4071124204 >> v2:172.21.15.103:3300/0 conn(0x7f6658073580 msgr2=0x7f66580ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:23.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.890+0000 7f6657e00640 1 --2- 172.21.15.28:0/4071124204 >> v2:172.21.15.103:3300/0 conn(0x7f6658073580 0x7f66580ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:23.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.890+0000 7f6657e00640 1 -- 172.21.15.28:0/4071124204 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6658072aa0 con 0x7f66580ce8d0 2024-04-26T20:57:23.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.891+0000 7f6657e00640 1 --2- 172.21.15.28:0/4071124204 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66580ce8d0 0x7f665806e610 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f66580b60e0 tx=0x7f665002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82eb3e23b8ce7298 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:23.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.892+0000 7f6656000640 1 -- 172.21.15.28:0/4071124204 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650036070 con 0x7f66580ce8d0 2024-04-26T20:57:23.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.892+0000 7f6656000640 1 -- 172.21.15.28:0/4071124204 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6650031600 con 0x7f66580ce8d0 2024-04-26T20:57:23.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.892+0000 7f6656000640 1 -- 172.21.15.28:0/4071124204 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66500319d0 con 0x7f66580ce8d0 2024-04-26T20:57:23.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.893+0000 7f665e000640 1 -- 172.21.15.28:0/4071124204 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66580ce8d0 msgr2=0x7f665806e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:23.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.893+0000 7f665e000640 1 --2- 172.21.15.28:0/4071124204 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66580ce8d0 0x7f665806e610 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f66580b60e0 tx=0x7f665002a610 comp rx=0 tx=0).stop 2024-04-26T20:57:23.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.893+0000 7f665e000640 1 -- 172.21.15.28:0/4071124204 shutdown_connections 2024-04-26T20:57:23.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.893+0000 7f665e000640 1 --2- 172.21.15.28:0/4071124204 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66580ce8d0 0x7f665806e610 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:23.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.893+0000 7f665e000640 1 --2- 172.21.15.28:0/4071124204 >> v2:172.21.15.103:3300/0 conn(0x7f6658073580 0x7f66580ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:23.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.893+0000 7f665e000640 1 --2- 172.21.15.28:0/4071124204 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f665806fc10 0x7f6658073f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:23.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.893+0000 7f665e000640 1 -- 172.21.15.28:0/4071124204 >> 172.21.15.28:0/4071124204 conn(0x7f665806c770 msgr2=0x7f665806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:23.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.893+0000 7f665e000640 1 -- 172.21.15.28:0/4071124204 shutdown_connections 2024-04-26T20:57:23.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.893+0000 7f665e000640 1 -- 172.21.15.28:0/4071124204 wait complete. 2024-04-26T20:57:23.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.895+0000 7f665e000640 1 Processor -- start 2024-04-26T20:57:23.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.895+0000 7f665e000640 1 -- start start 2024-04-26T20:57:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.896+0000 7f665e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f665806fc10 0x7f6658150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.896+0000 7f665e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6658073580 0x7f665814e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.896+0000 7f665e000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66580ce8d0 0x7f66581518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.896+0000 7f665e000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f665815dfa0 con 0x7f665806fc10 2024-04-26T20:57:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.896+0000 7f665e000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6658073060 con 0x7f6658073580 2024-04-26T20:57:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.896+0000 7f665e000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f6658073a00 con 0x7f66580ce8d0 2024-04-26T20:57:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.896+0000 7f6657400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f665806fc10 0x7f6658150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:23.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.896+0000 7f6657400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f665806fc10 0x7f6658150ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42302/0 (socket says 172.21.15.28:42302) 2024-04-26T20:57:23.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.896+0000 7f6657400640 1 -- 172.21.15.28:0/3944920794 learned_addr learned my addr 172.21.15.28:0/3944920794 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.897+0000 7f6657400640 1 -- 172.21.15.28:0/3944920794 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66580ce8d0 msgr2=0x7f66581518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.897+0000 7f6657400640 1 --2- 172.21.15.28:0/3944920794 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66580ce8d0 0x7f66581518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.897+0000 7f6657400640 1 -- 172.21.15.28:0/3944920794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6658073580 msgr2=0x7f665814e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.897+0000 7f6656a00640 1 --2- 172.21.15.28:0/3944920794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6658073580 0x7f665814e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.897+0000 7f6657400640 1 --2- 172.21.15.28:0/3944920794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6658073580 0x7f665814e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.897+0000 7f6657400640 1 -- 172.21.15.28:0/3944920794 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665814fe40 con 0x7f665806fc10 2024-04-26T20:57:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.897+0000 7f6657400640 1 --2- 172.21.15.28:0/3944920794 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f665806fc10 0x7f6658150ee0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f664800c570 tx=0x7f664800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.897+0000 7f664f400640 1 -- 172.21.15.28:0/3944920794 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648015070 con 0x7f665806fc10 2024-04-26T20:57:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.897+0000 7f664f400640 1 -- 172.21.15.28:0/3944920794 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6648011800 con 0x7f665806fc10 2024-04-26T20:57:23.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.897+0000 7f664f400640 1 -- 172.21.15.28:0/3944920794 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648011bf0 con 0x7f665806fc10 2024-04-26T20:57:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.897+0000 7f665e000640 1 -- 172.21.15.28:0/3944920794 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6658150120 con 0x7f665806fc10 2024-04-26T20:57:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.897+0000 7f665e000640 1 -- 172.21.15.28:0/3944920794 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66581506e0 con 0x7f665806fc10 2024-04-26T20:57:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.898+0000 7f665e000640 1 -- 172.21.15.28:0/3944920794 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f665806a360 con 0x7f665806fc10 2024-04-26T20:57:23.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.899+0000 7f664f400640 1 -- 172.21.15.28:0/3944920794 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6648018070 con 0x7f665806fc10 2024-04-26T20:57:23.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.900+0000 7f664f400640 1 --2- 172.21.15.28:0/3944920794 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f663806c8f0 0x7f663806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:23.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.900+0000 7f664f400640 1 -- 172.21.15.28:0/3944920794 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f664808ba00 con 0x7f665806fc10 2024-04-26T20:57:23.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.901+0000 7f6656a00640 1 --2- 172.21.15.28:0/3944920794 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f663806c8f0 0x7f663806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:23.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.902+0000 7f6656a00640 1 --2- 172.21.15.28:0/3944920794 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f663806c8f0 0x7f663806edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f665814fb80 tx=0x7f6644008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:23.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:23.907+0000 7f664f400640 1 -- 172.21.15.28:0/3944920794 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6648059d90 con 0x7f665806fc10 2024-04-26T20:57:24.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.105+0000 7f665e000640 1 -- 172.21.15.28:0/3944920794 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f66580b32f0 con 0x7f665806fc10 2024-04-26T20:57:24.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.106+0000 7f664f400640 1 -- 172.21.15.28:0/3944920794 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f6648059690 con 0x7f665806fc10 2024-04-26T20:57:24.107 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:57:24.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.108+0000 7f665e000640 1 -- 172.21.15.28:0/3944920794 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f663806c8f0 msgr2=0x7f663806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:24.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.108+0000 7f665e000640 1 --2- 172.21.15.28:0/3944920794 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f663806c8f0 0x7f663806edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f665814fb80 tx=0x7f6644008040 comp rx=0 tx=0).stop 2024-04-26T20:57:24.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.108+0000 7f665e000640 1 -- 172.21.15.28:0/3944920794 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f665806fc10 msgr2=0x7f6658150ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:24.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.108+0000 7f665e000640 1 --2- 172.21.15.28:0/3944920794 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f665806fc10 0x7f6658150ee0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f664800c570 tx=0x7f664800ca30 comp rx=0 tx=0).stop 2024-04-26T20:57:24.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.108+0000 7f665e000640 1 -- 172.21.15.28:0/3944920794 shutdown_connections 2024-04-26T20:57:24.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.108+0000 7f665e000640 1 --2- 172.21.15.28:0/3944920794 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f663806c8f0 0x7f663806edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:24.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.108+0000 7f665e000640 1 --2- 172.21.15.28:0/3944920794 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66580ce8d0 0x7f66581518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:24.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.108+0000 7f665e000640 1 --2- 172.21.15.28:0/3944920794 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6658073580 0x7f665814e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:24.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.108+0000 7f665e000640 1 --2- 172.21.15.28:0/3944920794 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f665806fc10 0x7f6658150ee0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:24.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.108+0000 7f665e000640 1 -- 172.21.15.28:0/3944920794 >> 172.21.15.28:0/3944920794 conn(0x7f665806c770 msgr2=0x7f66580b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:24.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.108+0000 7f665e000640 1 -- 172.21.15.28:0/3944920794 shutdown_connections 2024-04-26T20:57:24.110 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.108+0000 7f665e000640 1 -- 172.21.15.28:0/3944920794 wait complete. 2024-04-26T20:57:24.128 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:57:24.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.333+0000 7fb9cfe00640 1 Processor -- start 2024-04-26T20:57:24.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.334+0000 7fb9cfe00640 1 -- start start 2024-04-26T20:57:24.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.335+0000 7fb9cfe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb9d00b4f40 0x7fb9d006e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:24.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.335+0000 7fb9cfe00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9d00b3ea0 con 0x7fb9d015efd0 2024-04-26T20:57:24.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.335+0000 7fb9cfe00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9d00b4180 con 0x7fb9d00d1420 2024-04-26T20:57:24.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.335+0000 7fb9cfe00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb9d00b4460 con 0x7fb9d00b4f40 2024-04-26T20:57:24.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.336+0000 7fb9cea00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb9d015efd0 0x7fb9d00d0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33062/0 (socket says 172.21.15.28:33062) 2024-04-26T20:57:24.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.336+0000 7fb9cea00640 1 -- 172.21.15.28:0/1412132796 learned_addr learned my addr 172.21.15.28:0/1412132796 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:24.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.336+0000 7fb9cf400640 1 --2- 172.21.15.28:0/1412132796 >> v2:172.21.15.28:3300/0 conn(0x7fb9d00b4f40 0x7fb9d006e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:24.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.336+0000 7fb9cd600640 1 -- 172.21.15.28:0/1412132796 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2655116200 0 0) 0x7fb9d00b3ea0 con 0x7fb9d015efd0 2024-04-26T20:57:24.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.337+0000 7fb9cd600640 1 -- 172.21.15.28:0/1412132796 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb9b4003520 con 0x7fb9d015efd0 2024-04-26T20:57:24.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.337+0000 7fb9cf400640 1 -- 172.21.15.28:0/1412132796 >> v1:172.21.15.103:6789/0 conn(0x7fb9d00d1420 legacy=0x7fb9d006e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:24.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.337+0000 7fb9cf400640 1 -- 172.21.15.28:0/1412132796 >> v1:172.21.15.28:6789/0 conn(0x7fb9d015efd0 legacy=0x7fb9d00d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:24.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.337+0000 7fb9cf400640 1 -- 172.21.15.28:0/1412132796 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d00b4740 con 0x7fb9d00b4f40 2024-04-26T20:57:24.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.338+0000 7fb9cf400640 1 --2- 172.21.15.28:0/1412132796 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9d00b4f40 0x7fb9d006e980 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8009d90 tx=0x7fb9c802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191d115122441179 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:24.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.338+0000 7fb9cd600640 1 -- 172.21.15.28:0/1412132796 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8007740 con 0x7fb9d00b4f40 2024-04-26T20:57:24.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.338+0000 7fb9cd600640 1 -- 172.21.15.28:0/1412132796 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9c80078e0 con 0x7fb9d00b4f40 2024-04-26T20:57:24.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.338+0000 7fb9cd600640 1 -- 172.21.15.28:0/1412132796 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c8007cb0 con 0x7fb9d00b4f40 2024-04-26T20:57:24.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.339+0000 7fb9cfe00640 1 -- 172.21.15.28:0/1412132796 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9d00b4f40 msgr2=0x7fb9d006e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:24.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.339+0000 7fb9cfe00640 1 --2- 172.21.15.28:0/1412132796 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9d00b4f40 0x7fb9d006e980 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8009d90 tx=0x7fb9c802a6f0 comp rx=0 tx=0).stop 2024-04-26T20:57:24.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.339+0000 7fb9cfe00640 1 -- 172.21.15.28:0/1412132796 shutdown_connections 2024-04-26T20:57:24.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.339+0000 7fb9cfe00640 1 --2- 172.21.15.28:0/1412132796 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9d00b4f40 0x7fb9d006e980 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:24.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.339+0000 7fb9cfe00640 1 -- 172.21.15.28:0/1412132796 >> 172.21.15.28:0/1412132796 conn(0x7fb9d006c530 msgr2=0x7fb9d006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:24.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.339+0000 7fb9cfe00640 1 -- 172.21.15.28:0/1412132796 shutdown_connections 2024-04-26T20:57:24.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.339+0000 7fb9cfe00640 1 -- 172.21.15.28:0/1412132796 wait complete. 2024-04-26T20:57:24.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.340+0000 7fb9cfe00640 1 Processor -- start 2024-04-26T20:57:24.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.341+0000 7fb9cfe00640 1 -- start start 2024-04-26T20:57:24.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.342+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9d00b4f40 0x7fb9d01499f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:24.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.342+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9d00d1420 0x7fb9d01466c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:24.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.342+0000 7fb9cfe00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb9d015efd0 0x7fb9d00ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:24.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.342+0000 7fb9cfe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d00b56c0 con 0x7fb9d00b4f40 2024-04-26T20:57:24.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.342+0000 7fb9cfe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d015f3b0 con 0x7fb9d00d1420 2024-04-26T20:57:24.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.342+0000 7fb9cfe00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb9d00b5840 con 0x7fb9d015efd0 2024-04-26T20:57:24.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.342+0000 7fb9cea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9d00b4f40 0x7fb9d01499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:24.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.342+0000 7fb9cea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9d00b4f40 0x7fb9d01499f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42316/0 (socket says 172.21.15.28:42316) 2024-04-26T20:57:24.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.342+0000 7fb9cea00640 1 -- 172.21.15.28:0/1080839753 learned_addr learned my addr 172.21.15.28:0/1080839753 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:24.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.342+0000 7fb9cea00640 1 -- 172.21.15.28:0/1080839753 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb9d015efd0 msgr2=0x7fb9d00ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:24.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.342+0000 7fb9cea00640 1 --2- 172.21.15.28:0/1080839753 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb9d015efd0 0x7fb9d00ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:24.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.342+0000 7fb9cea00640 1 -- 172.21.15.28:0/1080839753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9d00d1420 msgr2=0x7fb9d01466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:24.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.342+0000 7fb9cea00640 1 --2- 172.21.15.28:0/1080839753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9d00d1420 0x7fb9d01466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:24.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.342+0000 7fb9cea00640 1 -- 172.21.15.28:0/1080839753 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d00cd710 con 0x7fb9d00b4f40 2024-04-26T20:57:24.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.343+0000 7fb9cea00640 1 --2- 172.21.15.28:0/1080839753 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9d00b4f40 0x7fb9d01499f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000c410 tx=0x7fb9c000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:24.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.343+0000 7fb9c7400640 1 -- 172.21.15.28:0/1080839753 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0017070 con 0x7fb9d00b4f40 2024-04-26T20:57:24.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.343+0000 7fb9c7400640 1 -- 172.21.15.28:0/1080839753 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9c0005ce0 con 0x7fb9d00b4f40 2024-04-26T20:57:24.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.343+0000 7fb9c7400640 1 -- 172.21.15.28:0/1080839753 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c00072f0 con 0x7fb9d00b4f40 2024-04-26T20:57:24.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.343+0000 7fb9cfe00640 1 -- 172.21.15.28:0/1080839753 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9d00cd9f0 con 0x7fb9d00b4f40 2024-04-26T20:57:24.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.343+0000 7fb9cfe00640 1 -- 172.21.15.28:0/1080839753 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9d0149f30 con 0x7fb9d00b4f40 2024-04-26T20:57:24.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.344+0000 7fb9cfe00640 1 -- 172.21.15.28:0/1080839753 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9d006a250 con 0x7fb9d00b4f40 2024-04-26T20:57:24.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.345+0000 7fb9c7400640 1 -- 172.21.15.28:0/1080839753 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb9c001a070 con 0x7fb9d00b4f40 2024-04-26T20:57:24.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.346+0000 7fb9c7400640 1 --2- 172.21.15.28:0/1080839753 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb9b006c8f0 0x7fb9b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:24.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.346+0000 7fb9c7400640 1 -- 172.21.15.28:0/1080839753 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb9c008afe0 con 0x7fb9d00b4f40 2024-04-26T20:57:24.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.347+0000 7fb9ce000640 1 --2- 172.21.15.28:0/1080839753 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb9b006c8f0 0x7fb9b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:24.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.347+0000 7fb9ce000640 1 --2- 172.21.15.28:0/1080839753 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb9b006c8f0 0x7fb9b006edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc000f60 tx=0x7fb9bc0079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:24.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.351+0000 7fb9c7400640 1 -- 172.21.15.28:0/1080839753 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb9c0059490 con 0x7fb9d00b4f40 2024-04-26T20:57:24.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.558+0000 7fb9cfe00640 1 -- 172.21.15.28:0/1080839753 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb9d0068d20 con 0x7fb9d00b4f40 2024-04-26T20:57:24.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.559+0000 7fb9c7400640 1 -- 172.21.15.28:0/1080839753 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fb9c0058d90 con 0x7fb9d00b4f40 2024-04-26T20:57:24.560 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-04-26T20:57:24.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.561+0000 7fb9cfe00640 1 -- 172.21.15.28:0/1080839753 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb9b006c8f0 msgr2=0x7fb9b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:24.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.561+0000 7fb9cfe00640 1 --2- 172.21.15.28:0/1080839753 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb9b006c8f0 0x7fb9b006edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc000f60 tx=0x7fb9bc0079d0 comp rx=0 tx=0).stop 2024-04-26T20:57:24.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.561+0000 7fb9cfe00640 1 -- 172.21.15.28:0/1080839753 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9d00b4f40 msgr2=0x7fb9d01499f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:24.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.561+0000 7fb9cfe00640 1 --2- 172.21.15.28:0/1080839753 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9d00b4f40 0x7fb9d01499f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000c410 tx=0x7fb9c000c8d0 comp rx=0 tx=0).stop 2024-04-26T20:57:24.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.561+0000 7fb9cfe00640 1 -- 172.21.15.28:0/1080839753 shutdown_connections 2024-04-26T20:57:24.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.561+0000 7fb9cfe00640 1 --2- 172.21.15.28:0/1080839753 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb9b006c8f0 0x7fb9b006edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:24.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.561+0000 7fb9cfe00640 1 --2- 172.21.15.28:0/1080839753 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb9d015efd0 0x7fb9d00ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:24.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.561+0000 7fb9cfe00640 1 --2- 172.21.15.28:0/1080839753 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb9d00d1420 0x7fb9d01466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:24.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.561+0000 7fb9cfe00640 1 --2- 172.21.15.28:0/1080839753 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb9d00b4f40 0x7fb9d01499f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:24.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.561+0000 7fb9cfe00640 1 -- 172.21.15.28:0/1080839753 >> 172.21.15.28:0/1080839753 conn(0x7fb9d006c530 msgr2=0x7fb9d00d0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:24.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.561+0000 7fb9cfe00640 1 -- 172.21.15.28:0/1080839753 shutdown_connections 2024-04-26T20:57:24.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.562+0000 7fb9cfe00640 1 -- 172.21.15.28:0/1080839753 wait complete. 2024-04-26T20:57:24.583 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:57:24.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.796+0000 7f815f200640 1 Processor -- start 2024-04-26T20:57:24.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.797+0000 7f815f200640 1 -- start start 2024-04-26T20:57:24.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.798+0000 7f815f200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f816015c8a0 0x7f816015f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:24.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.798+0000 7f815f200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f81600d1380 0x7f81600b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:24.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.798+0000 7f815f200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81600aef80 con 0x7f81600b0210 2024-04-26T20:57:24.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.798+0000 7f815f200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f81600aed30 con 0x7f816015c8a0 2024-04-26T20:57:24.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.798+0000 7f815f200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f81600af260 con 0x7f81600d1380 2024-04-26T20:57:24.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.799+0000 7f815de00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f816015c8a0 0x7f816015f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:24.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.799+0000 7f815de00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f816015c8a0 0x7f816015f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42318/0 (socket says 172.21.15.28:42318) 2024-04-26T20:57:24.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.799+0000 7f815de00640 1 -- 172.21.15.28:0/3083953269 learned_addr learned my addr 172.21.15.28:0/3083953269 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:24.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.799+0000 7f815e800640 1 --2- 172.21.15.28:0/3083953269 >> v2:172.21.15.103:3300/0 conn(0x7f81600d1380 0x7f81600b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:24.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.799+0000 7f815de00640 1 -- 172.21.15.28:0/3083953269 >> v2:172.21.15.103:3300/0 conn(0x7f81600d1380 msgr2=0x7f81600b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:24.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.799+0000 7f815de00640 1 --2- 172.21.15.28:0/3083953269 >> v2:172.21.15.103:3300/0 conn(0x7f81600d1380 0x7f81600b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:24.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.799+0000 7f815de00640 1 -- 172.21.15.28:0/3083953269 >> v1:172.21.15.103:6789/0 conn(0x7f81600b0210 legacy=0x7f81600d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:24.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.799+0000 7f815de00640 1 -- 172.21.15.28:0/3083953269 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81600af540 con 0x7f816015c8a0 2024-04-26T20:57:24.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.800+0000 7f815de00640 1 --2- 172.21.15.28:0/3083953269 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f816015c8a0 0x7f816015f4f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f81601605e0 tx=0x7f815002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2b33753fd3659b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:24.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.801+0000 7f815ca00640 1 -- 172.21.15.28:0/3083953269 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150037070 con 0x7f816015c8a0 2024-04-26T20:57:24.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.801+0000 7f815ca00640 1 -- 172.21.15.28:0/3083953269 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8150032440 con 0x7f816015c8a0 2024-04-26T20:57:24.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.801+0000 7f815ca00640 1 -- 172.21.15.28:0/3083953269 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150032810 con 0x7f816015c8a0 2024-04-26T20:57:24.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.801+0000 7f815f200640 1 -- 172.21.15.28:0/3083953269 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f816015c8a0 msgr2=0x7f816015f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:24.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.801+0000 7f815f200640 1 --2- 172.21.15.28:0/3083953269 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f816015c8a0 0x7f816015f4f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f81601605e0 tx=0x7f815002a230 comp rx=0 tx=0).stop 2024-04-26T20:57:24.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.801+0000 7f815f200640 1 -- 172.21.15.28:0/3083953269 shutdown_connections 2024-04-26T20:57:24.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.801+0000 7f815f200640 1 --2- 172.21.15.28:0/3083953269 >> v2:172.21.15.103:3300/0 conn(0x7f81600d1380 0x7f81600b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:24.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.801+0000 7f815f200640 1 --2- 172.21.15.28:0/3083953269 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f816015c8a0 0x7f816015f4f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:24.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.801+0000 7f815f200640 1 -- 172.21.15.28:0/3083953269 >> 172.21.15.28:0/3083953269 conn(0x7f816006c530 msgr2=0x7f816006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:24.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.801+0000 7f815f200640 1 -- 172.21.15.28:0/3083953269 shutdown_connections 2024-04-26T20:57:24.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.802+0000 7f815f200640 1 -- 172.21.15.28:0/3083953269 wait complete. 2024-04-26T20:57:24.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.803+0000 7f815f200640 1 Processor -- start 2024-04-26T20:57:24.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.803+0000 7f815f200640 1 -- start start 2024-04-26T20:57:24.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.804+0000 7f815f200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f81600b0210 0x7f8160151350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:24.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.804+0000 7f815f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81600d1380 0x7f8160161810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:24.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.804+0000 7f815f200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f816015c8a0 0x7f8160161d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:24.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.804+0000 7f815f200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f816015cc80 con 0x7f816015c8a0 2024-04-26T20:57:24.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.804+0000 7f815f200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f81600af880 con 0x7f81600d1380 2024-04-26T20:57:24.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.804+0000 7f815f200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f816015ce00 con 0x7f81600b0210 2024-04-26T20:57:24.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.805+0000 7f815e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f816015c8a0 0x7f8160161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:24.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.805+0000 7f815e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f816015c8a0 0x7f8160161d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42332/0 (socket says 172.21.15.28:42332) 2024-04-26T20:57:24.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.805+0000 7f815e800640 1 -- 172.21.15.28:0/2459969218 learned_addr learned my addr 172.21.15.28:0/2459969218 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:24.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.805+0000 7f815de00640 1 --2- 172.21.15.28:0/2459969218 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f81600b0210 0x7f8160151350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:24.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.805+0000 7f815d400640 1 --2- 172.21.15.28:0/2459969218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81600d1380 0x7f8160161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:24.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.805+0000 7f815e800640 1 -- 172.21.15.28:0/2459969218 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f81600b0210 msgr2=0x7f8160151350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:24.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.805+0000 7f815e800640 1 --2- 172.21.15.28:0/2459969218 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f81600b0210 0x7f8160151350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:24.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.805+0000 7f815e800640 1 -- 172.21.15.28:0/2459969218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81600d1380 msgr2=0x7f8160161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:24.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.805+0000 7f815e800640 1 --2- 172.21.15.28:0/2459969218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81600d1380 0x7f8160161810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:24.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.805+0000 7f815e800640 1 -- 172.21.15.28:0/2459969218 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81601f8d10 con 0x7f816015c8a0 2024-04-26T20:57:24.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.806+0000 7f815e800640 1 --2- 172.21.15.28:0/2459969218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f816015c8a0 0x7f8160161d50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f815800cd70 tx=0x7f815800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:24.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.806+0000 7f8156000640 1 -- 172.21.15.28:0/2459969218 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158003b90 con 0x7f816015c8a0 2024-04-26T20:57:24.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.806+0000 7f8156000640 1 -- 172.21.15.28:0/2459969218 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8158003d30 con 0x7f816015c8a0 2024-04-26T20:57:24.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.806+0000 7f8156000640 1 -- 172.21.15.28:0/2459969218 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158004240 con 0x7f816015c8a0 2024-04-26T20:57:24.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.806+0000 7f815f200640 1 -- 172.21.15.28:0/2459969218 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8160150870 con 0x7f816015c8a0 2024-04-26T20:57:24.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.806+0000 7f815f200640 1 -- 172.21.15.28:0/2459969218 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8160150e30 con 0x7f816015c8a0 2024-04-26T20:57:24.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.807+0000 7f815f200640 1 -- 172.21.15.28:0/2459969218 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f816006a2c0 con 0x7f816015c8a0 2024-04-26T20:57:24.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.809+0000 7f8156000640 1 -- 172.21.15.28:0/2459969218 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81580043e0 con 0x7f816015c8a0 2024-04-26T20:57:24.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.810+0000 7f8156000640 1 --2- 172.21.15.28:0/2459969218 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f814006c8f0 0x7f814006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:24.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.810+0000 7f8156000640 1 -- 172.21.15.28:0/2459969218 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f815808c690 con 0x7f816015c8a0 2024-04-26T20:57:24.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.810+0000 7f815de00640 1 --2- 172.21.15.28:0/2459969218 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f814006c8f0 0x7f814006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:24.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.811+0000 7f815de00640 1 --2- 172.21.15.28:0/2459969218 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f814006c8f0 0x7f814006edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8150034000 tx=0x7f815002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:24.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:24.814+0000 7f8156000640 1 -- 172.21.15.28:0/2459969218 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f815801d340 con 0x7f816015c8a0 2024-04-26T20:57:25.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.021+0000 7f815f200640 1 -- 172.21.15.28:0/2459969218 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f816006ee20 con 0x7f816015c8a0 2024-04-26T20:57:25.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.022+0000 7f8156000640 1 -- 172.21.15.28:0/2459969218 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f815805aab0 con 0x7f816015c8a0 2024-04-26T20:57:25.026 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-04-26T20:57:25.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.025+0000 7f815f200640 1 -- 172.21.15.28:0/2459969218 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f814006c8f0 msgr2=0x7f814006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:25.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.025+0000 7f815f200640 1 --2- 172.21.15.28:0/2459969218 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f814006c8f0 0x7f814006edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8150034000 tx=0x7f815002e000 comp rx=0 tx=0).stop 2024-04-26T20:57:25.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.025+0000 7f815f200640 1 -- 172.21.15.28:0/2459969218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f816015c8a0 msgr2=0x7f8160161d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:25.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.025+0000 7f815f200640 1 --2- 172.21.15.28:0/2459969218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f816015c8a0 0x7f8160161d50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f815800cd70 tx=0x7f815800a660 comp rx=0 tx=0).stop 2024-04-26T20:57:25.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.025+0000 7f815f200640 1 -- 172.21.15.28:0/2459969218 shutdown_connections 2024-04-26T20:57:25.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.025+0000 7f815f200640 1 --2- 172.21.15.28:0/2459969218 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f814006c8f0 0x7f814006edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.025+0000 7f815f200640 1 --2- 172.21.15.28:0/2459969218 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f816015c8a0 0x7f8160161d50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.025+0000 7f815f200640 1 --2- 172.21.15.28:0/2459969218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81600d1380 0x7f8160161810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.025+0000 7f815f200640 1 --2- 172.21.15.28:0/2459969218 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f81600b0210 0x7f8160151350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.025+0000 7f815f200640 1 -- 172.21.15.28:0/2459969218 >> 172.21.15.28:0/2459969218 conn(0x7f816006c530 msgr2=0x7f81600d0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:25.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.025+0000 7f815f200640 1 -- 172.21.15.28:0/2459969218 shutdown_connections 2024-04-26T20:57:25.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.025+0000 7f815f200640 1 -- 172.21.15.28:0/2459969218 wait complete. 2024-04-26T20:57:25.048 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:57:25.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.259+0000 7fb8bf200640 1 Processor -- start 2024-04-26T20:57:25.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.260+0000 7fb8bf200640 1 -- start start 2024-04-26T20:57:25.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.262+0000 7fb8bf200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb8c00c9da0 0x7fb8c00ca180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:25.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.262+0000 7fb8bf200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb8c00cb0a0 0x7fb8c0162bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:25.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.262+0000 7fb8bf200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fb8c00cb500 0x7fb8c0163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:25.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.262+0000 7fb8bf200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb8c01603f0 con 0x7fb8c00cb0a0 2024-04-26T20:57:25.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.262+0000 7fb8bf200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fb8c015f580 con 0x7fb8c00cb500 2024-04-26T20:57:25.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.262+0000 7fb8bf200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c0145a10 con 0x7fb8c00c9da0 2024-04-26T20:57:25.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.262+0000 7fb8bd400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb8c00cb0a0 0x7fb8c0162bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:25.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.262+0000 7fb8bd400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb8c00cb0a0 0x7fb8c0162bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42338/0 (socket says 172.21.15.28:42338) 2024-04-26T20:57:25.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.262+0000 7fb8bd400640 1 -- 172.21.15.28:0/2936351266 learned_addr learned my addr 172.21.15.28:0/2936351266 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:25.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.263+0000 7fb8bde00640 1 --2- 172.21.15.28:0/2936351266 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb8c00c9da0 0x7fb8c00ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:25.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.263+0000 7fb8be800640 1 --2- 172.21.15.28:0/2936351266 >> v2:172.21.15.103:3300/0 conn(0x7fb8c00cb500 0x7fb8c0163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:25.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.263+0000 7fb8bd400640 1 -- 172.21.15.28:0/2936351266 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb8c00c9da0 msgr2=0x7fb8c00ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:25.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.263+0000 7fb8bd400640 1 --2- 172.21.15.28:0/2936351266 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb8c00c9da0 0x7fb8c00ca180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.263+0000 7fb8bd400640 1 -- 172.21.15.28:0/2936351266 >> v2:172.21.15.103:3300/0 conn(0x7fb8c00cb500 msgr2=0x7fb8c0163100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:25.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.263+0000 7fb8bd400640 1 --2- 172.21.15.28:0/2936351266 >> v2:172.21.15.103:3300/0 conn(0x7fb8c00cb500 0x7fb8c0163100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.263+0000 7fb8bd400640 1 -- 172.21.15.28:0/2936351266 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c00ca6c0 con 0x7fb8c00cb0a0 2024-04-26T20:57:25.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.263+0000 7fb8be800640 1 --2- 172.21.15.28:0/2936351266 >> v2:172.21.15.103:3300/0 conn(0x7fb8c00cb500 0x7fb8c0163100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:25.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.264+0000 7fb8bd400640 1 --2- 172.21.15.28:0/2936351266 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8c00cb0a0 0x7fb8c0162bc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb8c006e260 tx=0x7fb8a802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bfe139ebb7c77bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:25.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.264+0000 7fb8bca00640 1 -- 172.21.15.28:0/2936351266 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8037070 con 0x7fb8c00cb0a0 2024-04-26T20:57:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.264+0000 7fb8bca00640 1 -- 172.21.15.28:0/2936351266 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8a8032440 con 0x7fb8c00cb0a0 2024-04-26T20:57:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.264+0000 7fb8bca00640 1 -- 172.21.15.28:0/2936351266 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8032810 con 0x7fb8c00cb0a0 2024-04-26T20:57:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.265+0000 7fb8bf200640 1 -- 172.21.15.28:0/2936351266 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8c00cb0a0 msgr2=0x7fb8c0162bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.265+0000 7fb8bf200640 1 --2- 172.21.15.28:0/2936351266 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8c00cb0a0 0x7fb8c0162bc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb8c006e260 tx=0x7fb8a802a200 comp rx=0 tx=0).stop 2024-04-26T20:57:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.265+0000 7fb8bf200640 1 -- 172.21.15.28:0/2936351266 shutdown_connections 2024-04-26T20:57:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.265+0000 7fb8bf200640 1 --2- 172.21.15.28:0/2936351266 >> v2:172.21.15.103:3300/0 conn(0x7fb8c00cb500 0x7fb8c0163100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.265+0000 7fb8bf200640 1 --2- 172.21.15.28:0/2936351266 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8c00cb0a0 0x7fb8c0162bc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.265+0000 7fb8bf200640 1 --2- 172.21.15.28:0/2936351266 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb8c00c9da0 0x7fb8c00ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.265+0000 7fb8bf200640 1 -- 172.21.15.28:0/2936351266 >> 172.21.15.28:0/2936351266 conn(0x7fb8c00d0a50 msgr2=0x7fb8c00d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.265+0000 7fb8bf200640 1 -- 172.21.15.28:0/2936351266 shutdown_connections 2024-04-26T20:57:25.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.265+0000 7fb8bf200640 1 -- 172.21.15.28:0/2936351266 wait complete. 2024-04-26T20:57:25.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.267+0000 7fb8bf200640 1 Processor -- start 2024-04-26T20:57:25.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.267+0000 7fb8bf200640 1 -- start start 2024-04-26T20:57:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.268+0000 7fb8bf200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8c00c9da0 0x7fb8c00b62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.268+0000 7fb8bf200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb8c00cb0a0 0x7fb8c0146d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.268+0000 7fb8bf200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8c00cb500 0x7fb8c0147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.268+0000 7fb8bf200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c015fda0 con 0x7fb8c00cb500 2024-04-26T20:57:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.268+0000 7fb8bf200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c015fac0 con 0x7fb8c00c9da0 2024-04-26T20:57:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.268+0000 7fb8bf200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c0160080 con 0x7fb8c00cb0a0 2024-04-26T20:57:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.269+0000 7fb8be800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8c00cb500 0x7fb8c0147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.269+0000 7fb8be800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8c00cb500 0x7fb8c0147280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42340/0 (socket says 172.21.15.28:42340) 2024-04-26T20:57:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.269+0000 7fb8be800640 1 -- 172.21.15.28:0/192508778 learned_addr learned my addr 172.21.15.28:0/192508778 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.269+0000 7fb8bd400640 1 --2- 172.21.15.28:0/192508778 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb8c00cb0a0 0x7fb8c0146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:25.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.269+0000 7fb8bde00640 1 --2- 172.21.15.28:0/192508778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8c00c9da0 0x7fb8c00b62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.269+0000 7fb8be800640 1 -- 172.21.15.28:0/192508778 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb8c00cb0a0 msgr2=0x7fb8c0146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.269+0000 7fb8be800640 1 --2- 172.21.15.28:0/192508778 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb8c00cb0a0 0x7fb8c0146d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.269+0000 7fb8be800640 1 -- 172.21.15.28:0/192508778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8c00c9da0 msgr2=0x7fb8c00b62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.269+0000 7fb8be800640 1 --2- 172.21.15.28:0/192508778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8c00c9da0 0x7fb8c00b62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.269+0000 7fb8be800640 1 -- 172.21.15.28:0/192508778 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c0141950 con 0x7fb8c00cb500 2024-04-26T20:57:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.270+0000 7fb8be800640 1 --2- 172.21.15.28:0/192508778 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8c00cb500 0x7fb8c0147280 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800c7f0 tx=0x7fb8b800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.270+0000 7fb8b6000640 1 -- 172.21.15.28:0/192508778 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8015070 con 0x7fb8c00cb500 2024-04-26T20:57:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.270+0000 7fb8b6000640 1 -- 172.21.15.28:0/192508778 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8b8007820 con 0x7fb8c00cb500 2024-04-26T20:57:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.270+0000 7fb8b6000640 1 -- 172.21.15.28:0/192508778 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8007c50 con 0x7fb8c00cb500 2024-04-26T20:57:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.270+0000 7fb8bf200640 1 -- 172.21.15.28:0/192508778 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c0141c10 con 0x7fb8c00cb500 2024-04-26T20:57:25.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.270+0000 7fb8bf200640 1 -- 172.21.15.28:0/192508778 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c01421d0 con 0x7fb8c00cb500 2024-04-26T20:57:25.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.271+0000 7fb8bf200640 1 -- 172.21.15.28:0/192508778 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c006a360 con 0x7fb8c00cb500 2024-04-26T20:57:25.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.272+0000 7fb8b6000640 1 -- 172.21.15.28:0/192508778 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8b8018070 con 0x7fb8c00cb500 2024-04-26T20:57:25.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.273+0000 7fb8b6000640 1 --2- 172.21.15.28:0/192508778 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb89c06c8f0 0x7fb89c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:25.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.274+0000 7fb8b6000640 1 -- 172.21.15.28:0/192508778 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb8b808ba90 con 0x7fb8c00cb500 2024-04-26T20:57:25.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.274+0000 7fb8bde00640 1 --2- 172.21.15.28:0/192508778 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb89c06c8f0 0x7fb89c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:25.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.274+0000 7fb8bde00640 1 --2- 172.21.15.28:0/192508778 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb89c06c8f0 0x7fb89c06edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00070b0 tx=0x7fb8b0007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:25.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.278+0000 7fb8b6000640 1 -- 172.21.15.28:0/192508778 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb8b8059ec0 con 0x7fb8c00cb500 2024-04-26T20:57:25.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.484+0000 7fb8bf200640 1 -- 172.21.15.28:0/192508778 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8c0074550 con 0x7fb8c00cb500 2024-04-26T20:57:25.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.486+0000 7fb8b6000640 1 -- 172.21.15.28:0/192508778 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7fb8b80597c0 con 0x7fb8c00cb500 2024-04-26T20:57:25.488 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:25.488 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":21,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:56:52.699663+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:57:25.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.489+0000 7fb8bf200640 1 -- 172.21.15.28:0/192508778 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb89c06c8f0 msgr2=0x7fb89c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:25.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.489+0000 7fb8bf200640 1 --2- 172.21.15.28:0/192508778 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb89c06c8f0 0x7fb89c06edb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00070b0 tx=0x7fb8b0007040 comp rx=0 tx=0).stop 2024-04-26T20:57:25.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.489+0000 7fb8bf200640 1 -- 172.21.15.28:0/192508778 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8c00cb500 msgr2=0x7fb8c0147280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:25.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.489+0000 7fb8bf200640 1 --2- 172.21.15.28:0/192508778 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8c00cb500 0x7fb8c0147280 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800c7f0 tx=0x7fb8b800ccb0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.489+0000 7fb8bf200640 1 -- 172.21.15.28:0/192508778 shutdown_connections 2024-04-26T20:57:25.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.489+0000 7fb8bf200640 1 --2- 172.21.15.28:0/192508778 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb89c06c8f0 0x7fb89c06edb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.489+0000 7fb8bf200640 1 --2- 172.21.15.28:0/192508778 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8c00cb500 0x7fb8c0147280 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.489+0000 7fb8bf200640 1 --2- 172.21.15.28:0/192508778 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb8c00cb0a0 0x7fb8c0146d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.489+0000 7fb8bf200640 1 --2- 172.21.15.28:0/192508778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb8c00c9da0 0x7fb8c00b62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.489+0000 7fb8bf200640 1 -- 172.21.15.28:0/192508778 >> 172.21.15.28:0/192508778 conn(0x7fb8c00d0a50 msgr2=0x7fb8c00b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:25.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.489+0000 7fb8bf200640 1 -- 172.21.15.28:0/192508778 shutdown_connections 2024-04-26T20:57:25.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.490+0000 7fb8bf200640 1 -- 172.21.15.28:0/192508778 wait complete. 2024-04-26T20:57:25.511 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:56:48.688498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:56:50.117541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:57:25.511 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:57:25.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.721+0000 7f89ba800640 1 Processor -- start 2024-04-26T20:57:25.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.721+0000 7f89ba800640 1 -- start start 2024-04-26T20:57:25.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.723+0000 7f89ba800640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f89b415efd0 0x7f89b40d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:25.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.723+0000 7f89ba800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f89b40b4f40 0x7f89b40d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:25.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.723+0000 7f89ba800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f89b40b5c50 0x7f89b40ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:25.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.723+0000 7f89ba800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f89b40b3ea0 con 0x7f89b40b4f40 2024-04-26T20:57:25.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.723+0000 7f89ba800640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f89b40b4180 con 0x7f89b415efd0 2024-04-26T20:57:25.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.723+0000 7f89ba800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f89b40b4460 con 0x7f89b40b5c50 2024-04-26T20:57:25.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.723+0000 7f89b8a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f89b40b4f40 0x7f89b40d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:25.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.723+0000 7f89b8a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f89b40b4f40 0x7f89b40d1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42356/0 (socket says 172.21.15.28:42356) 2024-04-26T20:57:25.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.723+0000 7f89b8a00640 1 -- 172.21.15.28:0/3517065557 learned_addr learned my addr 172.21.15.28:0/3517065557 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:25.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.723+0000 7f89b9400640 1 --2- 172.21.15.28:0/3517065557 >> v2:172.21.15.103:3300/0 conn(0x7f89b415efd0 0x7f89b40d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:25.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.723+0000 7f89b9e00640 1 --2- 172.21.15.28:0/3517065557 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f89b40b5c50 0x7f89b40ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:25.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.723+0000 7f89b8a00640 1 -- 172.21.15.28:0/3517065557 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f89b40b5c50 msgr2=0x7f89b40ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:25.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.724+0000 7f89b8a00640 1 --2- 172.21.15.28:0/3517065557 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f89b40b5c50 0x7f89b40ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.724+0000 7f89b8a00640 1 -- 172.21.15.28:0/3517065557 >> v2:172.21.15.103:3300/0 conn(0x7f89b415efd0 msgr2=0x7f89b40d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:25.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.724+0000 7f89b8a00640 1 --2- 172.21.15.28:0/3517065557 >> v2:172.21.15.103:3300/0 conn(0x7f89b415efd0 0x7f89b40d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.724+0000 7f89b8a00640 1 -- 172.21.15.28:0/3517065557 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b40b4740 con 0x7f89b40b4f40 2024-04-26T20:57:25.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.724+0000 7f89b9e00640 1 --2- 172.21.15.28:0/3517065557 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f89b40b5c50 0x7f89b40ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:25.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.724+0000 7f89b9400640 1 --2- 172.21.15.28:0/3517065557 >> v2:172.21.15.103:3300/0 conn(0x7f89b415efd0 0x7f89b40d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:25.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.725+0000 7f89b8a00640 1 --2- 172.21.15.28:0/3517065557 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b40b4f40 0x7f89b40d1250 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f89b40700c0 tx=0x7f89a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382f212b5b999601 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:25.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.725+0000 7f89b3e00640 1 -- 172.21.15.28:0/3517065557 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0037070 con 0x7f89b40b4f40 2024-04-26T20:57:25.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.725+0000 7f89b3e00640 1 -- 172.21.15.28:0/3517065557 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a0032440 con 0x7f89b40b4f40 2024-04-26T20:57:25.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.725+0000 7f89b3e00640 1 -- 172.21.15.28:0/3517065557 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0032810 con 0x7f89b40b4f40 2024-04-26T20:57:25.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.725+0000 7f89ba800640 1 -- 172.21.15.28:0/3517065557 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b40b4f40 msgr2=0x7f89b40d1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:25.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.725+0000 7f89ba800640 1 --2- 172.21.15.28:0/3517065557 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b40b4f40 0x7f89b40d1250 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f89b40700c0 tx=0x7f89a002a1e0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.726+0000 7f89ba800640 1 -- 172.21.15.28:0/3517065557 shutdown_connections 2024-04-26T20:57:25.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.726+0000 7f89ba800640 1 --2- 172.21.15.28:0/3517065557 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f89b40b5c50 0x7f89b40ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.726+0000 7f89ba800640 1 --2- 172.21.15.28:0/3517065557 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b40b4f40 0x7f89b40d1250 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.726+0000 7f89ba800640 1 --2- 172.21.15.28:0/3517065557 >> v2:172.21.15.103:3300/0 conn(0x7f89b415efd0 0x7f89b40d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.726+0000 7f89ba800640 1 -- 172.21.15.28:0/3517065557 >> 172.21.15.28:0/3517065557 conn(0x7f89b406c530 msgr2=0x7f89b406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:25.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.726+0000 7f89ba800640 1 -- 172.21.15.28:0/3517065557 shutdown_connections 2024-04-26T20:57:25.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.726+0000 7f89ba800640 1 -- 172.21.15.28:0/3517065557 wait complete. 2024-04-26T20:57:25.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.727+0000 7f89ba800640 1 Processor -- start 2024-04-26T20:57:25.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.728+0000 7f89ba800640 1 -- start start 2024-04-26T20:57:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.729+0000 7f89ba800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b40b4f40 0x7f89b4159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.729+0000 7f89ba800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f89b40b5c50 0x7f89b414e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.729+0000 7f89ba800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89b415efd0 0x7f89b4160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.729+0000 7f89ba800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f89b40b56c0 con 0x7f89b40b4f40 2024-04-26T20:57:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.729+0000 7f89ba800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f89b415f3b0 con 0x7f89b415efd0 2024-04-26T20:57:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.729+0000 7f89ba800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f89b40b5840 con 0x7f89b40b5c50 2024-04-26T20:57:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.729+0000 7f89b9400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b40b4f40 0x7f89b4159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.729+0000 7f89b9400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b40b4f40 0x7f89b4159140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42358/0 (socket says 172.21.15.28:42358) 2024-04-26T20:57:25.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.729+0000 7f89b9400640 1 -- 172.21.15.28:0/238849648 learned_addr learned my addr 172.21.15.28:0/238849648 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.729+0000 7f89b8a00640 1 --2- 172.21.15.28:0/238849648 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f89b40b5c50 0x7f89b414e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.729+0000 7f89b9e00640 1 --2- 172.21.15.28:0/238849648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89b415efd0 0x7f89b4160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.729+0000 7f89b9400640 1 -- 172.21.15.28:0/238849648 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f89b40b5c50 msgr2=0x7f89b414e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.729+0000 7f89b9400640 1 --2- 172.21.15.28:0/238849648 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f89b40b5c50 0x7f89b414e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.729+0000 7f89b9400640 1 -- 172.21.15.28:0/238849648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89b415efd0 msgr2=0x7f89b4160600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.729+0000 7f89b9400640 1 --2- 172.21.15.28:0/238849648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89b415efd0 0x7f89b4160600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.729+0000 7f89b9400640 1 -- 172.21.15.28:0/238849648 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b4160f30 con 0x7f89b40b4f40 2024-04-26T20:57:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.730+0000 7f89b9400640 1 --2- 172.21.15.28:0/238849648 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b40b4f40 0x7f89b4159140 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f89a800c7f0 tx=0x7f89a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.730+0000 7f89b1600640 1 -- 172.21.15.28:0/238849648 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8015070 con 0x7f89b40b4f40 2024-04-26T20:57:25.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.730+0000 7f89b1600640 1 -- 172.21.15.28:0/238849648 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89a8007820 con 0x7f89b40b4f40 2024-04-26T20:57:25.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.730+0000 7f89ba800640 1 -- 172.21.15.28:0/238849648 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b4161210 con 0x7f89b40b4f40 2024-04-26T20:57:25.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.730+0000 7f89b1600640 1 -- 172.21.15.28:0/238849648 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8007bd0 con 0x7f89b40b4f40 2024-04-26T20:57:25.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.730+0000 7f89ba800640 1 -- 172.21.15.28:0/238849648 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b41617d0 con 0x7f89b40b4f40 2024-04-26T20:57:25.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.731+0000 7f89ba800640 1 -- 172.21.15.28:0/238849648 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b40ccdb0 con 0x7f89b40b4f40 2024-04-26T20:57:25.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.732+0000 7f89b1600640 1 -- 172.21.15.28:0/238849648 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89a8018070 con 0x7f89b40b4f40 2024-04-26T20:57:25.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.733+0000 7f89b1600640 1 --2- 172.21.15.28:0/238849648 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f899406c920 0x7f899406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:25.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.733+0000 7f89b1600640 1 -- 172.21.15.28:0/238849648 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f89a808bbe0 con 0x7f89b40b4f40 2024-04-26T20:57:25.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.734+0000 7f89b8a00640 1 --2- 172.21.15.28:0/238849648 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f899406c920 0x7f899406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:25.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.734+0000 7f89b8a00640 1 --2- 172.21.15.28:0/238849648 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f899406c920 0x7f899406ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f89a0034000 tx=0x7f89a002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:25.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.738+0000 7f89b1600640 1 -- 172.21.15.28:0/238849648 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f89a805a000 con 0x7f89b40b4f40 2024-04-26T20:57:25.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.942+0000 7f89ba800640 1 -- 172.21.15.28:0/238849648 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f89b406a250 con 0x7f89b40b4f40 2024-04-26T20:57:25.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.944+0000 7f89b1600640 1 -- 172.21.15.28:0/238849648 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f89a8059900 con 0x7f89b40b4f40 2024-04-26T20:57:25.945 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:57:25.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.947+0000 7f89ba800640 1 -- 172.21.15.28:0/238849648 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f899406c920 msgr2=0x7f899406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:25.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.947+0000 7f89ba800640 1 --2- 172.21.15.28:0/238849648 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f899406c920 0x7f899406ede0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f89a0034000 tx=0x7f89a002e000 comp rx=0 tx=0).stop 2024-04-26T20:57:25.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.947+0000 7f89ba800640 1 -- 172.21.15.28:0/238849648 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b40b4f40 msgr2=0x7f89b4159140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:25.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.947+0000 7f89ba800640 1 --2- 172.21.15.28:0/238849648 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b40b4f40 0x7f89b4159140 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f89a800c7f0 tx=0x7f89a800ccb0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.947+0000 7f89ba800640 1 -- 172.21.15.28:0/238849648 shutdown_connections 2024-04-26T20:57:25.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.947+0000 7f89ba800640 1 --2- 172.21.15.28:0/238849648 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f899406c920 0x7f899406ede0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.947+0000 7f89ba800640 1 --2- 172.21.15.28:0/238849648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89b415efd0 0x7f89b4160600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.947+0000 7f89ba800640 1 --2- 172.21.15.28:0/238849648 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f89b40b5c50 0x7f89b414e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.947+0000 7f89ba800640 1 --2- 172.21.15.28:0/238849648 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89b40b4f40 0x7f89b4159140 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:25.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.947+0000 7f89ba800640 1 -- 172.21.15.28:0/238849648 >> 172.21.15.28:0/238849648 conn(0x7f89b406c530 msgr2=0x7f89b415caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:25.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.947+0000 7f89ba800640 1 -- 172.21.15.28:0/238849648 shutdown_connections 2024-04-26T20:57:25.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:25.947+0000 7f89ba800640 1 -- 172.21.15.28:0/238849648 wait complete. 2024-04-26T20:57:25.969 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:57:26.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.190+0000 7f1b31e00640 1 Processor -- start 2024-04-26T20:57:26.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.190+0000 7f1b31e00640 1 -- start start 2024-04-26T20:57:26.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.192+0000 7f1b31e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1b2c0731f0 0x7f1b2c06e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:26.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.192+0000 7f1b31e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1b2c06eb50 0x7f1b2c0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:26.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.192+0000 7f1b31e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b2c06f2d0 con 0x7f1b2c073b10 2024-04-26T20:57:26.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.192+0000 7f1b31e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f1b2c06f5b0 con 0x7f1b2c0731f0 2024-04-26T20:57:26.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.192+0000 7f1b31e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2c06f890 con 0x7f1b2c06eb50 2024-04-26T20:57:26.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.192+0000 7f1b2aa00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1b2c0731f0 0x7f1b2c06e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:26.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.192+0000 7f1b2aa00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1b2c0731f0 0x7f1b2c06e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42364/0 (socket says 172.21.15.28:42364) 2024-04-26T20:57:26.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.192+0000 7f1b2aa00640 1 -- 172.21.15.28:0/3220623051 learned_addr learned my addr 172.21.15.28:0/3220623051 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:26.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.192+0000 7f1b2a000640 1 --2- 172.21.15.28:0/3220623051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1b2c06eb50 0x7f1b2c0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:26.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.193+0000 7f1b2aa00640 1 -- 172.21.15.28:0/3220623051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1b2c06eb50 msgr2=0x7f1b2c0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:26.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.193+0000 7f1b2aa00640 1 --2- 172.21.15.28:0/3220623051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1b2c06eb50 0x7f1b2c0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:26.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.193+0000 7f1b2aa00640 1 -- 172.21.15.28:0/3220623051 >> v1:172.21.15.28:6789/0 conn(0x7f1b2c073b10 legacy=0x7f1b2c0b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:26.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.193+0000 7f1b2aa00640 1 -- 172.21.15.28:0/3220623051 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2c06fb70 con 0x7f1b2c0731f0 2024-04-26T20:57:26.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.193+0000 7f1b2a000640 1 --2- 172.21.15.28:0/3220623051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1b2c06eb50 0x7f1b2c0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:26.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.194+0000 7f1b2aa00640 1 --2- 172.21.15.28:0/3220623051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b2c0731f0 0x7f1b2c06e610 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009e50 tx=0x7f1b1c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c88b6f3354d66e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:26.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.194+0000 7f1b29600640 1 -- 172.21.15.28:0/3220623051 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c036070 con 0x7f1b2c0731f0 2024-04-26T20:57:26.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.194+0000 7f1b29600640 1 -- 172.21.15.28:0/3220623051 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b1c031600 con 0x7f1b2c0731f0 2024-04-26T20:57:26.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.194+0000 7f1b29600640 1 -- 172.21.15.28:0/3220623051 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0319d0 con 0x7f1b2c0731f0 2024-04-26T20:57:26.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.195+0000 7f1b31e00640 1 -- 172.21.15.28:0/3220623051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b2c0731f0 msgr2=0x7f1b2c06e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:26.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.195+0000 7f1b31e00640 1 --2- 172.21.15.28:0/3220623051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b2c0731f0 0x7f1b2c06e610 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c009e50 tx=0x7f1b1c02a710 comp rx=0 tx=0).stop 2024-04-26T20:57:26.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.195+0000 7f1b31e00640 1 -- 172.21.15.28:0/3220623051 shutdown_connections 2024-04-26T20:57:26.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.195+0000 7f1b31e00640 1 --2- 172.21.15.28:0/3220623051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1b2c06eb50 0x7f1b2c0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:26.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.195+0000 7f1b31e00640 1 --2- 172.21.15.28:0/3220623051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b2c0731f0 0x7f1b2c06e610 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:26.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.195+0000 7f1b31e00640 1 -- 172.21.15.28:0/3220623051 >> 172.21.15.28:0/3220623051 conn(0x7f1b2c06c770 msgr2=0x7f1b2c06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:26.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.195+0000 7f1b31e00640 1 -- 172.21.15.28:0/3220623051 shutdown_connections 2024-04-26T20:57:26.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.195+0000 7f1b31e00640 1 -- 172.21.15.28:0/3220623051 wait complete. 2024-04-26T20:57:26.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.197+0000 7f1b31e00640 1 Processor -- start 2024-04-26T20:57:26.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.197+0000 7f1b31e00640 1 -- start start 2024-04-26T20:57:26.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.198+0000 7f1b31e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1b2c06eb50 0x7f1b2c140ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:26.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.198+0000 7f1b31e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b2c0731f0 0x7f1b2c15eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:26.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.198+0000 7f1b31e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b2c073b10 0x7f1b2c142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:26.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.198+0000 7f1b31e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c073f70 con 0x7f1b2c073b10 2024-04-26T20:57:26.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.198+0000 7f1b31e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c06efb0 con 0x7f1b2c0731f0 2024-04-26T20:57:26.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.198+0000 7f1b31e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2c0b4690 con 0x7f1b2c06eb50 2024-04-26T20:57:26.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.198+0000 7f1b2b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b2c073b10 0x7f1b2c142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:26.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.198+0000 7f1b2b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b2c073b10 0x7f1b2c142ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42380/0 (socket says 172.21.15.28:42380) 2024-04-26T20:57:26.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.198+0000 7f1b2b400640 1 -- 172.21.15.28:0/1831352508 learned_addr learned my addr 172.21.15.28:0/1831352508 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:26.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.199+0000 7f1b2a000640 1 --2- 172.21.15.28:0/1831352508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b2c0731f0 0x7f1b2c15eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:26.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.199+0000 7f1b2aa00640 1 --2- 172.21.15.28:0/1831352508 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1b2c06eb50 0x7f1b2c140ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:26.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.199+0000 7f1b2b400640 1 -- 172.21.15.28:0/1831352508 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1b2c06eb50 msgr2=0x7f1b2c140ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:26.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.199+0000 7f1b2b400640 1 --2- 172.21.15.28:0/1831352508 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1b2c06eb50 0x7f1b2c140ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:26.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.199+0000 7f1b2b400640 1 -- 172.21.15.28:0/1831352508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b2c0731f0 msgr2=0x7f1b2c15eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:26.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.199+0000 7f1b2b400640 1 --2- 172.21.15.28:0/1831352508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b2c0731f0 0x7f1b2c15eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:26.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.199+0000 7f1b2b400640 1 -- 172.21.15.28:0/1831352508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2c1410e0 con 0x7f1b2c073b10 2024-04-26T20:57:26.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.199+0000 7f1b2b400640 1 --2- 172.21.15.28:0/1831352508 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b2c073b10 0x7f1b2c142ee0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400c3c0 tx=0x7f1b2400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:26.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.199+0000 7f1b23400640 1 -- 172.21.15.28:0/1831352508 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b240052d0 con 0x7f1b2c073b10 2024-04-26T20:57:26.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.200+0000 7f1b23400640 1 -- 172.21.15.28:0/1831352508 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b24005470 con 0x7f1b2c073b10 2024-04-26T20:57:26.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.200+0000 7f1b23400640 1 -- 172.21.15.28:0/1831352508 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24007240 con 0x7f1b2c073b10 2024-04-26T20:57:26.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.200+0000 7f1b31e00640 1 -- 172.21.15.28:0/1831352508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b2c1413c0 con 0x7f1b2c073b10 2024-04-26T20:57:26.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.200+0000 7f1b31e00640 1 -- 172.21.15.28:0/1831352508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b2c141980 con 0x7f1b2c073b10 2024-04-26T20:57:26.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.201+0000 7f1b31e00640 1 -- 172.21.15.28:0/1831352508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b2c06a360 con 0x7f1b2c073b10 2024-04-26T20:57:26.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.202+0000 7f1b23400640 1 -- 172.21.15.28:0/1831352508 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b240073e0 con 0x7f1b2c073b10 2024-04-26T20:57:26.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.203+0000 7f1b23400640 1 --2- 172.21.15.28:0/1831352508 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1b0806c8f0 0x7f1b0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:26.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.203+0000 7f1b23400640 1 -- 172.21.15.28:0/1831352508 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1b2408af20 con 0x7f1b2c073b10 2024-04-26T20:57:26.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.203+0000 7f1b2aa00640 1 --2- 172.21.15.28:0/1831352508 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1b0806c8f0 0x7f1b0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:26.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.204+0000 7f1b2aa00640 1 --2- 172.21.15.28:0/1831352508 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1b0806c8f0 0x7f1b0806edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c033000 tx=0x7f1b1c009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:26.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.208+0000 7f1b23400640 1 -- 172.21.15.28:0/1831352508 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1b24059340 con 0x7f1b2c073b10 2024-04-26T20:57:26.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.414+0000 7f1b31e00640 1 -- 172.21.15.28:0/1831352508 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1b2c0d0810 con 0x7f1b2c073b10 2024-04-26T20:57:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.415+0000 7f1b23400640 1 -- 172.21.15.28:0/1831352508 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f1b24058c40 con 0x7f1b2c073b10 2024-04-26T20:57:26.416 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-04-26T20:57:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.419+0000 7f1b31e00640 1 -- 172.21.15.28:0/1831352508 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1b0806c8f0 msgr2=0x7f1b0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.419+0000 7f1b31e00640 1 --2- 172.21.15.28:0/1831352508 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1b0806c8f0 0x7f1b0806edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c033000 tx=0x7f1b1c009000 comp rx=0 tx=0).stop 2024-04-26T20:57:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.419+0000 7f1b31e00640 1 -- 172.21.15.28:0/1831352508 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b2c073b10 msgr2=0x7f1b2c142ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:26.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.419+0000 7f1b31e00640 1 --2- 172.21.15.28:0/1831352508 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b2c073b10 0x7f1b2c142ee0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1b2400c3c0 tx=0x7f1b2400c880 comp rx=0 tx=0).stop 2024-04-26T20:57:26.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.419+0000 7f1b31e00640 1 -- 172.21.15.28:0/1831352508 shutdown_connections 2024-04-26T20:57:26.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.419+0000 7f1b31e00640 1 --2- 172.21.15.28:0/1831352508 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1b0806c8f0 0x7f1b0806edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:26.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.419+0000 7f1b31e00640 1 --2- 172.21.15.28:0/1831352508 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b2c073b10 0x7f1b2c142ee0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:26.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.419+0000 7f1b31e00640 1 --2- 172.21.15.28:0/1831352508 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b2c0731f0 0x7f1b2c15eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:26.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.419+0000 7f1b31e00640 1 --2- 172.21.15.28:0/1831352508 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1b2c06eb50 0x7f1b2c140ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:26.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.419+0000 7f1b31e00640 1 -- 172.21.15.28:0/1831352508 >> 172.21.15.28:0/1831352508 conn(0x7f1b2c06c770 msgr2=0x7f1b2c06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:26.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.419+0000 7f1b31e00640 1 -- 172.21.15.28:0/1831352508 shutdown_connections 2024-04-26T20:57:26.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.419+0000 7f1b31e00640 1 -- 172.21.15.28:0/1831352508 wait complete. 2024-04-26T20:57:26.438 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:57:26.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.641+0000 7f9522000640 1 Processor -- start 2024-04-26T20:57:26.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.641+0000 7f9522000640 1 -- start start 2024-04-26T20:57:26.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.643+0000 7f9522000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f951c0d0c70 0x7f951c161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:26.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.643+0000 7f9522000640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f951c0b4a30 0x7f951c06e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:26.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.643+0000 7f9522000640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f951c161860 con 0x7f951c0d1050 2024-04-26T20:57:26.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.643+0000 7f9522000640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f951c0cf2f0 con 0x7f951c0b4a30 2024-04-26T20:57:26.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.643+0000 7f9522000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f951c0cf5d0 con 0x7f951c0d0c70 2024-04-26T20:57:26.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.644+0000 7f9521600640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f951c0b4a30 0x7f951c06e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:26.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.644+0000 7f9520c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f951c0d0c70 0x7f951c161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:26.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.644+0000 7f9521600640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f951c0b4a30 0x7f951c06e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.28:35302/0 (socket says 172.21.15.28:35302) 2024-04-26T20:57:26.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.644+0000 7f9521600640 1 -- 172.21.15.28:0/1281202037 learned_addr learned my addr 172.21.15.28:0/1281202037 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:26.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.644+0000 7f9521600640 1 -- 172.21.15.28:0/1281202037 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f951c0d0c70 msgr2=0x7f951c161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:26.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.644+0000 7f9521600640 1 --2- 172.21.15.28:0/1281202037 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f951c0d0c70 0x7f951c161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:26.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.644+0000 7f9521600640 1 -- 172.21.15.28:0/1281202037 >> v1:172.21.15.103:6789/0 conn(0x7f951c0d1050 legacy=0x7f951c06df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:26.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.645+0000 7f9521600640 1 -- 172.21.15.28:0/1281202037 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951c0cf8b0 con 0x7f951c0b4a30 2024-04-26T20:57:26.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.646+0000 7f9521600640 1 --2- 172.21.15.28:0/1281202037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f951c0b4a30 0x7f951c06e630 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9514009870 tx=0x7f951402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfaad4ce124aa51c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:26.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.646+0000 7f951b400640 1 -- 172.21.15.28:0/1281202037 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514037070 con 0x7f951c0b4a30 2024-04-26T20:57:26.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.646+0000 7f951b400640 1 -- 172.21.15.28:0/1281202037 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9514032440 con 0x7f951c0b4a30 2024-04-26T20:57:26.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.646+0000 7f951b400640 1 -- 172.21.15.28:0/1281202037 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514032810 con 0x7f951c0b4a30 2024-04-26T20:57:26.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.647+0000 7f9522000640 1 -- 172.21.15.28:0/1281202037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f951c0b4a30 msgr2=0x7f951c06e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:26.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.647+0000 7f9522000640 1 --2- 172.21.15.28:0/1281202037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f951c0b4a30 0x7f951c06e630 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9514009870 tx=0x7f951402a250 comp rx=0 tx=0).stop 2024-04-26T20:57:26.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.647+0000 7f9522000640 1 -- 172.21.15.28:0/1281202037 shutdown_connections 2024-04-26T20:57:26.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.647+0000 7f9522000640 1 --2- 172.21.15.28:0/1281202037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f951c0b4a30 0x7f951c06e630 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:26.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.647+0000 7f9522000640 1 --2- 172.21.15.28:0/1281202037 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f951c0d0c70 0x7f951c161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:26.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.647+0000 7f9522000640 1 -- 172.21.15.28:0/1281202037 >> 172.21.15.28:0/1281202037 conn(0x7f951c06c530 msgr2=0x7f951c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:26.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.647+0000 7f9522000640 1 -- 172.21.15.28:0/1281202037 shutdown_connections 2024-04-26T20:57:26.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.647+0000 7f9522000640 1 -- 172.21.15.28:0/1281202037 wait complete. 2024-04-26T20:57:26.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.649+0000 7f9522000640 1 Processor -- start 2024-04-26T20:57:26.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.649+0000 7f9522000640 1 -- start start 2024-04-26T20:57:26.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.650+0000 7f9522000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f951c0b4a30 0x7f951c1f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:26.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.650+0000 7f9522000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f951c0d0c70 0x7f951c158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:26.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.650+0000 7f9522000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f951c0d1050 0x7f951c158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:26.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.650+0000 7f9522000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f951c146840 con 0x7f951c0b4a30 2024-04-26T20:57:26.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.650+0000 7f9522000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f951c0cfe70 con 0x7f951c0d1050 2024-04-26T20:57:26.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.650+0000 7f9522000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f951c0d14b0 con 0x7f951c0d0c70 2024-04-26T20:57:26.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.650+0000 7f9520c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f951c0b4a30 0x7f951c1f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:26.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.650+0000 7f9520c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f951c0b4a30 0x7f951c1f2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42396/0 (socket says 172.21.15.28:42396) 2024-04-26T20:57:26.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.650+0000 7f9520c00640 1 -- 172.21.15.28:0/3024225343 learned_addr learned my addr 172.21.15.28:0/3024225343 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:26.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.651+0000 7f9521600640 1 --2- 172.21.15.28:0/3024225343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f951c0d1050 0x7f951c158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:26.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.651+0000 7f9520c00640 1 -- 172.21.15.28:0/3024225343 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f951c0d0c70 msgr2=0x7f951c158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:26.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.651+0000 7f9520c00640 1 --2- 172.21.15.28:0/3024225343 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f951c0d0c70 0x7f951c158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:26.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.651+0000 7f9520c00640 1 -- 172.21.15.28:0/3024225343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f951c0d1050 msgr2=0x7f951c158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:26.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.651+0000 7f9520c00640 1 --2- 172.21.15.28:0/3024225343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f951c0d1050 0x7f951c158b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:26.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.651+0000 7f9520c00640 1 -- 172.21.15.28:0/3024225343 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f951c12fcc0 con 0x7f951c0b4a30 2024-04-26T20:57:26.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.651+0000 7f9520c00640 1 --2- 172.21.15.28:0/3024225343 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f951c0b4a30 0x7f951c1f2d30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f950c009db0 tx=0x7f950c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:26.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.651+0000 7f9518c00640 1 -- 172.21.15.28:0/3024225343 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c018020 con 0x7f951c0b4a30 2024-04-26T20:57:26.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.651+0000 7f9518c00640 1 -- 172.21.15.28:0/3024225343 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f950c014480 con 0x7f951c0b4a30 2024-04-26T20:57:26.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.651+0000 7f9518c00640 1 -- 172.21.15.28:0/3024225343 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950c014850 con 0x7f951c0b4a30 2024-04-26T20:57:26.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.651+0000 7f9522000640 1 -- 172.21.15.28:0/3024225343 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f951c1f3270 con 0x7f951c0b4a30 2024-04-26T20:57:26.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.651+0000 7f9522000640 1 -- 172.21.15.28:0/3024225343 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f951c145700 con 0x7f951c0b4a30 2024-04-26T20:57:26.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.652+0000 7f9522000640 1 -- 172.21.15.28:0/3024225343 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f951c06a2c0 con 0x7f951c0b4a30 2024-04-26T20:57:26.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.653+0000 7f9518c00640 1 -- 172.21.15.28:0/3024225343 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f950c005d10 con 0x7f951c0b4a30 2024-04-26T20:57:26.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.655+0000 7f9518c00640 1 --2- 172.21.15.28:0/3024225343 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f94fc06c8f0 0x7f94fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:26.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.655+0000 7f9518c00640 1 -- 172.21.15.28:0/3024225343 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f950c08c690 con 0x7f951c0b4a30 2024-04-26T20:57:26.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.655+0000 7f951be00640 1 --2- 172.21.15.28:0/3024225343 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f94fc06c8f0 0x7f94fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:26.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.656+0000 7f951be00640 1 --2- 172.21.15.28:0/3024225343 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f94fc06c8f0 0x7f94fc06edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9508000f60 tx=0x7f95080079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:26.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.660+0000 7f9518c00640 1 -- 172.21.15.28:0/3024225343 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f950c022070 con 0x7f951c0b4a30 2024-04-26T20:57:26.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.861+0000 7f9522000640 1 -- 172.21.15.28:0/3024225343 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f951c06f760 con 0x7f951c0b4a30 2024-04-26T20:57:26.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.862+0000 7f9518c00640 1 -- 172.21.15.28:0/3024225343 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f950c010040 con 0x7f951c0b4a30 2024-04-26T20:57:26.863 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-04-26T20:57:26.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.864+0000 7f9522000640 1 -- 172.21.15.28:0/3024225343 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f94fc06c8f0 msgr2=0x7f94fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.864+0000 7f9522000640 1 --2- 172.21.15.28:0/3024225343 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f94fc06c8f0 0x7f94fc06edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9508000f60 tx=0x7f95080079d0 comp rx=0 tx=0).stop 2024-04-26T20:57:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.864+0000 7f9522000640 1 -- 172.21.15.28:0/3024225343 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f951c0b4a30 msgr2=0x7f951c1f2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.864+0000 7f9522000640 1 --2- 172.21.15.28:0/3024225343 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f951c0b4a30 0x7f951c1f2d30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f950c009db0 tx=0x7f950c00c490 comp rx=0 tx=0).stop 2024-04-26T20:57:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.864+0000 7f9522000640 1 -- 172.21.15.28:0/3024225343 shutdown_connections 2024-04-26T20:57:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.864+0000 7f9522000640 1 --2- 172.21.15.28:0/3024225343 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f94fc06c8f0 0x7f94fc06edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.864+0000 7f9522000640 1 --2- 172.21.15.28:0/3024225343 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f951c0d1050 0x7f951c158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.864+0000 7f9522000640 1 --2- 172.21.15.28:0/3024225343 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f951c0d0c70 0x7f951c158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.864+0000 7f9522000640 1 --2- 172.21.15.28:0/3024225343 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f951c0b4a30 0x7f951c1f2d30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.864+0000 7f9522000640 1 -- 172.21.15.28:0/3024225343 >> 172.21.15.28:0/3024225343 conn(0x7f951c06c530 msgr2=0x7f951c0b2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.865+0000 7f9522000640 1 -- 172.21.15.28:0/3024225343 shutdown_connections 2024-04-26T20:57:26.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:26.865+0000 7f9522000640 1 -- 172.21.15.28:0/3024225343 wait complete. 2024-04-26T20:57:26.884 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:57:27.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.091+0000 7f7c10e00640 1 Processor -- start 2024-04-26T20:57:27.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.091+0000 7f7c10e00640 1 -- start start 2024-04-26T20:57:27.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.093+0000 7f7c10e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c0c15f1b0 0x7f7c0c0d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:27.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.093+0000 7f7c10e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7c0c0cf700 0x7f7c0c0d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:27.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.093+0000 7f7c10e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c0c0d0710 con 0x7f7c0c0d5370 2024-04-26T20:57:27.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.093+0000 7f7c10e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f7c0c0cff00 con 0x7f7c0c0cf700 2024-04-26T20:57:27.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.093+0000 7f7c10e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7c0c0d4270 con 0x7f7c0c15f1b0 2024-04-26T20:57:27.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.093+0000 7f7c03e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7c0c0cf700 0x7f7c0c0d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:27.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.093+0000 7f7c0be00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7c0c0d5370 0x7f7c0c0b80b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33076/0 (socket says 172.21.15.28:33076) 2024-04-26T20:57:27.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.094+0000 7f7c0be00640 1 -- 172.21.15.28:0/3654387063 learned_addr learned my addr 172.21.15.28:0/3654387063 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:27.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.094+0000 7f7c0b400640 1 --2- 172.21.15.28:0/3654387063 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c0c15f1b0 0x7f7c0c0d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:27.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.094+0000 7f7c03e00640 1 -- 172.21.15.28:0/3654387063 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c0c15f1b0 msgr2=0x7f7c0c0d4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:27.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.094+0000 7f7c03e00640 1 --2- 172.21.15.28:0/3654387063 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c0c15f1b0 0x7f7c0c0d4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:27.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.094+0000 7f7c03e00640 1 -- 172.21.15.28:0/3654387063 >> v1:172.21.15.28:6789/0 conn(0x7f7c0c0d5370 legacy=0x7f7c0c0b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:27.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.094+0000 7f7c03e00640 1 -- 172.21.15.28:0/3654387063 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0c0d0150 con 0x7f7c0c0cf700 2024-04-26T20:57:27.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.094+0000 7f7c0aa00640 1 -- 172.21.15.28:0/3654387063 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 202776374 0 0) 0x7f7c04001490 con 0x7f7c0c0d5370 2024-04-26T20:57:27.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.095+0000 7f7c03e00640 1 --2- 172.21.15.28:0/3654387063 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c0cf700 0x7f7c0c0d4e30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40097a0 tx=0x7f7bf402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b8d23db105fb06 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:27.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.095+0000 7f7c0aa00640 1 -- 172.21.15.28:0/3654387063 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4037070 con 0x7f7c0c0cf700 2024-04-26T20:57:27.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.096+0000 7f7c0aa00640 1 -- 172.21.15.28:0/3654387063 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bf4032440 con 0x7f7c0c0cf700 2024-04-26T20:57:27.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.096+0000 7f7c0aa00640 1 -- 172.21.15.28:0/3654387063 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4032810 con 0x7f7c0c0cf700 2024-04-26T20:57:27.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.096+0000 7f7c10e00640 1 -- 172.21.15.28:0/3654387063 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c0cf700 msgr2=0x7f7c0c0d4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:27.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.096+0000 7f7c10e00640 1 --2- 172.21.15.28:0/3654387063 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c0cf700 0x7f7c0c0d4e30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7bf40097a0 tx=0x7f7bf402a210 comp rx=0 tx=0).stop 2024-04-26T20:57:27.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.096+0000 7f7c10e00640 1 -- 172.21.15.28:0/3654387063 shutdown_connections 2024-04-26T20:57:27.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.096+0000 7f7c10e00640 1 --2- 172.21.15.28:0/3654387063 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c0cf700 0x7f7c0c0d4e30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:27.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.096+0000 7f7c10e00640 1 --2- 172.21.15.28:0/3654387063 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c0c15f1b0 0x7f7c0c0d4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:27.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.096+0000 7f7c10e00640 1 -- 172.21.15.28:0/3654387063 >> 172.21.15.28:0/3654387063 conn(0x7f7c0c06c770 msgr2=0x7f7c0c06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:27.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.096+0000 7f7c10e00640 1 -- 172.21.15.28:0/3654387063 shutdown_connections 2024-04-26T20:57:27.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.096+0000 7f7c10e00640 1 -- 172.21.15.28:0/3654387063 wait complete. 2024-04-26T20:57:27.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.098+0000 7f7c10e00640 1 Processor -- start 2024-04-26T20:57:27.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.098+0000 7f7c10e00640 1 -- start start 2024-04-26T20:57:27.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.099+0000 7f7c10e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0c0cf700 0x7f7c0c1f98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:27.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.099+0000 7f7c10e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c0c0d5370 0x7f7c0c0cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:27.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.099+0000 7f7c10e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c15f1b0 0x7f7c0c0c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:27.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.099+0000 7f7c10e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c15f590 con 0x7f7c0c15f1b0 2024-04-26T20:57:27.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.099+0000 7f7c10e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c0c0d3f00 con 0x7f7c0c0cf700 2024-04-26T20:57:27.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.099+0000 7f7c10e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7c0c0b4410 con 0x7f7c0c0d5370 2024-04-26T20:57:27.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.100+0000 7f7c0be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c15f1b0 0x7f7c0c0c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:27.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.100+0000 7f7c0be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c15f1b0 0x7f7c0c0c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42428/0 (socket says 172.21.15.28:42428) 2024-04-26T20:57:27.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.100+0000 7f7c0be00640 1 -- 172.21.15.28:0/1781115120 learned_addr learned my addr 172.21.15.28:0/1781115120 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:27.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.100+0000 7f7c0b400640 1 --2- 172.21.15.28:0/1781115120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0c0cf700 0x7f7c0c1f98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:27.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.100+0000 7f7c03e00640 1 --2- 172.21.15.28:0/1781115120 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c0c0d5370 0x7f7c0c0cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:27.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.100+0000 7f7c0be00640 1 -- 172.21.15.28:0/1781115120 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c0c0d5370 msgr2=0x7f7c0c0cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:27.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.100+0000 7f7c0be00640 1 --2- 172.21.15.28:0/1781115120 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c0c0d5370 0x7f7c0c0cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:27.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.100+0000 7f7c0be00640 1 -- 172.21.15.28:0/1781115120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0c0cf700 msgr2=0x7f7c0c1f98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:27.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.100+0000 7f7c0be00640 1 --2- 172.21.15.28:0/1781115120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0c0cf700 0x7f7c0c1f98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:27.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.100+0000 7f7c0be00640 1 -- 172.21.15.28:0/1781115120 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0c0c9ad0 con 0x7f7c0c15f1b0 2024-04-26T20:57:27.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.100+0000 7f7c0be00640 1 --2- 172.21.15.28:0/1781115120 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c15f1b0 0x7f7c0c0c9590 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400c7f0 tx=0x7f7c0400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:27.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.101+0000 7f7c03400640 1 -- 172.21.15.28:0/1781115120 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04018070 con 0x7f7c0c15f1b0 2024-04-26T20:57:27.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.101+0000 7f7c03400640 1 -- 172.21.15.28:0/1781115120 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c0400a5e0 con 0x7f7c0c15f1b0 2024-04-26T20:57:27.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.101+0000 7f7c03400640 1 -- 172.21.15.28:0/1781115120 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c0400aa10 con 0x7f7c0c15f1b0 2024-04-26T20:57:27.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.101+0000 7f7c10e00640 1 -- 172.21.15.28:0/1781115120 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0c0cdac0 con 0x7f7c0c15f1b0 2024-04-26T20:57:27.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.101+0000 7f7c10e00640 1 -- 172.21.15.28:0/1781115120 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c0c0ce080 con 0x7f7c0c15f1b0 2024-04-26T20:57:27.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.102+0000 7f7c10e00640 1 -- 172.21.15.28:0/1781115120 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c0c0cf290 con 0x7f7c0c15f1b0 2024-04-26T20:57:27.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.103+0000 7f7c03400640 1 -- 172.21.15.28:0/1781115120 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c0400abb0 con 0x7f7c0c15f1b0 2024-04-26T20:57:27.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.104+0000 7f7c03400640 1 --2- 172.21.15.28:0/1781115120 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7be806c8f0 0x7f7be806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:27.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.104+0000 7f7c03400640 1 -- 172.21.15.28:0/1781115120 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7c0408b1e0 con 0x7f7c0c15f1b0 2024-04-26T20:57:27.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.104+0000 7f7c0b400640 1 --2- 172.21.15.28:0/1781115120 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7be806c8f0 0x7f7be806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:27.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.105+0000 7f7c0b400640 1 --2- 172.21.15.28:0/1781115120 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7be806c8f0 0x7f7be806edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc006fd0 tx=0x7f7bfc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:27.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.108+0000 7f7c03400640 1 -- 172.21.15.28:0/1781115120 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7c0c0cf290 con 0x7f7c0c15f1b0 2024-04-26T20:57:27.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.311+0000 7f7c10e00640 1 -- 172.21.15.28:0/1781115120 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c0c06a360 con 0x7f7c0c15f1b0 2024-04-26T20:57:27.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.313+0000 7f7c03400640 1 -- 172.21.15.28:0/1781115120 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7f7c0c0d0710 con 0x7f7c0c15f1b0 2024-04-26T20:57:27.315 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:27.315 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":21,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:56:52.699663+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:57:27.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.316+0000 7f7c10e00640 1 -- 172.21.15.28:0/1781115120 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7be806c8f0 msgr2=0x7f7be806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:27.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.316+0000 7f7c10e00640 1 --2- 172.21.15.28:0/1781115120 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7be806c8f0 0x7f7be806edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc006fd0 tx=0x7f7bfc008040 comp rx=0 tx=0).stop 2024-04-26T20:57:27.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.316+0000 7f7c10e00640 1 -- 172.21.15.28:0/1781115120 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c15f1b0 msgr2=0x7f7c0c0c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:27.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.316+0000 7f7c10e00640 1 --2- 172.21.15.28:0/1781115120 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c15f1b0 0x7f7c0c0c9590 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7c0400c7f0 tx=0x7f7c0400ccb0 comp rx=0 tx=0).stop 2024-04-26T20:57:27.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.316+0000 7f7c10e00640 1 -- 172.21.15.28:0/1781115120 shutdown_connections 2024-04-26T20:57:27.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.316+0000 7f7c10e00640 1 --2- 172.21.15.28:0/1781115120 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7be806c8f0 0x7f7be806edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:27.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.316+0000 7f7c10e00640 1 --2- 172.21.15.28:0/1781115120 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c0c15f1b0 0x7f7c0c0c9590 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:27.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.316+0000 7f7c10e00640 1 --2- 172.21.15.28:0/1781115120 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c0c0d5370 0x7f7c0c0cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:27.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.316+0000 7f7c10e00640 1 --2- 172.21.15.28:0/1781115120 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c0c0cf700 0x7f7c0c1f98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:27.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.316+0000 7f7c10e00640 1 -- 172.21.15.28:0/1781115120 >> 172.21.15.28:0/1781115120 conn(0x7f7c0c06c770 msgr2=0x7f7c0c1e7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:27.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.316+0000 7f7c10e00640 1 -- 172.21.15.28:0/1781115120 shutdown_connections 2024-04-26T20:57:27.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.316+0000 7f7c10e00640 1 -- 172.21.15.28:0/1781115120 wait complete. 2024-04-26T20:57:27.337 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:56:48.688498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:56:50.117541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:57:27.337 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:57:27.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.545+0000 7f84dbe00640 1 Processor -- start 2024-04-26T20:57:27.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.546+0000 7f84dbe00640 1 -- start start 2024-04-26T20:57:27.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.548+0000 7f84dbe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f84dc0cf040 0x7f84dc0d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:27.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.548+0000 7f84dbe00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84dc0b5ba0 con 0x7f84dc0d1250 2024-04-26T20:57:27.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.548+0000 7f84dbe00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84dc0d1660 con 0x7f84dc1465c0 2024-04-26T20:57:27.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.548+0000 7f84dbe00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f84dc1469a0 con 0x7f84dc0cf040 2024-04-26T20:57:27.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.548+0000 7f84db400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f84dc0d1250 0x7f84dc0b3a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33090/0 (socket says 172.21.15.28:33090) 2024-04-26T20:57:27.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.548+0000 7f84db400640 1 -- 172.21.15.28:0/2060655957 learned_addr learned my addr 172.21.15.28:0/2060655957 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:27.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.548+0000 7f84d2000640 1 --2- 172.21.15.28:0/2060655957 >> v2:172.21.15.28:3300/0 conn(0x7f84dc0cf040 0x7f84dc0d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:27.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.549+0000 7f84da000640 1 -- 172.21.15.28:0/2060655957 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3252740673 0 0) 0x7f84d4001110 con 0x7f84dc0d1250 2024-04-26T20:57:27.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.549+0000 7f84da000640 1 -- 172.21.15.28:0/2060655957 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84c0003520 con 0x7f84dc0d1250 2024-04-26T20:57:27.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.549+0000 7f84d2000640 1 -- 172.21.15.28:0/2060655957 >> v1:172.21.15.103:6789/0 conn(0x7f84dc1465c0 legacy=0x7f84dc06e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:27.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.549+0000 7f84d2000640 1 -- 172.21.15.28:0/2060655957 >> v1:172.21.15.28:6789/0 conn(0x7f84dc0d1250 legacy=0x7f84dc0b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:27.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.549+0000 7f84d2000640 1 -- 172.21.15.28:0/2060655957 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84dc0cf520 con 0x7f84dc0cf040 2024-04-26T20:57:27.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.550+0000 7f84d2000640 1 --2- 172.21.15.28:0/2060655957 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84dc0cf040 0x7f84dc0d0d10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f84c40097a0 tx=0x7f84c402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e975551d867c74a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:27.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.551+0000 7f84da000640 1 -- 172.21.15.28:0/2060655957 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c4037070 con 0x7f84dc0cf040 2024-04-26T20:57:27.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.551+0000 7f84da000640 1 -- 172.21.15.28:0/2060655957 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84c4032440 con 0x7f84dc0cf040 2024-04-26T20:57:27.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.551+0000 7f84da000640 1 -- 172.21.15.28:0/2060655957 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c40327f0 con 0x7f84dc0cf040 2024-04-26T20:57:27.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.551+0000 7f84dbe00640 1 -- 172.21.15.28:0/2060655957 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84dc0cf040 msgr2=0x7f84dc0d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:27.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.551+0000 7f84dbe00640 1 --2- 172.21.15.28:0/2060655957 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84dc0cf040 0x7f84dc0d0d10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f84c40097a0 tx=0x7f84c402a1a0 comp rx=0 tx=0).stop 2024-04-26T20:57:27.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.551+0000 7f84dbe00640 1 -- 172.21.15.28:0/2060655957 shutdown_connections 2024-04-26T20:57:27.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.551+0000 7f84dbe00640 1 --2- 172.21.15.28:0/2060655957 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84dc0cf040 0x7f84dc0d0d10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:27.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.551+0000 7f84dbe00640 1 -- 172.21.15.28:0/2060655957 >> 172.21.15.28:0/2060655957 conn(0x7f84dc06c530 msgr2=0x7f84dc06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:27.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.551+0000 7f84dbe00640 1 -- 172.21.15.28:0/2060655957 shutdown_connections 2024-04-26T20:57:27.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.551+0000 7f84dbe00640 1 -- 172.21.15.28:0/2060655957 wait complete. 2024-04-26T20:57:27.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.553+0000 7f84dbe00640 1 Processor -- start 2024-04-26T20:57:27.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.553+0000 7f84dbe00640 1 -- start start 2024-04-26T20:57:27.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.554+0000 7f84dbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84dc0cf040 0x7f84dc148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:27.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.554+0000 7f84dbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84dc0d1250 0x7f84dc1445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.554+0000 7f84dbe00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f84dc1465c0 0x7f84dc13c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.554+0000 7f84dbe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc0cfdc0 con 0x7f84dc0d1250 2024-04-26T20:57:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.554+0000 7f84dbe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc0cfae0 con 0x7f84dc0cf040 2024-04-26T20:57:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.554+0000 7f84dbe00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f84dc0d00a0 con 0x7f84dc1465c0 2024-04-26T20:57:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.554+0000 7f84d2000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84dc0d1250 0x7f84dc1445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.554+0000 7f84d2000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84dc0d1250 0x7f84dc1445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42446/0 (socket says 172.21.15.28:42446) 2024-04-26T20:57:27.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.555+0000 7f84d2000640 1 -- 172.21.15.28:0/3451822608 learned_addr learned my addr 172.21.15.28:0/3451822608 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:27.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.555+0000 7f84d2000640 1 -- 172.21.15.28:0/3451822608 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f84dc1465c0 msgr2=0x7f84dc13c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:27.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.555+0000 7f84d2000640 1 --2- 172.21.15.28:0/3451822608 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f84dc1465c0 0x7f84dc13c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:27.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.555+0000 7f84d2000640 1 -- 172.21.15.28:0/3451822608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84dc0cf040 msgr2=0x7f84dc148690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:27.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.555+0000 7f84daa00640 1 --2- 172.21.15.28:0/3451822608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84dc0cf040 0x7f84dc148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:27.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.555+0000 7f84d2000640 1 --2- 172.21.15.28:0/3451822608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84dc0cf040 0x7f84dc148690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:27.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.555+0000 7f84d2000640 1 -- 172.21.15.28:0/3451822608 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84dc15da40 con 0x7f84dc0d1250 2024-04-26T20:57:27.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.555+0000 7f84d2000640 1 --2- 172.21.15.28:0/3451822608 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84dc0d1250 0x7f84dc1445b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f84c40097a0 tx=0x7f84c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:27.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.556+0000 7f84d3400640 1 -- 172.21.15.28:0/3451822608 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c4037050 con 0x7f84dc0d1250 2024-04-26T20:57:27.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.556+0000 7f84d3400640 1 -- 172.21.15.28:0/3451822608 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84c4043070 con 0x7f84dc0d1250 2024-04-26T20:57:27.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.556+0000 7f84d3400640 1 -- 172.21.15.28:0/3451822608 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c4006e10 con 0x7f84dc0d1250 2024-04-26T20:57:27.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.556+0000 7f84dbe00640 1 -- 172.21.15.28:0/3451822608 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84dc15eea0 con 0x7f84dc0d1250 2024-04-26T20:57:27.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.556+0000 7f84dbe00640 1 -- 172.21.15.28:0/3451822608 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84dc15f420 con 0x7f84dc0d1250 2024-04-26T20:57:27.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.557+0000 7f84dbe00640 1 -- 172.21.15.28:0/3451822608 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84dc06a250 con 0x7f84dc0d1250 2024-04-26T20:57:27.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.558+0000 7f84d3400640 1 -- 172.21.15.28:0/3451822608 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f84c4032810 con 0x7f84dc0d1250 2024-04-26T20:57:27.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.559+0000 7f84d3400640 1 --2- 172.21.15.28:0/3451822608 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f84b806c8f0 0x7f84b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:27.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.559+0000 7f84d3400640 1 -- 172.21.15.28:0/3451822608 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f84c40ac5a0 con 0x7f84dc0d1250 2024-04-26T20:57:27.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.559+0000 7f84daa00640 1 --2- 172.21.15.28:0/3451822608 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f84b806c8f0 0x7f84b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:27.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.560+0000 7f84daa00640 1 --2- 172.21.15.28:0/3451822608 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f84b806c8f0 0x7f84b806edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f84cc007fd0 tx=0x7f84cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:27.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.563+0000 7f84d3400640 1 -- 172.21.15.28:0/3451822608 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f84dc15da40 con 0x7f84dc0d1250 2024-04-26T20:57:27.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.759+0000 7f84dbe00640 1 -- 172.21.15.28:0/3451822608 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f84dc0b5580 con 0x7f84dc0d1250 2024-04-26T20:57:27.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.760+0000 7f84d3400640 1 -- 172.21.15.28:0/3451822608 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f84c407a9c0 con 0x7f84dc0d1250 2024-04-26T20:57:27.761 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:57:27.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.763+0000 7f84dbe00640 1 -- 172.21.15.28:0/3451822608 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f84b806c8f0 msgr2=0x7f84b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:27.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.763+0000 7f84dbe00640 1 --2- 172.21.15.28:0/3451822608 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f84b806c8f0 0x7f84b806edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f84cc007fd0 tx=0x7f84cc009040 comp rx=0 tx=0).stop 2024-04-26T20:57:27.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.763+0000 7f84dbe00640 1 -- 172.21.15.28:0/3451822608 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84dc0d1250 msgr2=0x7f84dc1445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:27.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.763+0000 7f84dbe00640 1 --2- 172.21.15.28:0/3451822608 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84dc0d1250 0x7f84dc1445b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f84c40097a0 tx=0x7f84c40046c0 comp rx=0 tx=0).stop 2024-04-26T20:57:27.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.763+0000 7f84dbe00640 1 -- 172.21.15.28:0/3451822608 shutdown_connections 2024-04-26T20:57:27.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.763+0000 7f84dbe00640 1 --2- 172.21.15.28:0/3451822608 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f84b806c8f0 0x7f84b806edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:27.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.763+0000 7f84dbe00640 1 --2- 172.21.15.28:0/3451822608 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f84dc1465c0 0x7f84dc13c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:27.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.763+0000 7f84dbe00640 1 --2- 172.21.15.28:0/3451822608 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84dc0d1250 0x7f84dc1445b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:27.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.763+0000 7f84dbe00640 1 --2- 172.21.15.28:0/3451822608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84dc0cf040 0x7f84dc148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:27.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.763+0000 7f84dbe00640 1 -- 172.21.15.28:0/3451822608 >> 172.21.15.28:0/3451822608 conn(0x7f84dc06c530 msgr2=0x7f84dc0b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:27.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.763+0000 7f84dbe00640 1 -- 172.21.15.28:0/3451822608 shutdown_connections 2024-04-26T20:57:27.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.763+0000 7f84dbe00640 1 -- 172.21.15.28:0/3451822608 wait complete. 2024-04-26T20:57:27.782 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:57:27.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.995+0000 7f1015e00640 1 Processor -- start 2024-04-26T20:57:27.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.995+0000 7f1015e00640 1 -- start start 2024-04-26T20:57:27.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.997+0000 7f1015e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f10100730b0 0x7f1010166200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:27.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.997+0000 7f1015e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10100b45d0 con 0x7f101015b350 2024-04-26T20:57:27.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.997+0000 7f1015e00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10100b48b0 con 0x7f101015b730 2024-04-26T20:57:27.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.997+0000 7f1015e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f101006e150 con 0x7f10100730b0 2024-04-26T20:57:27.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.997+0000 7f1014a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f101015b350 0x7f1010073c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33092/0 (socket says 172.21.15.28:33092) 2024-04-26T20:57:27.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.997+0000 7f1014a00640 1 -- 172.21.15.28:0/2996954174 learned_addr learned my addr 172.21.15.28:0/2996954174 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:27.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.997+0000 7f1015400640 1 --2- 172.21.15.28:0/2996954174 >> v2:172.21.15.28:3300/0 conn(0x7f10100730b0 0x7f1010166200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:27.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.998+0000 7f100fe00640 1 -- 172.21.15.28:0/2996954174 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3397831101 0 0) 0x7f10100b45d0 con 0x7f101015b350 2024-04-26T20:57:27.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.998+0000 7f100fe00640 1 -- 172.21.15.28:0/2996954174 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ff4003520 con 0x7f101015b350 2024-04-26T20:57:27.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.998+0000 7f1015400640 1 -- 172.21.15.28:0/2996954174 >> v1:172.21.15.103:6789/0 conn(0x7f101015b730 legacy=0x7f10100729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:27.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.998+0000 7f1015400640 1 -- 172.21.15.28:0/2996954174 >> v1:172.21.15.28:6789/0 conn(0x7f101015b350 legacy=0x7f1010073c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:27.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.998+0000 7f1015400640 1 -- 172.21.15.28:0/2996954174 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10100b4b90 con 0x7f10100730b0 2024-04-26T20:57:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.999+0000 7f1015400640 1 --2- 172.21.15.28:0/2996954174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10100730b0 0x7f1010166200 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1008009d90 tx=0x7f100802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a718e0d80cc86 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.999+0000 7f100fe00640 1 -- 172.21.15.28:0/2996954174 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1008007740 con 0x7f10100730b0 2024-04-26T20:57:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.999+0000 7f100fe00640 1 -- 172.21.15.28:0/2996954174 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10080078e0 con 0x7f10100730b0 2024-04-26T20:57:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:27.999+0000 7f100fe00640 1 -- 172.21.15.28:0/2996954174 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1008007cb0 con 0x7f10100730b0 2024-04-26T20:57:28.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.000+0000 7f1015e00640 1 -- 172.21.15.28:0/2996954174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10100730b0 msgr2=0x7f1010166200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:28.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.000+0000 7f1015e00640 1 --2- 172.21.15.28:0/2996954174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10100730b0 0x7f1010166200 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1008009d90 tx=0x7f100802a6a0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.000+0000 7f1015e00640 1 -- 172.21.15.28:0/2996954174 shutdown_connections 2024-04-26T20:57:28.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.000+0000 7f1015e00640 1 --2- 172.21.15.28:0/2996954174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10100730b0 0x7f1010166200 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.000+0000 7f1015e00640 1 -- 172.21.15.28:0/2996954174 >> 172.21.15.28:0/2996954174 conn(0x7f101006c530 msgr2=0x7f101006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:28.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.000+0000 7f1015e00640 1 -- 172.21.15.28:0/2996954174 shutdown_connections 2024-04-26T20:57:28.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.000+0000 7f1015e00640 1 -- 172.21.15.28:0/2996954174 wait complete. 2024-04-26T20:57:28.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.002+0000 7f1015e00640 1 Processor -- start 2024-04-26T20:57:28.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.002+0000 7f1015e00640 1 -- start start 2024-04-26T20:57:28.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.003+0000 7f1015e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10100730b0 0x7f1010147710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:28.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.003+0000 7f1015e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f101015b350 0x7f1010169f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:28.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.003+0000 7f1015e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1010144a10 0x7f1010144ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:28.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.003+0000 7f1015e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f101006f5f0 con 0x7f1010144a10 2024-04-26T20:57:28.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.003+0000 7f1015e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1010073840 con 0x7f10100730b0 2024-04-26T20:57:28.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.003+0000 7f1015e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f101006f770 con 0x7f101015b350 2024-04-26T20:57:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.003+0000 7f1015400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1010144a10 0x7f1010144ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.003+0000 7f1015400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1010144a10 0x7f1010144ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42476/0 (socket says 172.21.15.28:42476) 2024-04-26T20:57:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.003+0000 7f1015400640 1 -- 172.21.15.28:0/3306357426 learned_addr learned my addr 172.21.15.28:0/3306357426 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.004+0000 7f1014a00640 1 --2- 172.21.15.28:0/3306357426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10100730b0 0x7f1010147710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.004+0000 7f1007e00640 1 --2- 172.21.15.28:0/3306357426 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f101015b350 0x7f1010169f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.004+0000 7f1015400640 1 -- 172.21.15.28:0/3306357426 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f101015b350 msgr2=0x7f1010169f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.004+0000 7f1015400640 1 --2- 172.21.15.28:0/3306357426 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f101015b350 0x7f1010169f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.004+0000 7f1015400640 1 -- 172.21.15.28:0/3306357426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10100730b0 msgr2=0x7f1010147710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.004+0000 7f1015400640 1 --2- 172.21.15.28:0/3306357426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10100730b0 0x7f1010147710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.004+0000 7f1015400640 1 -- 172.21.15.28:0/3306357426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101016a490 con 0x7f1010144a10 2024-04-26T20:57:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.004+0000 7f1015400640 1 --2- 172.21.15.28:0/3306357426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1010144a10 0x7f1010144ea0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1008009eb0 tx=0x7f1008004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.004+0000 7f100d600640 1 -- 172.21.15.28:0/3306357426 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100803f070 con 0x7f1010144a10 2024-04-26T20:57:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.005+0000 7f100d600640 1 -- 172.21.15.28:0/3306357426 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1008031bc0 con 0x7f1010144a10 2024-04-26T20:57:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.005+0000 7f100d600640 1 -- 172.21.15.28:0/3306357426 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1008009450 con 0x7f1010144a10 2024-04-26T20:57:28.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.005+0000 7f1015e00640 1 -- 172.21.15.28:0/3306357426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f101016a710 con 0x7f1010144a10 2024-04-26T20:57:28.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.005+0000 7f1015e00640 1 -- 172.21.15.28:0/3306357426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f101016acd0 con 0x7f1010144a10 2024-04-26T20:57:28.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.006+0000 7f1015e00640 1 -- 172.21.15.28:0/3306357426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f101006a250 con 0x7f1010144a10 2024-04-26T20:57:28.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.007+0000 7f100d600640 1 -- 172.21.15.28:0/3306357426 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f100803b410 con 0x7f1010144a10 2024-04-26T20:57:28.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.008+0000 7f100d600640 1 --2- 172.21.15.28:0/3306357426 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0fec06c940 0x7f0fec06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:28.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.008+0000 7f100d600640 1 -- 172.21.15.28:0/3306357426 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f10080acf70 con 0x7f1010144a10 2024-04-26T20:57:28.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.008+0000 7f1014a00640 1 --2- 172.21.15.28:0/3306357426 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0fec06c940 0x7f0fec06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:28.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.009+0000 7f1014a00640 1 --2- 172.21.15.28:0/3306357426 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0fec06c940 0x7f0fec06ee00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1010169ed0 tx=0x7f1000009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:28.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.012+0000 7f100d600640 1 -- 172.21.15.28:0/3306357426 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f101016a490 con 0x7f1010144a10 2024-04-26T20:57:28.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.219+0000 7f1015e00640 1 -- 172.21.15.28:0/3306357426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f10100d08b0 con 0x7f1010144a10 2024-04-26T20:57:28.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.220+0000 7f100d600640 1 -- 172.21.15.28:0/3306357426 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f100807b380 con 0x7f1010144a10 2024-04-26T20:57:28.221 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-04-26T20:57:28.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.223+0000 7f1015e00640 1 -- 172.21.15.28:0/3306357426 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0fec06c940 msgr2=0x7f0fec06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:28.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.223+0000 7f1015e00640 1 --2- 172.21.15.28:0/3306357426 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0fec06c940 0x7f0fec06ee00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1010169ed0 tx=0x7f1000009040 comp rx=0 tx=0).stop 2024-04-26T20:57:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.223+0000 7f1015e00640 1 -- 172.21.15.28:0/3306357426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1010144a10 msgr2=0x7f1010144ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.223+0000 7f1015e00640 1 --2- 172.21.15.28:0/3306357426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1010144a10 0x7f1010144ea0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1008009eb0 tx=0x7f1008004a40 comp rx=0 tx=0).stop 2024-04-26T20:57:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.223+0000 7f1015e00640 1 -- 172.21.15.28:0/3306357426 shutdown_connections 2024-04-26T20:57:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.223+0000 7f1015e00640 1 --2- 172.21.15.28:0/3306357426 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0fec06c940 0x7f0fec06ee00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.223+0000 7f1015e00640 1 --2- 172.21.15.28:0/3306357426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1010144a10 0x7f1010144ea0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.223+0000 7f1015e00640 1 --2- 172.21.15.28:0/3306357426 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f101015b350 0x7f1010169f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.223+0000 7f1015e00640 1 --2- 172.21.15.28:0/3306357426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10100730b0 0x7f1010147710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.223+0000 7f1015e00640 1 -- 172.21.15.28:0/3306357426 >> 172.21.15.28:0/3306357426 conn(0x7f101006c530 msgr2=0x7f10100b2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.223+0000 7f1015e00640 1 -- 172.21.15.28:0/3306357426 shutdown_connections 2024-04-26T20:57:28.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.223+0000 7f1015e00640 1 -- 172.21.15.28:0/3306357426 wait complete. 2024-04-26T20:57:28.245 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:57:28.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.454+0000 7fa672c00640 1 Processor -- start 2024-04-26T20:57:28.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.455+0000 7fa672c00640 1 -- start start 2024-04-26T20:57:28.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.456+0000 7fa672c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa66c146860 0x7fa66c06f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:28.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.456+0000 7fa672c00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa66c06f810 0x7fa66c0b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:28.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.456+0000 7fa672c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa66c0b4560 0x7fa66c0b5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:28.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.456+0000 7fa672c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa66c06fc70 con 0x7fa66c0b4560 2024-04-26T20:57:28.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.456+0000 7fa672c00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fa66c0b4d90 con 0x7fa66c06f810 2024-04-26T20:57:28.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.456+0000 7fa672c00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa66c15d850 con 0x7fa66c146860 2024-04-26T20:57:28.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.457+0000 7fa670c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa66c0b4560 0x7fa66c0b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:28.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.457+0000 7fa670c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa66c0b4560 0x7fa66c0b5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42478/0 (socket says 172.21.15.28:42478) 2024-04-26T20:57:28.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.457+0000 7fa670c00640 1 -- 172.21.15.28:0/1118964129 learned_addr learned my addr 172.21.15.28:0/1118964129 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:28.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.457+0000 7fa66b400640 1 --2- 172.21.15.28:0/1118964129 >> v2:172.21.15.103:3300/0 conn(0x7fa66c06f810 0x7fa66c0b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:28.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.457+0000 7fa66be00640 1 --2- 172.21.15.28:0/1118964129 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa66c146860 0x7fa66c06f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:28.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.457+0000 7fa670c00640 1 -- 172.21.15.28:0/1118964129 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa66c146860 msgr2=0x7fa66c06f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:28.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.457+0000 7fa670c00640 1 --2- 172.21.15.28:0/1118964129 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa66c146860 0x7fa66c06f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.457+0000 7fa670c00640 1 -- 172.21.15.28:0/1118964129 >> v2:172.21.15.103:3300/0 conn(0x7fa66c06f810 msgr2=0x7fa66c0b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:28.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.457+0000 7fa670c00640 1 --2- 172.21.15.28:0/1118964129 >> v2:172.21.15.103:3300/0 conn(0x7fa66c06f810 0x7fa66c0b4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.457+0000 7fa670c00640 1 -- 172.21.15.28:0/1118964129 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66c15db30 con 0x7fa66c0b4560 2024-04-26T20:57:28.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.458+0000 7fa670c00640 1 --2- 172.21.15.28:0/1118964129 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa66c0b4560 0x7fa66c0b5440 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa66c071c90 tx=0x7fa66402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b4c2c7dbedb13d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:28.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.459+0000 7fa66aa00640 1 -- 172.21.15.28:0/1118964129 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa664037070 con 0x7fa66c0b4560 2024-04-26T20:57:28.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.459+0000 7fa66aa00640 1 -- 172.21.15.28:0/1118964129 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa664032440 con 0x7fa66c0b4560 2024-04-26T20:57:28.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.459+0000 7fa66aa00640 1 -- 172.21.15.28:0/1118964129 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa664032810 con 0x7fa66c0b4560 2024-04-26T20:57:28.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.459+0000 7fa672c00640 1 -- 172.21.15.28:0/1118964129 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa66c0b4560 msgr2=0x7fa66c0b5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:28.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.459+0000 7fa672c00640 1 --2- 172.21.15.28:0/1118964129 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa66c0b4560 0x7fa66c0b5440 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa66c071c90 tx=0x7fa66402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.460+0000 7fa672c00640 1 -- 172.21.15.28:0/1118964129 shutdown_connections 2024-04-26T20:57:28.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.460+0000 7fa672c00640 1 --2- 172.21.15.28:0/1118964129 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa66c0b4560 0x7fa66c0b5440 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.460+0000 7fa672c00640 1 --2- 172.21.15.28:0/1118964129 >> v2:172.21.15.103:3300/0 conn(0x7fa66c06f810 0x7fa66c0b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.460+0000 7fa672c00640 1 --2- 172.21.15.28:0/1118964129 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa66c146860 0x7fa66c06f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.460+0000 7fa672c00640 1 -- 172.21.15.28:0/1118964129 >> 172.21.15.28:0/1118964129 conn(0x7fa66c0b2b60 msgr2=0x7fa66c0b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:28.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.460+0000 7fa672c00640 1 -- 172.21.15.28:0/1118964129 shutdown_connections 2024-04-26T20:57:28.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.460+0000 7fa672c00640 1 -- 172.21.15.28:0/1118964129 wait complete. 2024-04-26T20:57:28.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.461+0000 7fa672c00640 1 Processor -- start 2024-04-26T20:57:28.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.462+0000 7fa672c00640 1 -- start start 2024-04-26T20:57:28.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.462+0000 7fa672c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa66c06f810 0x7fa66c13b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:28.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.462+0000 7fa672c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa66c0b4560 0x7fa66c13d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:28.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.462+0000 7fa672c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa66c146860 0x7fa66c15e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:28.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.462+0000 7fa672c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c0b4b70 con 0x7fa66c06f810 2024-04-26T20:57:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.462+0000 7fa672c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c0b49f0 con 0x7fa66c0b4560 2024-04-26T20:57:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.462+0000 7fa672c00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa66c0b5980 con 0x7fa66c146860 2024-04-26T20:57:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.463+0000 7fa66be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa66c06f810 0x7fa66c13b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.463+0000 7fa66be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa66c06f810 0x7fa66c13b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42482/0 (socket says 172.21.15.28:42482) 2024-04-26T20:57:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.463+0000 7fa66be00640 1 -- 172.21.15.28:0/2995440202 learned_addr learned my addr 172.21.15.28:0/2995440202 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.463+0000 7fa66b400640 1 --2- 172.21.15.28:0/2995440202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa66c0b4560 0x7fa66c13d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.463+0000 7fa670c00640 1 --2- 172.21.15.28:0/2995440202 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa66c146860 0x7fa66c15e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.463+0000 7fa66be00640 1 -- 172.21.15.28:0/2995440202 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa66c146860 msgr2=0x7fa66c15e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.463+0000 7fa66be00640 1 --2- 172.21.15.28:0/2995440202 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa66c146860 0x7fa66c15e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.463+0000 7fa66be00640 1 -- 172.21.15.28:0/2995440202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa66c0b4560 msgr2=0x7fa66c13d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.463+0000 7fa66be00640 1 --2- 172.21.15.28:0/2995440202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa66c0b4560 0x7fa66c13d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.463+0000 7fa66be00640 1 -- 172.21.15.28:0/2995440202 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66c13ddf0 con 0x7fa66c06f810 2024-04-26T20:57:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.463+0000 7fa670c00640 1 --2- 172.21.15.28:0/2995440202 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa66c146860 0x7fa66c15e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:28.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.464+0000 7fa66be00640 1 --2- 172.21.15.28:0/2995440202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa66c06f810 0x7fa66c13b450 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00c7f0 tx=0x7fa65c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:28.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.464+0000 7fa663e00640 1 -- 172.21.15.28:0/2995440202 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c015070 con 0x7fa66c06f810 2024-04-26T20:57:28.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.464+0000 7fa663e00640 1 -- 172.21.15.28:0/2995440202 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa65c004220 con 0x7fa66c06f810 2024-04-26T20:57:28.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.464+0000 7fa663e00640 1 -- 172.21.15.28:0/2995440202 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c007620 con 0x7fa66c06f810 2024-04-26T20:57:28.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.464+0000 7fa672c00640 1 -- 172.21.15.28:0/2995440202 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa66c13e0d0 con 0x7fa66c06f810 2024-04-26T20:57:28.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.464+0000 7fa672c00640 1 -- 172.21.15.28:0/2995440202 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa66c13e690 con 0x7fa66c06f810 2024-04-26T20:57:28.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.465+0000 7fa672c00640 1 -- 172.21.15.28:0/2995440202 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa66c076520 con 0x7fa66c06f810 2024-04-26T20:57:28.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.466+0000 7fa663e00640 1 -- 172.21.15.28:0/2995440202 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa65c0079d0 con 0x7fa66c06f810 2024-04-26T20:57:28.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.467+0000 7fa663e00640 1 --2- 172.21.15.28:0/2995440202 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa64806c8f0 0x7fa64806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:28.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.467+0000 7fa663e00640 1 -- 172.21.15.28:0/2995440202 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa65c08be90 con 0x7fa66c06f810 2024-04-26T20:57:28.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.468+0000 7fa66b400640 1 --2- 172.21.15.28:0/2995440202 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa64806c8f0 0x7fa64806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:28.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.469+0000 7fa66b400640 1 --2- 172.21.15.28:0/2995440202 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa64806c8f0 0x7fa64806edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa6540070b0 tx=0x7fa654007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:28.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.472+0000 7fa663e00640 1 -- 172.21.15.28:0/2995440202 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa65c05a220 con 0x7fa66c06f810 2024-04-26T20:57:28.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.674+0000 7fa672c00640 1 -- 172.21.15.28:0/2995440202 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa66c15d260 con 0x7fa66c06f810 2024-04-26T20:57:28.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.675+0000 7fa663e00640 1 -- 172.21.15.28:0/2995440202 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fa65c08d0a0 con 0x7fa66c06f810 2024-04-26T20:57:28.676 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-04-26T20:57:28.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.678+0000 7fa672c00640 1 -- 172.21.15.28:0/2995440202 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa64806c8f0 msgr2=0x7fa64806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:28.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.678+0000 7fa672c00640 1 --2- 172.21.15.28:0/2995440202 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa64806c8f0 0x7fa64806edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa6540070b0 tx=0x7fa654007040 comp rx=0 tx=0).stop 2024-04-26T20:57:28.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.678+0000 7fa672c00640 1 -- 172.21.15.28:0/2995440202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa66c06f810 msgr2=0x7fa66c13b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:28.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.678+0000 7fa672c00640 1 --2- 172.21.15.28:0/2995440202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa66c06f810 0x7fa66c13b450 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00c7f0 tx=0x7fa65c00ccb0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.678+0000 7fa672c00640 1 -- 172.21.15.28:0/2995440202 shutdown_connections 2024-04-26T20:57:28.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.678+0000 7fa672c00640 1 --2- 172.21.15.28:0/2995440202 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa64806c8f0 0x7fa64806edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.678+0000 7fa672c00640 1 --2- 172.21.15.28:0/2995440202 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa66c146860 0x7fa66c15e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.678+0000 7fa672c00640 1 --2- 172.21.15.28:0/2995440202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa66c0b4560 0x7fa66c13d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.678+0000 7fa672c00640 1 --2- 172.21.15.28:0/2995440202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa66c06f810 0x7fa66c13b450 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.678+0000 7fa672c00640 1 -- 172.21.15.28:0/2995440202 >> 172.21.15.28:0/2995440202 conn(0x7fa66c0b2b60 msgr2=0x7fa66c0b5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:28.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.678+0000 7fa672c00640 1 -- 172.21.15.28:0/2995440202 shutdown_connections 2024-04-26T20:57:28.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.678+0000 7fa672c00640 1 -- 172.21.15.28:0/2995440202 wait complete. 2024-04-26T20:57:28.724 INFO:tasks.cephfs_test_runner:Starting test: test_basic (tasks.cephfs.test_dump_tree.TestDumpTree) 2024-04-26T20:57:28.724 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:57:28.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.937+0000 7f122f600640 1 Processor -- start 2024-04-26T20:57:28.938 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.937+0000 7f122f600640 1 -- start start 2024-04-26T20:57:28.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.939+0000 7f122f600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1228146980 0x7f12280b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:28.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.939+0000 7f122f600640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f12280740e0 0x7f12280b6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:28.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.939+0000 7f122f600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f122815d5b0 0x7f122815da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:28.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.939+0000 7f122f600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f122815e510 con 0x7f122815d5b0 2024-04-26T20:57:28.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.939+0000 7f122f600640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f12280d0a30 con 0x7f12280740e0 2024-04-26T20:57:28.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.939+0000 7f122f600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f122815df80 con 0x7f1228146980 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.939+0000 7f122d600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f122815d5b0 0x7f122815da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.939+0000 7f122d600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f122815d5b0 0x7f122815da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42498/0 (socket says 172.21.15.28:42498) 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.939+0000 7f122d600640 1 -- 172.21.15.28:0/1384832263 learned_addr learned my addr 172.21.15.28:0/1384832263 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.940+0000 7f122cc00640 1 --2- 172.21.15.28:0/1384832263 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1228146980 0x7f12280b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.940+0000 7f1227e00640 1 --2- 172.21.15.28:0/1384832263 >> v2:172.21.15.103:3300/0 conn(0x7f12280740e0 0x7f12280b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.940+0000 7f122d600640 1 -- 172.21.15.28:0/1384832263 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1228146980 msgr2=0x7f12280b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.940+0000 7f122d600640 1 --2- 172.21.15.28:0/1384832263 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1228146980 0x7f12280b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.940+0000 7f122d600640 1 -- 172.21.15.28:0/1384832263 >> v2:172.21.15.103:3300/0 conn(0x7f12280740e0 msgr2=0x7f12280b6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:28.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.940+0000 7f122d600640 1 --2- 172.21.15.28:0/1384832263 >> v2:172.21.15.103:3300/0 conn(0x7f12280740e0 0x7f12280b6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.940+0000 7f122d600640 1 -- 172.21.15.28:0/1384832263 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122815e230 con 0x7f122815d5b0 2024-04-26T20:57:28.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.941+0000 7f122d600640 1 --2- 172.21.15.28:0/1384832263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f122815d5b0 0x7f122815da40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f12280b3ab0 tx=0x7f122002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24953a50effaf73d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:28.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.941+0000 7f1227400640 1 -- 172.21.15.28:0/1384832263 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220036070 con 0x7f122815d5b0 2024-04-26T20:57:28.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.942+0000 7f1227400640 1 -- 172.21.15.28:0/1384832263 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1220031600 con 0x7f122815d5b0 2024-04-26T20:57:28.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.942+0000 7f1227400640 1 -- 172.21.15.28:0/1384832263 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12200319d0 con 0x7f122815d5b0 2024-04-26T20:57:28.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.942+0000 7f122f600640 1 -- 172.21.15.28:0/1384832263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f122815d5b0 msgr2=0x7f122815da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:28.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.942+0000 7f122f600640 1 --2- 172.21.15.28:0/1384832263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f122815d5b0 0x7f122815da40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f12280b3ab0 tx=0x7f122002a610 comp rx=0 tx=0).stop 2024-04-26T20:57:28.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.942+0000 7f122f600640 1 -- 172.21.15.28:0/1384832263 shutdown_connections 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.942+0000 7f122f600640 1 --2- 172.21.15.28:0/1384832263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f122815d5b0 0x7f122815da40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.942+0000 7f122f600640 1 --2- 172.21.15.28:0/1384832263 >> v2:172.21.15.103:3300/0 conn(0x7f12280740e0 0x7f12280b6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.942+0000 7f122f600640 1 --2- 172.21.15.28:0/1384832263 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1228146980 0x7f12280b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.942+0000 7f122f600640 1 -- 172.21.15.28:0/1384832263 >> 172.21.15.28:0/1384832263 conn(0x7f122806e220 msgr2=0x7f122806e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.942+0000 7f122f600640 1 -- 172.21.15.28:0/1384832263 shutdown_connections 2024-04-26T20:57:28.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.942+0000 7f122f600640 1 -- 172.21.15.28:0/1384832263 wait complete. 2024-04-26T20:57:28.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.944+0000 7f122f600640 1 Processor -- start 2024-04-26T20:57:28.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.944+0000 7f122f600640 1 -- start start 2024-04-26T20:57:28.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.945+0000 7f122f600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f12280740e0 0x7f1228143e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:28.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.945+0000 7f122f600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1228146980 0x7f12281397d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:28.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.945+0000 7f122f600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f122815d5b0 0x7f1228139f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:28.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.945+0000 7f122f600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1228073bc0 con 0x7f1228146980 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.945+0000 7f122f600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f12280738e0 con 0x7f122815d5b0 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.945+0000 7f122f600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f1228160bd0 con 0x7f12280740e0 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.946+0000 7f1227e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1228146980 0x7f12281397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.946+0000 7f1227e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1228146980 0x7f12281397d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42508/0 (socket says 172.21.15.28:42508) 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.946+0000 7f1227e00640 1 -- 172.21.15.28:0/1581750694 learned_addr learned my addr 172.21.15.28:0/1581750694 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.946+0000 7f122d600640 1 --2- 172.21.15.28:0/1581750694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f122815d5b0 0x7f1228139f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.946+0000 7f122cc00640 1 --2- 172.21.15.28:0/1581750694 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f12280740e0 0x7f1228143e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.946+0000 7f1227e00640 1 -- 172.21.15.28:0/1581750694 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f12280740e0 msgr2=0x7f1228143e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.946+0000 7f1227e00640 1 --2- 172.21.15.28:0/1581750694 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f12280740e0 0x7f1228143e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.946+0000 7f1227e00640 1 -- 172.21.15.28:0/1581750694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f122815d5b0 msgr2=0x7f1228139f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.946+0000 7f1227e00640 1 --2- 172.21.15.28:0/1581750694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f122815d5b0 0x7f1228139f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.946+0000 7f1227e00640 1 -- 172.21.15.28:0/1581750694 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1228144440 con 0x7f1228146980 2024-04-26T20:57:28.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.946+0000 7f122d600640 1 --2- 172.21.15.28:0/1581750694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f122815d5b0 0x7f1228139f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:28.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.947+0000 7f1227e00640 1 --2- 172.21.15.28:0/1581750694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1228146980 0x7f12281397d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f121400d800 tx=0x7f121400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:28.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.947+0000 7f1224c00640 1 -- 172.21.15.28:0/1581750694 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1214018020 con 0x7f1228146980 2024-04-26T20:57:28.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.947+0000 7f1224c00640 1 -- 172.21.15.28:0/1581750694 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12140096a0 con 0x7f1228146980 2024-04-26T20:57:28.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.947+0000 7f1224c00640 1 -- 172.21.15.28:0/1581750694 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1214009a70 con 0x7f1228146980 2024-04-26T20:57:28.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.947+0000 7f122f600640 1 -- 172.21.15.28:0/1581750694 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1228144720 con 0x7f1228146980 2024-04-26T20:57:28.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.947+0000 7f122f600640 1 -- 172.21.15.28:0/1581750694 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1228144ce0 con 0x7f1228146980 2024-04-26T20:57:28.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.949+0000 7f122f600640 1 -- 172.21.15.28:0/1581750694 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f122806a400 con 0x7f1228146980 2024-04-26T20:57:28.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.950+0000 7f1224c00640 1 -- 172.21.15.28:0/1581750694 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f121401b020 con 0x7f1228146980 2024-04-26T20:57:28.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.950+0000 7f1224c00640 1 --2- 172.21.15.28:0/1581750694 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f120806c940 0x7f120806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:28.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.951+0000 7f1224c00640 1 -- 172.21.15.28:0/1581750694 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f121408bd70 con 0x7f1228146980 2024-04-26T20:57:28.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.951+0000 7f122cc00640 1 --2- 172.21.15.28:0/1581750694 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f120806c940 0x7f120806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:28.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.952+0000 7f122cc00640 1 --2- 172.21.15.28:0/1581750694 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f120806c940 0x7f120806ee00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f121c006fd0 tx=0x7f121c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:28.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:28.955+0000 7f1224c00640 1 -- 172.21.15.28:0/1581750694 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1214022030 con 0x7f1228146980 2024-04-26T20:57:29.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.158+0000 7f122f600640 1 -- 172.21.15.28:0/1581750694 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f122806fad0 con 0x7f1228146980 2024-04-26T20:57:29.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.160+0000 7f1224c00640 1 -- 172.21.15.28:0/1581750694 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7f121405a210 con 0x7f1228146980 2024-04-26T20:57:29.162 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:29.162 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":21,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:56:52.699663+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:57:29.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.163+0000 7f122f600640 1 -- 172.21.15.28:0/1581750694 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f120806c940 msgr2=0x7f120806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:29.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.163+0000 7f122f600640 1 --2- 172.21.15.28:0/1581750694 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f120806c940 0x7f120806ee00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f121c006fd0 tx=0x7f121c008040 comp rx=0 tx=0).stop 2024-04-26T20:57:29.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.163+0000 7f122f600640 1 -- 172.21.15.28:0/1581750694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1228146980 msgr2=0x7f12281397d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:29.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.163+0000 7f122f600640 1 --2- 172.21.15.28:0/1581750694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1228146980 0x7f12281397d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f121400d800 tx=0x7f121400dcc0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.163+0000 7f122f600640 1 -- 172.21.15.28:0/1581750694 shutdown_connections 2024-04-26T20:57:29.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.163+0000 7f122f600640 1 --2- 172.21.15.28:0/1581750694 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f120806c940 0x7f120806ee00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.163+0000 7f122f600640 1 --2- 172.21.15.28:0/1581750694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f122815d5b0 0x7f1228139f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.163+0000 7f122f600640 1 --2- 172.21.15.28:0/1581750694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1228146980 0x7f12281397d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.163+0000 7f122f600640 1 --2- 172.21.15.28:0/1581750694 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f12280740e0 0x7f1228143e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.163+0000 7f122f600640 1 -- 172.21.15.28:0/1581750694 >> 172.21.15.28:0/1581750694 conn(0x7f122806e220 msgr2=0x7f122815b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:29.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.163+0000 7f122f600640 1 -- 172.21.15.28:0/1581750694 shutdown_connections 2024-04-26T20:57:29.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.163+0000 7f122f600640 1 -- 172.21.15.28:0/1581750694 wait complete. 2024-04-26T20:57:29.185 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:56:48.688498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:56:50.117541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:57:29.186 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:57:29.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.405+0000 7f4a8a400640 1 Processor -- start 2024-04-26T20:57:29.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.405+0000 7f4a8a400640 1 -- start start 2024-04-26T20:57:29.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.407+0000 7f4a8a400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4a84075540 0x7f4a840b5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:29.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.407+0000 7f4a8a400640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f4a841468f0 0x7f4a840ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:29.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.407+0000 7f4a8a400640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a840b5210 con 0x7f4a84075920 2024-04-26T20:57:29.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.407+0000 7f4a8a400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f4a840b54b0 con 0x7f4a84075540 2024-04-26T20:57:29.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.407+0000 7f4a8a400640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f4a840b5790 con 0x7f4a841468f0 2024-04-26T20:57:29.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.407+0000 7f4a89000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4a84075540 0x7f4a840b5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:29.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.407+0000 7f4a89000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4a84075540 0x7f4a840b5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42514/0 (socket says 172.21.15.28:42514) 2024-04-26T20:57:29.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.407+0000 7f4a89000640 1 -- 172.21.15.28:0/3930649214 learned_addr learned my addr 172.21.15.28:0/3930649214 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:29.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.408+0000 7f4a89000640 1 -- 172.21.15.28:0/3930649214 >> v2:172.21.15.103:3300/0 conn(0x7f4a841468f0 msgr2=0x7f4a840ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:29.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.408+0000 7f4a89a00640 1 --2- 172.21.15.28:0/3930649214 >> v2:172.21.15.103:3300/0 conn(0x7f4a841468f0 0x7f4a840ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:29.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.408+0000 7f4a89000640 1 --2- 172.21.15.28:0/3930649214 >> v2:172.21.15.103:3300/0 conn(0x7f4a841468f0 0x7f4a840ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.408+0000 7f4a89000640 1 -- 172.21.15.28:0/3930649214 >> v1:172.21.15.103:6789/0 conn(0x7f4a84075920 legacy=0x7f4a841461e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:29.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.408+0000 7f4a89000640 1 -- 172.21.15.28:0/3930649214 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a840b5a70 con 0x7f4a84075540 2024-04-26T20:57:29.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.408+0000 7f4a89a00640 1 --2- 172.21.15.28:0/3930649214 >> v2:172.21.15.103:3300/0 conn(0x7f4a841468f0 0x7f4a840ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:29.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.409+0000 7f4a89000640 1 --2- 172.21.15.28:0/3930649214 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a84075540 0x7f4a840b5f90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4a840b7290 tx=0x7f4a7402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9785efd959438d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:29.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.409+0000 7f4a83e00640 1 -- 172.21.15.28:0/3930649214 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74037070 con 0x7f4a84075540 2024-04-26T20:57:29.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.409+0000 7f4a83e00640 1 -- 172.21.15.28:0/3930649214 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a74032440 con 0x7f4a84075540 2024-04-26T20:57:29.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.409+0000 7f4a83e00640 1 -- 172.21.15.28:0/3930649214 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74032810 con 0x7f4a84075540 2024-04-26T20:57:29.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.410+0000 7f4a8a400640 1 -- 172.21.15.28:0/3930649214 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a84075540 msgr2=0x7f4a840b5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:29.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.410+0000 7f4a8a400640 1 --2- 172.21.15.28:0/3930649214 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a84075540 0x7f4a840b5f90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4a840b7290 tx=0x7f4a7402a210 comp rx=0 tx=0).stop 2024-04-26T20:57:29.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.410+0000 7f4a8a400640 1 -- 172.21.15.28:0/3930649214 shutdown_connections 2024-04-26T20:57:29.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.410+0000 7f4a8a400640 1 --2- 172.21.15.28:0/3930649214 >> v2:172.21.15.103:3300/0 conn(0x7f4a841468f0 0x7f4a840ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.410+0000 7f4a8a400640 1 --2- 172.21.15.28:0/3930649214 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a84075540 0x7f4a840b5f90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.410+0000 7f4a8a400640 1 -- 172.21.15.28:0/3930649214 >> 172.21.15.28:0/3930649214 conn(0x7f4a840d1b30 msgr2=0x7f4a840d1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:29.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.410+0000 7f4a8a400640 1 -- 172.21.15.28:0/3930649214 shutdown_connections 2024-04-26T20:57:29.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.410+0000 7f4a8a400640 1 -- 172.21.15.28:0/3930649214 wait complete. 2024-04-26T20:57:29.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.412+0000 7f4a8a400640 1 Processor -- start 2024-04-26T20:57:29.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.412+0000 7f4a8a400640 1 -- start start 2024-04-26T20:57:29.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.413+0000 7f4a8a400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a84075540 0x7f4a8415c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:29.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.413+0000 7f4a8a400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4a84075920 0x7f4a84141460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:29.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.413+0000 7f4a8a400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a841468f0 0x7f4a841419a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:29.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.413+0000 7f4a8a400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4a84161580 con 0x7f4a84075540 2024-04-26T20:57:29.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.413+0000 7f4a8a400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4a84161400 con 0x7f4a841468f0 2024-04-26T20:57:29.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.413+0000 7f4a8a400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f4a84161700 con 0x7f4a84075920 2024-04-26T20:57:29.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.413+0000 7f4a89000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a84075540 0x7f4a8415c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:29.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.413+0000 7f4a89000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a84075540 0x7f4a8415c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:42518/0 (socket says 172.21.15.28:42518) 2024-04-26T20:57:29.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.413+0000 7f4a89000640 1 -- 172.21.15.28:0/3249804504 learned_addr learned my addr 172.21.15.28:0/3249804504 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:29.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.414+0000 7f4a89000640 1 -- 172.21.15.28:0/3249804504 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4a84075920 msgr2=0x7f4a84141460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:29.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.414+0000 7f4a89000640 1 --2- 172.21.15.28:0/3249804504 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4a84075920 0x7f4a84141460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.414+0000 7f4a89000640 1 -- 172.21.15.28:0/3249804504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a841468f0 msgr2=0x7f4a841419a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:29.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.414+0000 7f4a89000640 1 --2- 172.21.15.28:0/3249804504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a841468f0 0x7f4a841419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.414+0000 7f4a89000640 1 -- 172.21.15.28:0/3249804504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8413c160 con 0x7f4a84075540 2024-04-26T20:57:29.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.414+0000 7f4a89000640 1 --2- 172.21.15.28:0/3249804504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a84075540 0x7f4a8415c770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4a74000c00 tx=0x7f4a740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:29.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.414+0000 7f4a81600640 1 -- 172.21.15.28:0/3249804504 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74041070 con 0x7f4a84075540 2024-04-26T20:57:29.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.414+0000 7f4a81600640 1 -- 172.21.15.28:0/3249804504 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a7403c950 con 0x7f4a84075540 2024-04-26T20:57:29.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.414+0000 7f4a81600640 1 -- 172.21.15.28:0/3249804504 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74037040 con 0x7f4a84075540 2024-04-26T20:57:29.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.414+0000 7f4a8a400640 1 -- 172.21.15.28:0/3249804504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a8413e710 con 0x7f4a84075540 2024-04-26T20:57:29.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.415+0000 7f4a8a400640 1 -- 172.21.15.28:0/3249804504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a840cbd50 con 0x7f4a84075540 2024-04-26T20:57:29.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.416+0000 7f4a8a400640 1 -- 172.21.15.28:0/3249804504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a8406a250 con 0x7f4a84075540 2024-04-26T20:57:29.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.417+0000 7f4a81600640 1 -- 172.21.15.28:0/3249804504 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a74044070 con 0x7f4a84075540 2024-04-26T20:57:29.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.418+0000 7f4a81600640 1 --2- 172.21.15.28:0/3249804504 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4a6006c8f0 0x7f4a6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:29.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.418+0000 7f4a81600640 1 -- 172.21.15.28:0/3249804504 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4a740ac600 con 0x7f4a84075540 2024-04-26T20:57:29.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.419+0000 7f4a7be00640 1 --2- 172.21.15.28:0/3249804504 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4a6006c8f0 0x7f4a6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:29.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.419+0000 7f4a7be00640 1 --2- 172.21.15.28:0/3249804504 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4a6006c8f0 0x7f4a6006edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4a840b6d60 tx=0x7f4a6c007990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:29.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.423+0000 7f4a81600640 1 -- 172.21.15.28:0/3249804504 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4a7407aa20 con 0x7f4a84075540 2024-04-26T20:57:29.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.629+0000 7f4a8a400640 1 -- 172.21.15.28:0/3249804504 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4a8406d730 con 0x7f4a84075540 2024-04-26T20:57:29.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.629+0000 7f4a81600640 1 -- 172.21.15.28:0/3249804504 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f4a7407a320 con 0x7f4a84075540 2024-04-26T20:57:29.631 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:57:29.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.632+0000 7f4a8a400640 1 -- 172.21.15.28:0/3249804504 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4a6006c8f0 msgr2=0x7f4a6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:29.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.632+0000 7f4a8a400640 1 --2- 172.21.15.28:0/3249804504 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4a6006c8f0 0x7f4a6006edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4a840b6d60 tx=0x7f4a6c007990 comp rx=0 tx=0).stop 2024-04-26T20:57:29.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.632+0000 7f4a8a400640 1 -- 172.21.15.28:0/3249804504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a84075540 msgr2=0x7f4a8415c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:29.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.632+0000 7f4a8a400640 1 --2- 172.21.15.28:0/3249804504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a84075540 0x7f4a8415c770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f4a74000c00 tx=0x7f4a740046c0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.632+0000 7f4a8a400640 1 -- 172.21.15.28:0/3249804504 shutdown_connections 2024-04-26T20:57:29.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.632+0000 7f4a8a400640 1 --2- 172.21.15.28:0/3249804504 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4a6006c8f0 0x7f4a6006edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.632+0000 7f4a8a400640 1 --2- 172.21.15.28:0/3249804504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4a841468f0 0x7f4a841419a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.632+0000 7f4a8a400640 1 --2- 172.21.15.28:0/3249804504 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4a84075920 0x7f4a84141460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.632+0000 7f4a8a400640 1 --2- 172.21.15.28:0/3249804504 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4a84075540 0x7f4a8415c770 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.632+0000 7f4a8a400640 1 -- 172.21.15.28:0/3249804504 >> 172.21.15.28:0/3249804504 conn(0x7f4a840d1b30 msgr2=0x7f4a840b5f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:29.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.633+0000 7f4a8a400640 1 -- 172.21.15.28:0/3249804504 shutdown_connections 2024-04-26T20:57:29.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.633+0000 7f4a8a400640 1 -- 172.21.15.28:0/3249804504 wait complete. 2024-04-26T20:57:29.653 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:57:29.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.872+0000 7f7ef3e00640 1 Processor -- start 2024-04-26T20:57:29.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.872+0000 7f7ef3e00640 1 -- start start 2024-04-26T20:57:29.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.874+0000 7f7ef3e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f7ef40d1420 0x7f7ef4167b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:29.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.874+0000 7f7ef3e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ef40b5ba0 con 0x7f7ef41465c0 2024-04-26T20:57:29.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.874+0000 7f7ef3e00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ef41469a0 con 0x7f7ef40cf040 2024-04-26T20:57:29.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.874+0000 7f7ef3e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f7ef40cf520 con 0x7f7ef40d1420 2024-04-26T20:57:29.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.874+0000 7f7ef2a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7ef41465c0 0x7f7ef406e930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57964/0 (socket says 172.21.15.28:57964) 2024-04-26T20:57:29.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.874+0000 7f7ef2a00640 1 -- 172.21.15.28:0/764783171 learned_addr learned my addr 172.21.15.28:0/764783171 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:29.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.874+0000 7f7ef3400640 1 --2- 172.21.15.28:0/764783171 >> v2:172.21.15.103:3300/0 conn(0x7f7ef40d1420 0x7f7ef4167b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:29.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.874+0000 7f7ef1600640 1 -- 172.21.15.28:0/764783171 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 158295795 0 0) 0x7f7ee4001110 con 0x7f7ef41465c0 2024-04-26T20:57:29.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.874+0000 7f7ef1600640 1 -- 172.21.15.28:0/764783171 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ed8003520 con 0x7f7ef41465c0 2024-04-26T20:57:29.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.875+0000 7f7ef1600640 1 -- 172.21.15.28:0/764783171 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2120840829 0 0) 0x7f7ed8003520 con 0x7f7ef41465c0 2024-04-26T20:57:29.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.875+0000 7f7ef1600640 1 -- 172.21.15.28:0/764783171 >> v2:172.21.15.103:3300/0 conn(0x7f7ef40d1420 msgr2=0x7f7ef4167b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:29.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.875+0000 7f7ef1600640 1 --2- 172.21.15.28:0/764783171 >> v2:172.21.15.103:3300/0 conn(0x7f7ef40d1420 0x7f7ef4167b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.875+0000 7f7ef1600640 1 -- 172.21.15.28:0/764783171 >> v1:172.21.15.103:6789/0 conn(0x7f7ef40cf040 legacy=0x7f7ef40d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:29.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.875+0000 7f7ef1600640 1 -- 172.21.15.28:0/764783171 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef40cf800 con 0x7f7ef41465c0 2024-04-26T20:57:29.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.875+0000 7f7ef1600640 1 -- 172.21.15.28:0/764783171 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f7ee4001840 con 0x7f7ef41465c0 2024-04-26T20:57:29.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.875+0000 7f7ef1600640 1 -- 172.21.15.28:0/764783171 >> v1:172.21.15.28:6789/0 conn(0x7f7ef41465c0 legacy=0x7f7ef406e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:29.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.875+0000 7f7ef1600640 1 --2- 172.21.15.28:0/764783171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed8003a00 0x7f7ed8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:29.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.875+0000 7f7ef1600640 1 --2- 172.21.15.28:0/764783171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ed8004800 0x7f7ed8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:29.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.875+0000 7f7ef1600640 1 --2- 172.21.15.28:0/764783171 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ed8005210 0x7f7ed800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:29.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.875+0000 7f7ef1600640 1 -- 172.21.15.28:0/764783171 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef40cf520 con 0x7f7ed8004800 2024-04-26T20:57:29.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.875+0000 7f7ef1600640 1 -- 172.21.15.28:0/764783171 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed800c160 con 0x7f7ed8003a00 2024-04-26T20:57:29.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.875+0000 7f7ef1600640 1 -- 172.21.15.28:0/764783171 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed800c360 con 0x7f7ed8005210 2024-04-26T20:57:29.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.876+0000 7f7ef2000640 1 --2- 172.21.15.28:0/764783171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ed8004800 0x7f7ed8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:29.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.876+0000 7f7ef2a00640 1 --2- 172.21.15.28:0/764783171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed8003a00 0x7f7ed8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:29.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.876+0000 7f7ef3400640 1 --2- 172.21.15.28:0/764783171 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ed8005210 0x7f7ed800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:29.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.876+0000 7f7ef2000640 1 -- 172.21.15.28:0/764783171 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ed8005210 msgr2=0x7f7ed800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:29.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.876+0000 7f7ef2000640 1 --2- 172.21.15.28:0/764783171 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ed8005210 0x7f7ed800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.876+0000 7f7ef2000640 1 -- 172.21.15.28:0/764783171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed8003a00 msgr2=0x7f7ed8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:29.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.876+0000 7f7ef2000640 1 --2- 172.21.15.28:0/764783171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed8003a00 0x7f7ed8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.876+0000 7f7ef2000640 1 -- 172.21.15.28:0/764783171 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed8003520 con 0x7f7ed8004800 2024-04-26T20:57:29.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.876+0000 7f7ef3400640 1 -- 172.21.15.28:0/764783171 reap_dead start 2024-04-26T20:57:29.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.876+0000 7f7ef2a00640 1 --2- 172.21.15.28:0/764783171 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed8003a00 0x7f7ed8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:57:29.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.877+0000 7f7ef2000640 1 --2- 172.21.15.28:0/764783171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ed8004800 0x7f7ed8004c80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40cf450 tx=0x7f7ed8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad51f14d083183f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:29.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.877+0000 7f7ef1600640 1 -- 172.21.15.28:0/764783171 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0001370 con 0x7f7ed8004800 2024-04-26T20:57:29.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.878+0000 7f7ef1600640 1 -- 172.21.15.28:0/764783171 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee00052d0 con 0x7f7ed8004800 2024-04-26T20:57:29.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.878+0000 7f7ef1600640 1 -- 172.21.15.28:0/764783171 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee002ab70 con 0x7f7ed8004800 2024-04-26T20:57:29.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.878+0000 7f7ef3e00640 1 -- 172.21.15.28:0/764783171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ed8004800 msgr2=0x7f7ed8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:29.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.878+0000 7f7ef3e00640 1 --2- 172.21.15.28:0/764783171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ed8004800 0x7f7ed8004c80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40cf450 tx=0x7f7ed8005830 comp rx=0 tx=0).stop 2024-04-26T20:57:29.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.878+0000 7f7ef3e00640 1 -- 172.21.15.28:0/764783171 shutdown_connections 2024-04-26T20:57:29.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.878+0000 7f7ef3e00640 1 --2- 172.21.15.28:0/764783171 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ed8004800 0x7f7ed8004c80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.878+0000 7f7ef3e00640 1 -- 172.21.15.28:0/764783171 >> 172.21.15.28:0/764783171 conn(0x7f7ef406c530 msgr2=0x7f7ef406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:29.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.878+0000 7f7ef3e00640 1 -- 172.21.15.28:0/764783171 shutdown_connections 2024-04-26T20:57:29.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.878+0000 7f7ef3e00640 1 -- 172.21.15.28:0/764783171 wait complete. 2024-04-26T20:57:29.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.880+0000 7f7ef3e00640 1 Processor -- start 2024-04-26T20:57:29.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.880+0000 7f7ef3e00640 1 -- start start 2024-04-26T20:57:29.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.881+0000 7f7ef3e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed8004800 0x7f7ef413c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:29.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.881+0000 7f7ef3e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ef414acd0 0x7f7ef413d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:29.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.881+0000 7f7ef3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ef41e8f20 0x7f7ef41e93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:29.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.881+0000 7f7ef3e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef40d00a0 con 0x7f7ef41e8f20 2024-04-26T20:57:29.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.881+0000 7f7ef3e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef40cfdc0 con 0x7f7ed8004800 2024-04-26T20:57:29.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.881+0000 7f7ef3e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef40d0d10 con 0x7f7ef414acd0 2024-04-26T20:57:29.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.882+0000 7f7ef3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ef41e8f20 0x7f7ef41e93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:29.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.882+0000 7f7ef3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ef41e8f20 0x7f7ef41e93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54692/0 (socket says 172.21.15.28:54692) 2024-04-26T20:57:29.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.882+0000 7f7ef3400640 1 -- 172.21.15.28:0/1239503664 learned_addr learned my addr 172.21.15.28:0/1239503664 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:29.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.882+0000 7f7ef2a00640 1 --2- 172.21.15.28:0/1239503664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed8004800 0x7f7ef413c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:29.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.882+0000 7f7ef2000640 1 --2- 172.21.15.28:0/1239503664 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ef414acd0 0x7f7ef413d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:29.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.882+0000 7f7ef3400640 1 -- 172.21.15.28:0/1239503664 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ef414acd0 msgr2=0x7f7ef413d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:29.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.882+0000 7f7ef3400640 1 --2- 172.21.15.28:0/1239503664 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ef414acd0 0x7f7ef413d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.882+0000 7f7ef3400640 1 -- 172.21.15.28:0/1239503664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed8004800 msgr2=0x7f7ef413c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:29.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.882+0000 7f7ef3400640 1 --2- 172.21.15.28:0/1239503664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed8004800 0x7f7ef413c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:29.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.882+0000 7f7ef3400640 1 -- 172.21.15.28:0/1239503664 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef4147c60 con 0x7f7ef41e8f20 2024-04-26T20:57:29.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.883+0000 7f7ef3400640 1 --2- 172.21.15.28:0/1239503664 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ef41e8f20 0x7f7ef41e93d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4146c20 tx=0x7f7eec0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:29.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.883+0000 7f7eeaa00640 1 -- 172.21.15.28:0/1239503664 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec011de0 con 0x7f7ef41e8f20 2024-04-26T20:57:29.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.883+0000 7f7eeaa00640 1 -- 172.21.15.28:0/1239503664 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7eec01b070 con 0x7f7ef41e8f20 2024-04-26T20:57:29.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.883+0000 7f7eeaa00640 1 -- 172.21.15.28:0/1239503664 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec003990 con 0x7f7ef41e8f20 2024-04-26T20:57:29.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.883+0000 7f7ef3e00640 1 -- 172.21.15.28:0/1239503664 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef4146ea0 con 0x7f7ef41e8f20 2024-04-26T20:57:29.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.883+0000 7f7ef3e00640 1 -- 172.21.15.28:0/1239503664 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef4147460 con 0x7f7ef41e8f20 2024-04-26T20:57:29.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.884+0000 7f7ef3e00640 1 -- 172.21.15.28:0/1239503664 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef406a250 con 0x7f7ef41e8f20 2024-04-26T20:57:29.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.885+0000 7f7eeaa00640 1 -- 172.21.15.28:0/1239503664 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7eec003b30 con 0x7f7ef41e8f20 2024-04-26T20:57:29.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.887+0000 7f7eeaa00640 1 --2- 172.21.15.28:0/1239503664 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7ed406c920 0x7f7ed406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:29.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.887+0000 7f7eeaa00640 1 -- 172.21.15.28:0/1239503664 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7eec08c4e0 con 0x7f7ef41e8f20 2024-04-26T20:57:29.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.887+0000 7f7ef2a00640 1 --2- 172.21.15.28:0/1239503664 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7ed406c920 0x7f7ed406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:29.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.888+0000 7f7ef2a00640 1 --2- 172.21.15.28:0/1239503664 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7ed406c920 0x7f7ed406ede0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4000fb0 tx=0x7f7ee4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:29.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:29.891+0000 7f7eeaa00640 1 -- 172.21.15.28:0/1239503664 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7eec003e20 con 0x7f7ef41e8f20 2024-04-26T20:57:30.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.096+0000 7f7ef3e00640 1 -- 172.21.15.28:0/1239503664 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7ef40b5980 con 0x7f7ef41e8f20 2024-04-26T20:57:30.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.096+0000 7f7eeaa00640 1 -- 172.21.15.28:0/1239503664 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f7eec05a870 con 0x7f7ef41e8f20 2024-04-26T20:57:30.098 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-04-26T20:57:30.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.099+0000 7f7ef3e00640 1 -- 172.21.15.28:0/1239503664 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7ed406c920 msgr2=0x7f7ed406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:30.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.099+0000 7f7ef3e00640 1 --2- 172.21.15.28:0/1239503664 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7ed406c920 0x7f7ed406ede0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4000fb0 tx=0x7f7ee4009040 comp rx=0 tx=0).stop 2024-04-26T20:57:30.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.099+0000 7f7ef3e00640 1 -- 172.21.15.28:0/1239503664 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ef41e8f20 msgr2=0x7f7ef41e93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:30.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.099+0000 7f7ef3e00640 1 --2- 172.21.15.28:0/1239503664 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ef41e8f20 0x7f7ef41e93d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4146c20 tx=0x7f7eec0115a0 comp rx=0 tx=0).stop 2024-04-26T20:57:30.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.099+0000 7f7ef3e00640 1 -- 172.21.15.28:0/1239503664 shutdown_connections 2024-04-26T20:57:30.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.099+0000 7f7ef3e00640 1 --2- 172.21.15.28:0/1239503664 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7ed406c920 0x7f7ed406ede0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:30.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.099+0000 7f7ef3e00640 1 --2- 172.21.15.28:0/1239503664 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ef41e8f20 0x7f7ef41e93d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:30.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.099+0000 7f7ef3e00640 1 --2- 172.21.15.28:0/1239503664 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ef414acd0 0x7f7ef413d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:30.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.099+0000 7f7ef3e00640 1 --2- 172.21.15.28:0/1239503664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed8004800 0x7f7ef413c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:30.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.099+0000 7f7ef3e00640 1 -- 172.21.15.28:0/1239503664 >> 172.21.15.28:0/1239503664 conn(0x7f7ef406c530 msgr2=0x7f7ef40b46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:30.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.099+0000 7f7ef3e00640 1 -- 172.21.15.28:0/1239503664 shutdown_connections 2024-04-26T20:57:30.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.100+0000 7f7ef3e00640 1 -- 172.21.15.28:0/1239503664 wait complete. 2024-04-26T20:57:30.120 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:57:30.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.335+0000 7fc6eea00640 1 Processor -- start 2024-04-26T20:57:30.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.335+0000 7fc6eea00640 1 -- start start 2024-04-26T20:57:30.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.336+0000 7fc6eea00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fc6e80d1050 0x7fc6e80b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:30.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.337+0000 7fc6eea00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6e80cf2f0 con 0x7fc6e80d0c70 2024-04-26T20:57:30.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.337+0000 7fc6eea00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6e80cf5d0 con 0x7fc6e80b3f60 2024-04-26T20:57:30.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.337+0000 7fc6eea00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fc6e80cf8b0 con 0x7fc6e80d1050 2024-04-26T20:57:30.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.337+0000 7fc6ed600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc6e80d0c70 0x7fc6e8161320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57966/0 (socket says 172.21.15.28:57966) 2024-04-26T20:57:30.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.337+0000 7fc6ed600640 1 -- 172.21.15.28:0/4025633903 learned_addr learned my addr 172.21.15.28:0/4025633903 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:30.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.337+0000 7fc6ecc00640 1 --2- 172.21.15.28:0/4025633903 >> v2:172.21.15.103:3300/0 conn(0x7fc6e80d1050 0x7fc6e80b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.337+0000 7fc6e7e00640 1 -- 172.21.15.28:0/4025633903 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3623194853 0 0) 0x7fc6e80cf2f0 con 0x7fc6e80d0c70 2024-04-26T20:57:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.337+0000 7fc6e7e00640 1 -- 172.21.15.28:0/4025633903 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6d0003520 con 0x7fc6e80d0c70 2024-04-26T20:57:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.338+0000 7fc6e7e00640 1 -- 172.21.15.28:0/4025633903 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 452757934 0 0) 0x7fc6d0003520 con 0x7fc6e80d0c70 2024-04-26T20:57:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.338+0000 7fc6e7e00640 1 -- 172.21.15.28:0/4025633903 >> v2:172.21.15.103:3300/0 conn(0x7fc6e80d1050 msgr2=0x7fc6e80b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.338+0000 7fc6e7e00640 1 --2- 172.21.15.28:0/4025633903 >> v2:172.21.15.103:3300/0 conn(0x7fc6e80d1050 0x7fc6e80b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.338+0000 7fc6e7e00640 1 -- 172.21.15.28:0/4025633903 >> v1:172.21.15.103:6789/0 conn(0x7fc6e80b3f60 legacy=0x7fc6e80b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.338+0000 7fc6e7e00640 1 -- 172.21.15.28:0/4025633903 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e80cfb90 con 0x7fc6e80d0c70 2024-04-26T20:57:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.338+0000 7fc6e7e00640 1 -- 172.21.15.28:0/4025633903 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fc6dc001670 con 0x7fc6e80d0c70 2024-04-26T20:57:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.338+0000 7fc6e7e00640 1 -- 172.21.15.28:0/4025633903 >> v1:172.21.15.28:6789/0 conn(0x7fc6e80d0c70 legacy=0x7fc6e8161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.338+0000 7fc6e7e00640 1 --2- 172.21.15.28:0/4025633903 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc6d0003a00 0x7fc6d0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.338+0000 7fc6e7e00640 1 --2- 172.21.15.28:0/4025633903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6d0004800 0x7fc6d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.338+0000 7fc6e7e00640 1 --2- 172.21.15.28:0/4025633903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6d00051c0 0x7fc6d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.338+0000 7fc6e7e00640 1 -- 172.21.15.28:0/4025633903 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e80cf8b0 con 0x7fc6d00051c0 2024-04-26T20:57:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.338+0000 7fc6e7e00640 1 -- 172.21.15.28:0/4025633903 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d000c0c0 con 0x7fc6d0004800 2024-04-26T20:57:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.338+0000 7fc6e7e00640 1 -- 172.21.15.28:0/4025633903 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc6d000c2c0 con 0x7fc6d0003a00 2024-04-26T20:57:30.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.338+0000 7fc6ee000640 1 --2- 172.21.15.28:0/4025633903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6d00051c0 0x7fc6d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.338+0000 7fc6ecc00640 1 --2- 172.21.15.28:0/4025633903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6d0004800 0x7fc6d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.339+0000 7fc6ed600640 1 --2- 172.21.15.28:0/4025633903 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc6d0003a00 0x7fc6d0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.339+0000 7fc6ee000640 1 -- 172.21.15.28:0/4025633903 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc6d0003a00 msgr2=0x7fc6d0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.339+0000 7fc6ee000640 1 --2- 172.21.15.28:0/4025633903 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc6d0003a00 0x7fc6d0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.339+0000 7fc6ee000640 1 -- 172.21.15.28:0/4025633903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6d0004800 msgr2=0x7fc6d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.339+0000 7fc6ee000640 1 --2- 172.21.15.28:0/4025633903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6d0004800 0x7fc6d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.339+0000 7fc6ee000640 1 -- 172.21.15.28:0/4025633903 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d0003520 con 0x7fc6d00051c0 2024-04-26T20:57:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.339+0000 7fc6ed600640 1 --2- 172.21.15.28:0/4025633903 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc6d0003a00 0x7fc6d0003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.340+0000 7fc6ee000640 1 -- 172.21.15.28:0/4025633903 reap_dead start 2024-04-26T20:57:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.340+0000 7fc6ee000640 1 --2- 172.21.15.28:0/4025633903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6d00051c0 0x7fc6d000b9c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0001e50 tx=0x7fc6d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2adfe851789b8b56 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.340+0000 7fc6e7e00640 1 -- 172.21.15.28:0/4025633903 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0001a70 con 0x7fc6d00051c0 2024-04-26T20:57:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.340+0000 7fc6e7e00640 1 -- 172.21.15.28:0/4025633903 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6e0003970 con 0x7fc6d00051c0 2024-04-26T20:57:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.340+0000 7fc6e7e00640 1 -- 172.21.15.28:0/4025633903 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0003d60 con 0x7fc6d00051c0 2024-04-26T20:57:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.340+0000 7fc6eea00640 1 -- 172.21.15.28:0/4025633903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6d00051c0 msgr2=0x7fc6d000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:30.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.340+0000 7fc6eea00640 1 --2- 172.21.15.28:0/4025633903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6d00051c0 0x7fc6d000b9c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0001e50 tx=0x7fc6d0005830 comp rx=0 tx=0).stop 2024-04-26T20:57:30.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.341+0000 7fc6eea00640 1 -- 172.21.15.28:0/4025633903 shutdown_connections 2024-04-26T20:57:30.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.341+0000 7fc6eea00640 1 --2- 172.21.15.28:0/4025633903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6d00051c0 0x7fc6d000b9c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:30.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.341+0000 7fc6eea00640 1 -- 172.21.15.28:0/4025633903 >> 172.21.15.28:0/4025633903 conn(0x7fc6e806c530 msgr2=0x7fc6e806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:30.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.341+0000 7fc6eea00640 1 -- 172.21.15.28:0/4025633903 shutdown_connections 2024-04-26T20:57:30.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.341+0000 7fc6eea00640 1 -- 172.21.15.28:0/4025633903 wait complete. 2024-04-26T20:57:30.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.343+0000 7fc6eea00640 1 Processor -- start 2024-04-26T20:57:30.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.343+0000 7fc6eea00640 1 -- start start 2024-04-26T20:57:30.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.344+0000 7fc6eea00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc6d00051c0 0x7fc6e8073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:30.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.344+0000 7fc6eea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6e813c470 0x7fc6e8071f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:30.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.344+0000 7fc6eea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6e8072470 0x7fc6e8155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:30.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.344+0000 7fc6eea00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e80d14b0 con 0x7fc6e813c470 2024-04-26T20:57:30.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.344+0000 7fc6eea00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e8146840 con 0x7fc6e8072470 2024-04-26T20:57:30.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.344+0000 7fc6eea00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc6e80b5920 con 0x7fc6d00051c0 2024-04-26T20:57:30.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.344+0000 7fc6ecc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6e813c470 0x7fc6e8071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:30.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.344+0000 7fc6ecc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6e813c470 0x7fc6e8071f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54710/0 (socket says 172.21.15.28:54710) 2024-04-26T20:57:30.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.344+0000 7fc6ecc00640 1 -- 172.21.15.28:0/1739037849 learned_addr learned my addr 172.21.15.28:0/1739037849 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:30.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.344+0000 7fc6ed600640 1 --2- 172.21.15.28:0/1739037849 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc6d00051c0 0x7fc6e8073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:30.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.345+0000 7fc6ecc00640 1 -- 172.21.15.28:0/1739037849 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc6d00051c0 msgr2=0x7fc6e8073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:30.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.345+0000 7fc6ecc00640 1 --2- 172.21.15.28:0/1739037849 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc6d00051c0 0x7fc6e8073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:30.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.345+0000 7fc6ee000640 1 --2- 172.21.15.28:0/1739037849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6e8072470 0x7fc6e8155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:30.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.345+0000 7fc6ecc00640 1 -- 172.21.15.28:0/1739037849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6e8072470 msgr2=0x7fc6e8155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:30.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.345+0000 7fc6ecc00640 1 --2- 172.21.15.28:0/1739037849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6e8072470 0x7fc6e8155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:30.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.345+0000 7fc6ecc00640 1 -- 172.21.15.28:0/1739037849 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e80729f0 con 0x7fc6e813c470 2024-04-26T20:57:30.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.345+0000 7fc6ecc00640 1 --2- 172.21.15.28:0/1739037849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6e813c470 0x7fc6e8071f30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4011540 tx=0x7fc6d4011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:30.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.345+0000 7fc6e5600640 1 -- 172.21.15.28:0/1739037849 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d4015930 con 0x7fc6e813c470 2024-04-26T20:57:30.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.345+0000 7fc6e5600640 1 -- 172.21.15.28:0/1739037849 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6d4015ad0 con 0x7fc6e813c470 2024-04-26T20:57:30.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.345+0000 7fc6e5600640 1 -- 172.21.15.28:0/1739037849 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d400f460 con 0x7fc6e813c470 2024-04-26T20:57:30.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.345+0000 7fc6eea00640 1 -- 172.21.15.28:0/1739037849 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e8072cd0 con 0x7fc6e813c470 2024-04-26T20:57:30.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.345+0000 7fc6eea00640 1 -- 172.21.15.28:0/1739037849 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e8073290 con 0x7fc6e813c470 2024-04-26T20:57:30.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.347+0000 7fc6eea00640 1 -- 172.21.15.28:0/1739037849 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e806a2c0 con 0x7fc6e813c470 2024-04-26T20:57:30.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.347+0000 7fc6e5600640 1 -- 172.21.15.28:0/1739037849 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc6d400f600 con 0x7fc6e813c470 2024-04-26T20:57:30.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.348+0000 7fc6e5600640 1 --2- 172.21.15.28:0/1739037849 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc6c806c8f0 0x7fc6c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:30.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.349+0000 7fc6e5600640 1 -- 172.21.15.28:0/1739037849 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc6d408c560 con 0x7fc6e813c470 2024-04-26T20:57:30.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.349+0000 7fc6ed600640 1 --2- 172.21.15.28:0/1739037849 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc6c806c8f0 0x7fc6c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:30.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.349+0000 7fc6ed600640 1 --2- 172.21.15.28:0/1739037849 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc6c806c8f0 0x7fc6c806edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc001eb0 tx=0x7fc6dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:30.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.353+0000 7fc6e5600640 1 -- 172.21.15.28:0/1739037849 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc6d4019030 con 0x7fc6e813c470 2024-04-26T20:57:30.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.555+0000 7fc6eea00640 1 -- 172.21.15.28:0/1739037849 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc6e80d07a0 con 0x7fc6e813c470 2024-04-26T20:57:30.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.556+0000 7fc6e5600640 1 -- 172.21.15.28:0/1739037849 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fc6d405a8f0 con 0x7fc6e813c470 2024-04-26T20:57:30.557 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-04-26T20:57:30.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.559+0000 7fc6eea00640 1 -- 172.21.15.28:0/1739037849 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc6c806c8f0 msgr2=0x7fc6c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:30.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.559+0000 7fc6eea00640 1 --2- 172.21.15.28:0/1739037849 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc6c806c8f0 0x7fc6c806edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc001eb0 tx=0x7fc6dc009040 comp rx=0 tx=0).stop 2024-04-26T20:57:30.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.559+0000 7fc6eea00640 1 -- 172.21.15.28:0/1739037849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6e813c470 msgr2=0x7fc6e8071f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:30.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.559+0000 7fc6eea00640 1 --2- 172.21.15.28:0/1739037849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6e813c470 0x7fc6e8071f30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4011540 tx=0x7fc6d4011a00 comp rx=0 tx=0).stop 2024-04-26T20:57:30.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.559+0000 7fc6eea00640 1 -- 172.21.15.28:0/1739037849 shutdown_connections 2024-04-26T20:57:30.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.559+0000 7fc6eea00640 1 --2- 172.21.15.28:0/1739037849 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc6c806c8f0 0x7fc6c806edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:30.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.559+0000 7fc6eea00640 1 --2- 172.21.15.28:0/1739037849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6e8072470 0x7fc6e8155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:30.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.559+0000 7fc6eea00640 1 --2- 172.21.15.28:0/1739037849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc6e813c470 0x7fc6e8071f30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:30.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.559+0000 7fc6eea00640 1 --2- 172.21.15.28:0/1739037849 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc6d00051c0 0x7fc6e8073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:30.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.559+0000 7fc6eea00640 1 -- 172.21.15.28:0/1739037849 >> 172.21.15.28:0/1739037849 conn(0x7fc6e806c530 msgr2=0x7fc6e815f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:30.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.560+0000 7fc6eea00640 1 -- 172.21.15.28:0/1739037849 shutdown_connections 2024-04-26T20:57:30.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.560+0000 7fc6eea00640 1 -- 172.21.15.28:0/1739037849 wait complete. 2024-04-26T20:57:30.580 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_dump_tree.TestDumpTree.test_basic' 2024-04-26T20:57:30.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.790+0000 7f1c45e00640 1 Processor -- start 2024-04-26T20:57:30.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.791+0000 7f1c45e00640 1 -- start start 2024-04-26T20:57:30.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.792+0000 7f1c45e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1c4015aab0 0x7f1c400cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:30.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.792+0000 7f1c45e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c4015ae90 con 0x7f1c401611f0 2024-04-26T20:57:30.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.792+0000 7f1c45e00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c400cfc90 con 0x7f1c40073d10 2024-04-26T20:57:30.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.792+0000 7f1c45e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f1c400cfa40 con 0x7f1c4015aab0 2024-04-26T20:57:30.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.793+0000 7f1c45400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1c401611f0 0x7f1c400724d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57980/0 (socket says 172.21.15.28:57980) 2024-04-26T20:57:30.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.793+0000 7f1c45400640 1 -- 172.21.15.28:0/1545442145 learned_addr learned my addr 172.21.15.28:0/1545442145 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:30.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.793+0000 7f1c44a00640 1 --2- 172.21.15.28:0/1545442145 >> v2:172.21.15.28:3300/0 conn(0x7f1c4015aab0 0x7f1c400cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:30.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.793+0000 7f1c3f400640 1 -- 172.21.15.28:0/1545442145 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1555150473 0 0) 0x7f1c4015ae90 con 0x7f1c401611f0 2024-04-26T20:57:30.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.793+0000 7f1c3f400640 1 -- 172.21.15.28:0/1545442145 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c28003520 con 0x7f1c401611f0 2024-04-26T20:57:30.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.793+0000 7f1c44a00640 1 -- 172.21.15.28:0/1545442145 >> v1:172.21.15.103:6789/0 conn(0x7f1c40073d10 legacy=0x7f1c40160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:30.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.794+0000 7f1c44a00640 1 -- 172.21.15.28:0/1545442145 >> v1:172.21.15.28:6789/0 conn(0x7f1c401611f0 legacy=0x7f1c400724d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:30.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.794+0000 7f1c44a00640 1 -- 172.21.15.28:0/1545442145 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c400cff70 con 0x7f1c4015aab0 2024-04-26T20:57:30.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.795+0000 7f1c44a00640 1 --2- 172.21.15.28:0/1545442145 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c4015aab0 0x7f1c400cd450 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1c340097a0 tx=0x7f1c3402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c553f92102d1909 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:30.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.795+0000 7f1c3f400640 1 -- 172.21.15.28:0/1545442145 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c34037070 con 0x7f1c4015aab0 2024-04-26T20:57:30.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.795+0000 7f1c3f400640 1 -- 172.21.15.28:0/1545442145 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c34032440 con 0x7f1c4015aab0 2024-04-26T20:57:30.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.795+0000 7f1c3f400640 1 -- 172.21.15.28:0/1545442145 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c34032810 con 0x7f1c4015aab0 2024-04-26T20:57:30.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.796+0000 7f1c45e00640 1 -- 172.21.15.28:0/1545442145 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c4015aab0 msgr2=0x7f1c400cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:30.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.796+0000 7f1c45e00640 1 --2- 172.21.15.28:0/1545442145 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c4015aab0 0x7f1c400cd450 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1c340097a0 tx=0x7f1c3402a1a0 comp rx=0 tx=0).stop 2024-04-26T20:57:30.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.796+0000 7f1c45e00640 1 -- 172.21.15.28:0/1545442145 shutdown_connections 2024-04-26T20:57:30.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.796+0000 7f1c45e00640 1 --2- 172.21.15.28:0/1545442145 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c4015aab0 0x7f1c400cd450 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:30.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.796+0000 7f1c45e00640 1 -- 172.21.15.28:0/1545442145 >> 172.21.15.28:0/1545442145 conn(0x7f1c400b2720 msgr2=0x7f1c400b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:30.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.796+0000 7f1c45e00640 1 -- 172.21.15.28:0/1545442145 shutdown_connections 2024-04-26T20:57:30.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.796+0000 7f1c45e00640 1 -- 172.21.15.28:0/1545442145 wait complete. 2024-04-26T20:57:30.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.798+0000 7f1c45e00640 1 Processor -- start 2024-04-26T20:57:30.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.798+0000 7f1c45e00640 1 -- start start 2024-04-26T20:57:30.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.799+0000 7f1c45e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c40073d10 0x7f1c40143d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:30.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.799+0000 7f1c45e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1c4015aab0 0x7f1c401515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:30.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.799+0000 7f1c45e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c401611f0 0x7f1c400b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:30.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.799+0000 7f1c45e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1c4006df00 con 0x7f1c40073d10 2024-04-26T20:57:30.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.799+0000 7f1c45e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1c40072040 con 0x7f1c401611f0 2024-04-26T20:57:30.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.799+0000 7f1c45e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f1c400d0250 con 0x7f1c4015aab0 2024-04-26T20:57:30.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.799+0000 7f1c44a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c40073d10 0x7f1c40143d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:30.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.799+0000 7f1c44a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c40073d10 0x7f1c40143d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54736/0 (socket says 172.21.15.28:54736) 2024-04-26T20:57:30.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.799+0000 7f1c44a00640 1 -- 172.21.15.28:0/848895172 learned_addr learned my addr 172.21.15.28:0/848895172 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:30.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.799+0000 7f1c44a00640 1 -- 172.21.15.28:0/848895172 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1c4015aab0 msgr2=0x7f1c401515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:30.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.799+0000 7f1c44a00640 1 --2- 172.21.15.28:0/848895172 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1c4015aab0 0x7f1c401515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:30.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.799+0000 7f1c44a00640 1 -- 172.21.15.28:0/848895172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c401611f0 msgr2=0x7f1c400b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:30.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.799+0000 7f1c44a00640 1 --2- 172.21.15.28:0/848895172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c401611f0 0x7f1c400b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:30.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.799+0000 7f1c44a00640 1 -- 172.21.15.28:0/848895172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c40144310 con 0x7f1c40073d10 2024-04-26T20:57:30.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.800+0000 7f1c44a00640 1 --2- 172.21.15.28:0/848895172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c40073d10 0x7f1c40143d30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1c34035040 tx=0x7f1c340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:30.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.800+0000 7f1c3cc00640 1 -- 172.21.15.28:0/848895172 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c34037040 con 0x7f1c40073d10 2024-04-26T20:57:30.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.800+0000 7f1c3cc00640 1 -- 172.21.15.28:0/848895172 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c34030500 con 0x7f1c40073d10 2024-04-26T20:57:30.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.800+0000 7f1c3cc00640 1 -- 172.21.15.28:0/848895172 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c34030910 con 0x7f1c40073d10 2024-04-26T20:57:30.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.800+0000 7f1c45e00640 1 -- 172.21.15.28:0/848895172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c401445f0 con 0x7f1c40073d10 2024-04-26T20:57:30.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.800+0000 7f1c45e00640 1 -- 172.21.15.28:0/848895172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c400b5070 con 0x7f1c40073d10 2024-04-26T20:57:30.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.801+0000 7f1c45e00640 1 -- 172.21.15.28:0/848895172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c4006a360 con 0x7f1c40073d10 2024-04-26T20:57:30.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.803+0000 7f1c3cc00640 1 -- 172.21.15.28:0/848895172 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c34044020 con 0x7f1c40073d10 2024-04-26T20:57:30.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.803+0000 7f1c3cc00640 1 --2- 172.21.15.28:0/848895172 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1c2006c920 0x7f1c2006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:30.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.804+0000 7f1c3cc00640 1 -- 172.21.15.28:0/848895172 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1c340abdb0 con 0x7f1c40073d10 2024-04-26T20:57:30.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.804+0000 7f1c3fe00640 1 --2- 172.21.15.28:0/848895172 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1c2006c920 0x7f1c2006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:30.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.804+0000 7f1c3fe00640 1 --2- 172.21.15.28:0/848895172 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1c2006c920 0x7f1c2006ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c000f60 tx=0x7f1c2c0079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:30.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:30.808+0000 7f1c3cc00640 1 -- 172.21.15.28:0/848895172 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1c3407a1e0 con 0x7f1c40073d10 2024-04-26T20:57:31.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:31.096+0000 7f1c45e00640 1 -- 172.21.15.28:0/848895172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_dump_tree.TestDumpTree.test_basic"]} v 0) -- 0x7f1c4006fc80 con 0x7f1c40073d10 2024-04-26T20:57:31.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:31.961+0000 7f1c3cc00640 1 -- 172.21.15.28:0/848895172 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_dump_tree.TestDumpTree.test_basic"]}]=0 v57) ==== 133+0+0 (secure 0 0 0) 0x7f1c34079ae0 con 0x7f1c40073d10 2024-04-26T20:57:31.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:31.964+0000 7f1c45e00640 1 -- 172.21.15.28:0/848895172 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1c2006c920 msgr2=0x7f1c2006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:31.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:31.964+0000 7f1c45e00640 1 --2- 172.21.15.28:0/848895172 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1c2006c920 0x7f1c2006ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c000f60 tx=0x7f1c2c0079d0 comp rx=0 tx=0).stop 2024-04-26T20:57:31.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:31.965+0000 7f1c45e00640 1 -- 172.21.15.28:0/848895172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c40073d10 msgr2=0x7f1c40143d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:31.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:31.965+0000 7f1c45e00640 1 --2- 172.21.15.28:0/848895172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c40073d10 0x7f1c40143d30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1c34035040 tx=0x7f1c340046c0 comp rx=0 tx=0).stop 2024-04-26T20:57:31.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:31.965+0000 7f1c45e00640 1 -- 172.21.15.28:0/848895172 shutdown_connections 2024-04-26T20:57:31.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:31.965+0000 7f1c45e00640 1 --2- 172.21.15.28:0/848895172 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1c2006c920 0x7f1c2006ede0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:31.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:31.965+0000 7f1c45e00640 1 --2- 172.21.15.28:0/848895172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1c401611f0 0x7f1c400b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:31.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:31.965+0000 7f1c45e00640 1 --2- 172.21.15.28:0/848895172 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1c4015aab0 0x7f1c401515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:31.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:31.965+0000 7f1c45e00640 1 --2- 172.21.15.28:0/848895172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1c40073d10 0x7f1c40143d30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:31.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:31.965+0000 7f1c45e00640 1 -- 172.21.15.28:0/848895172 >> 172.21.15.28:0/848895172 conn(0x7f1c400b2720 msgr2=0x7f1c4006f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:31.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:31.965+0000 7f1c45e00640 1 -- 172.21.15.28:0/848895172 shutdown_connections 2024-04-26T20:57:31.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:31.965+0000 7f1c45e00640 1 -- 172.21.15.28:0/848895172 wait complete. 2024-04-26T20:57:31.990 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T20:57:32.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.203+0000 7f49a4600640 1 Processor -- start 2024-04-26T20:57:32.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.203+0000 7f49a4600640 1 -- start start 2024-04-26T20:57:32.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.204+0000 7f49a4600640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f499c06fc10 0x7f499c073f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:32.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.204+0000 7f49a4600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f499c15e310 con 0x7f499c0ceaa0 2024-04-26T20:57:32.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.204+0000 7f49a4600640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f499c072aa0 con 0x7f499c073580 2024-04-26T20:57:32.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.204+0000 7f49a4600640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f499c074490 con 0x7f499c06fc10 2024-04-26T20:57:32.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.205+0000 7f49a2600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f499c0ceaa0 0x7f499c06e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:57994/0 (socket says 172.21.15.28:57994) 2024-04-26T20:57:32.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.205+0000 7f49a2600640 1 -- 172.21.15.28:0/1353844335 learned_addr learned my addr 172.21.15.28:0/1353844335 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:32.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.205+0000 7f49a1c00640 1 --2- 172.21.15.28:0/1353844335 >> v2:172.21.15.103:3300/0 conn(0x7f499c06fc10 0x7f499c073f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:32.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.205+0000 7f499be00640 1 -- 172.21.15.28:0/1353844335 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3696915521 0 0) 0x7f499c15e310 con 0x7f499c0ceaa0 2024-04-26T20:57:32.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.205+0000 7f499be00640 1 -- 172.21.15.28:0/1353844335 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4980003520 con 0x7f499c0ceaa0 2024-04-26T20:57:32.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.206+0000 7f499be00640 1 -- 172.21.15.28:0/1353844335 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3374308264 0 0) 0x7f4980003520 con 0x7f499c0ceaa0 2024-04-26T20:57:32.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.206+0000 7f499be00640 1 -- 172.21.15.28:0/1353844335 >> v2:172.21.15.103:3300/0 conn(0x7f499c06fc10 msgr2=0x7f499c073f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:32.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.206+0000 7f499be00640 1 --2- 172.21.15.28:0/1353844335 >> v2:172.21.15.103:3300/0 conn(0x7f499c06fc10 0x7f499c073f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:32.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.206+0000 7f499be00640 1 -- 172.21.15.28:0/1353844335 >> v1:172.21.15.103:6789/0 conn(0x7f499c073580 legacy=0x7f499c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:32.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.206+0000 7f499be00640 1 -- 172.21.15.28:0/1353844335 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499c072d80 con 0x7f499c0ceaa0 2024-04-26T20:57:32.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.206+0000 7f499be00640 1 -- 172.21.15.28:0/1353844335 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f4994001670 con 0x7f499c0ceaa0 2024-04-26T20:57:32.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.206+0000 7f499be00640 1 -- 172.21.15.28:0/1353844335 >> v1:172.21.15.28:6789/0 conn(0x7f499c0ceaa0 legacy=0x7f499c06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:32.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.206+0000 7f499be00640 1 --2- 172.21.15.28:0/1353844335 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4980003a00 0x7f4980003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:32.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.206+0000 7f499be00640 1 --2- 172.21.15.28:0/1353844335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4980004800 0x7f4980004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:32.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.206+0000 7f499be00640 1 --2- 172.21.15.28:0/1353844335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4980005210 0x7f498000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:32.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.206+0000 7f499be00640 1 -- 172.21.15.28:0/1353844335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f498000c130 con 0x7f4980005210 2024-04-26T20:57:32.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.206+0000 7f499be00640 1 -- 172.21.15.28:0/1353844335 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f498000c2e0 con 0x7f4980004800 2024-04-26T20:57:32.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.206+0000 7f499be00640 1 -- 172.21.15.28:0/1353844335 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f498000c4e0 con 0x7f4980003a00 2024-04-26T20:57:32.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.206+0000 7f49a2600640 1 --2- 172.21.15.28:0/1353844335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4980005210 0x7f498000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:32.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.207+0000 7f49a1c00640 1 --2- 172.21.15.28:0/1353844335 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4980003a00 0x7f4980003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:32.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.207+0000 7f49a1200640 1 --2- 172.21.15.28:0/1353844335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4980004800 0x7f4980004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:32.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.207+0000 7f49a2600640 1 -- 172.21.15.28:0/1353844335 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4980003a00 msgr2=0x7f4980003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:32.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.207+0000 7f49a2600640 1 --2- 172.21.15.28:0/1353844335 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4980003a00 0x7f4980003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:32.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.207+0000 7f49a2600640 1 -- 172.21.15.28:0/1353844335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4980004800 msgr2=0x7f4980004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:32.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.207+0000 7f49a2600640 1 --2- 172.21.15.28:0/1353844335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4980004800 0x7f4980004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:32.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.207+0000 7f49a2600640 1 -- 172.21.15.28:0/1353844335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4980003520 con 0x7f4980005210 2024-04-26T20:57:32.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.208+0000 7f49a2600640 1 -- 172.21.15.28:0/1353844335 reap_dead start 2024-04-26T20:57:32.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.208+0000 7f49a2600640 1 --2- 172.21.15.28:0/1353844335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4980005210 0x7f498000b9c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4994007fd0 tx=0x7f4980005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9838a5b8fd63024 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:32.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.208+0000 7f499be00640 1 -- 172.21.15.28:0/1353844335 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994007c20 con 0x7f4980005210 2024-04-26T20:57:32.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.208+0000 7f499be00640 1 -- 172.21.15.28:0/1353844335 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4994005af0 con 0x7f4980005210 2024-04-26T20:57:32.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.208+0000 7f499be00640 1 -- 172.21.15.28:0/1353844335 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994007070 con 0x7f4980005210 2024-04-26T20:57:32.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.209+0000 7f49a4600640 1 -- 172.21.15.28:0/1353844335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4980005210 msgr2=0x7f498000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:32.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.209+0000 7f49a4600640 1 --2- 172.21.15.28:0/1353844335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4980005210 0x7f498000b9c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4994007fd0 tx=0x7f4980005830 comp rx=0 tx=0).stop 2024-04-26T20:57:32.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.209+0000 7f49a4600640 1 -- 172.21.15.28:0/1353844335 shutdown_connections 2024-04-26T20:57:32.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.209+0000 7f49a4600640 1 --2- 172.21.15.28:0/1353844335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4980005210 0x7f498000b9c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:32.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.209+0000 7f49a4600640 1 -- 172.21.15.28:0/1353844335 >> 172.21.15.28:0/1353844335 conn(0x7f499c06c770 msgr2=0x7f499c06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:32.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.209+0000 7f49a4600640 1 -- 172.21.15.28:0/1353844335 shutdown_connections 2024-04-26T20:57:32.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.209+0000 7f49a4600640 1 -- 172.21.15.28:0/1353844335 wait complete. 2024-04-26T20:57:32.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.211+0000 7f49a4600640 1 Processor -- start 2024-04-26T20:57:32.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.211+0000 7f49a4600640 1 -- start start 2024-04-26T20:57:32.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.212+0000 7f49a4600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4980005210 0x7f499c14c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:32.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.212+0000 7f49a4600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f499c0ad470 0x7f499c0ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:32.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.212+0000 7f49a4600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f499c0af490 0x7f499c0af940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:32.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.212+0000 7f49a4600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f499c073a30 con 0x7f499c0af490 2024-04-26T20:57:32.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.212+0000 7f49a4600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f499c15dfa0 con 0x7f4980005210 2024-04-26T20:57:32.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.212+0000 7f49a4600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f499c1f4de0 con 0x7f499c0ad470 2024-04-26T20:57:32.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.212+0000 7f49a2600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f499c0af490 0x7f499c0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:32.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.212+0000 7f49a2600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f499c0af490 0x7f499c0af940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54744/0 (socket says 172.21.15.28:54744) 2024-04-26T20:57:32.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.212+0000 7f49a2600640 1 -- 172.21.15.28:0/823830026 learned_addr learned my addr 172.21.15.28:0/823830026 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:32.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.213+0000 7f49a1200640 1 --2- 172.21.15.28:0/823830026 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f499c0ad470 0x7f499c0ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:32.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.213+0000 7f49a1c00640 1 --2- 172.21.15.28:0/823830026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4980005210 0x7f499c14c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:32.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.213+0000 7f49a2600640 1 -- 172.21.15.28:0/823830026 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f499c0ad470 msgr2=0x7f499c0ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:32.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.213+0000 7f49a2600640 1 --2- 172.21.15.28:0/823830026 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f499c0ad470 0x7f499c0ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:32.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.213+0000 7f49a2600640 1 -- 172.21.15.28:0/823830026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4980005210 msgr2=0x7f499c14c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:32.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.213+0000 7f49a2600640 1 --2- 172.21.15.28:0/823830026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4980005210 0x7f499c14c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:32.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.213+0000 7f49a2600640 1 -- 172.21.15.28:0/823830026 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f499c1539b0 con 0x7f499c0af490 2024-04-26T20:57:32.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.213+0000 7f49a2600640 1 --2- 172.21.15.28:0/823830026 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f499c0af490 0x7f499c0af940 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f499400c1d0 tx=0x7f499c15e4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:32.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.213+0000 7f4999600640 1 -- 172.21.15.28:0/823830026 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994037040 con 0x7f499c0af490 2024-04-26T20:57:32.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.213+0000 7f4999600640 1 -- 172.21.15.28:0/823830026 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4994041070 con 0x7f499c0af490 2024-04-26T20:57:32.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.213+0000 7f4999600640 1 -- 172.21.15.28:0/823830026 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4994007290 con 0x7f499c0af490 2024-04-26T20:57:32.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.214+0000 7f49a4600640 1 -- 172.21.15.28:0/823830026 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f499c0b0170 con 0x7f499c0af490 2024-04-26T20:57:32.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.214+0000 7f49a4600640 1 -- 172.21.15.28:0/823830026 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f499c0ab020 con 0x7f499c0af490 2024-04-26T20:57:32.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.215+0000 7f49a4600640 1 -- 172.21.15.28:0/823830026 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f499c0ceef0 con 0x7f499c0af490 2024-04-26T20:57:32.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.216+0000 7f4999600640 1 -- 172.21.15.28:0/823830026 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4994007430 con 0x7f499c0af490 2024-04-26T20:57:32.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.217+0000 7f4999600640 1 --2- 172.21.15.28:0/823830026 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f497c06c920 0x7f497c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:32.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.217+0000 7f4999600640 1 -- 172.21.15.28:0/823830026 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f49940ab980 con 0x7f499c0af490 2024-04-26T20:57:32.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.217+0000 7f49a1c00640 1 --2- 172.21.15.28:0/823830026 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f497c06c920 0x7f497c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:32.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.218+0000 7f49a1c00640 1 --2- 172.21.15.28:0/823830026 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f497c06c920 0x7f497c06ede0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f498c0074a0 tx=0x7f498c00d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:32.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.222+0000 7f4999600640 1 -- 172.21.15.28:0/823830026 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f499c0ceef0 con 0x7f499c0af490 2024-04-26T20:57:32.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.429+0000 7f49a4600640 1 -- 172.21.15.28:0/823830026 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f499c06a360 con 0x7f499c0af490 2024-04-26T20:57:32.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.434+0000 7f4999600640 1 -- 172.21.15.28:0/823830026 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f499c1539b0 con 0x7f499c0af490 2024-04-26T20:57:32.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.437+0000 7f49a4600640 1 -- 172.21.15.28:0/823830026 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f497c06c920 msgr2=0x7f497c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:32.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.437+0000 7f49a4600640 1 --2- 172.21.15.28:0/823830026 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f497c06c920 0x7f497c06ede0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f498c0074a0 tx=0x7f498c00d040 comp rx=0 tx=0).stop 2024-04-26T20:57:32.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.437+0000 7f49a4600640 1 -- 172.21.15.28:0/823830026 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f499c0af490 msgr2=0x7f499c0af940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:32.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.437+0000 7f49a4600640 1 --2- 172.21.15.28:0/823830026 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f499c0af490 0x7f499c0af940 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f499400c1d0 tx=0x7f499c15e4e0 comp rx=0 tx=0).stop 2024-04-26T20:57:32.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.437+0000 7f49a4600640 1 -- 172.21.15.28:0/823830026 shutdown_connections 2024-04-26T20:57:32.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.437+0000 7f49a4600640 1 --2- 172.21.15.28:0/823830026 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f497c06c920 0x7f497c06ede0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:32.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.437+0000 7f49a4600640 1 --2- 172.21.15.28:0/823830026 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f499c0af490 0x7f499c0af940 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:32.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.437+0000 7f49a4600640 1 --2- 172.21.15.28:0/823830026 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f499c0ad470 0x7f499c0ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:32.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.437+0000 7f49a4600640 1 --2- 172.21.15.28:0/823830026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4980005210 0x7f499c14c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:32.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.437+0000 7f49a4600640 1 -- 172.21.15.28:0/823830026 >> 172.21.15.28:0/823830026 conn(0x7f499c06c770 msgr2=0x7f499c0cde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:32.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.438+0000 7f49a4600640 1 -- 172.21.15.28:0/823830026 shutdown_connections 2024-04-26T20:57:32.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:32.438+0000 7f49a4600640 1 -- 172.21.15.28:0/823830026 wait complete. 2024-04-26T20:57:32.459 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T20:57:32.459 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:57:32.573 DEBUG:teuthology.orchestra.run.smithi103:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:57:32.685 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T20:57:32.685 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:57:32.685 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:57:32.746 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:57:32.746 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:57:32.799 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:57:33.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.008+0000 7f8d36200640 1 Processor -- start 2024-04-26T20:57:33.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.008+0000 7f8d36200640 1 -- start start 2024-04-26T20:57:33.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.010+0000 7f8d36200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f8d3006da70 0x7f8d3015f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.010+0000 7f8d36200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d30160350 con 0x7f8d30163840 2024-04-26T20:57:33.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.010+0000 7f8d36200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d30162d60 con 0x7f8d30070690 2024-04-26T20:57:33.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.010+0000 7f8d36200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f8d30162b10 con 0x7f8d3006da70 2024-04-26T20:57:33.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.010+0000 7f8d35800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8d30163840 0x7f8d3015fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58010/0 (socket says 172.21.15.28:58010) 2024-04-26T20:57:33.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.010+0000 7f8d35800640 1 -- 172.21.15.28:0/3475087035 learned_addr learned my addr 172.21.15.28:0/3475087035 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:33.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.011+0000 7f8d2fe00640 1 --2- 172.21.15.28:0/3475087035 >> v2:172.21.15.103:3300/0 conn(0x7f8d3006da70 0x7f8d3015f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.011+0000 7f8d2f400640 1 -- 172.21.15.28:0/3475087035 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1127908490 0 0) 0x7f8d30160350 con 0x7f8d30163840 2024-04-26T20:57:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.011+0000 7f8d2f400640 1 -- 172.21.15.28:0/3475087035 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d18003520 con 0x7f8d30163840 2024-04-26T20:57:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.011+0000 7f8d2f400640 1 -- 172.21.15.28:0/3475087035 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2352857176 0 0) 0x7f8d18003520 con 0x7f8d30163840 2024-04-26T20:57:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.011+0000 7f8d2f400640 1 -- 172.21.15.28:0/3475087035 >> v2:172.21.15.103:3300/0 conn(0x7f8d3006da70 msgr2=0x7f8d3015f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.011+0000 7f8d2f400640 1 --2- 172.21.15.28:0/3475087035 >> v2:172.21.15.103:3300/0 conn(0x7f8d3006da70 0x7f8d3015f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.011+0000 7f8d2f400640 1 -- 172.21.15.28:0/3475087035 >> v1:172.21.15.103:6789/0 conn(0x7f8d30070690 legacy=0x7f8d3006d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.011+0000 7f8d2f400640 1 -- 172.21.15.28:0/3475087035 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d30163040 con 0x7f8d30163840 2024-04-26T20:57:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.011+0000 7f8d2f400640 1 -- 172.21.15.28:0/3475087035 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f8d280019f0 con 0x7f8d30163840 2024-04-26T20:57:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.011+0000 7f8d2f400640 1 -- 172.21.15.28:0/3475087035 >> v1:172.21.15.28:6789/0 conn(0x7f8d30163840 legacy=0x7f8d3015fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.011+0000 7f8d2f400640 1 --2- 172.21.15.28:0/3475087035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d18003a00 0x7f8d18003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.011+0000 7f8d2f400640 1 --2- 172.21.15.28:0/3475087035 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d18004800 0x7f8d18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.011+0000 7f8d2f400640 1 --2- 172.21.15.28:0/3475087035 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8d18005230 0x7f8d1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.011+0000 7f8d2f400640 1 -- 172.21.15.28:0/3475087035 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8d30162b10 con 0x7f8d18004800 2024-04-26T20:57:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.011+0000 7f8d2f400640 1 -- 172.21.15.28:0/3475087035 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1800c190 con 0x7f8d18003a00 2024-04-26T20:57:33.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.011+0000 7f8d2f400640 1 -- 172.21.15.28:0/3475087035 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f8d1800c390 con 0x7f8d18005230 2024-04-26T20:57:33.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.012+0000 7f8d2fe00640 1 --2- 172.21.15.28:0/3475087035 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d18004800 0x7f8d18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:33.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.012+0000 7f8d34e00640 1 --2- 172.21.15.28:0/3475087035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d18003a00 0x7f8d18003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:33.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.012+0000 7f8d2fe00640 1 -- 172.21.15.28:0/3475087035 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8d18005230 msgr2=0x7f8d1800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:33.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.012+0000 7f8d35800640 1 --2- 172.21.15.28:0/3475087035 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8d18005230 0x7f8d1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:33.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.012+0000 7f8d2fe00640 1 --2- 172.21.15.28:0/3475087035 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8d18005230 0x7f8d1800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.012+0000 7f8d2fe00640 1 -- 172.21.15.28:0/3475087035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d18003a00 msgr2=0x7f8d18003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:33.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.012+0000 7f8d2fe00640 1 --2- 172.21.15.28:0/3475087035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d18003a00 0x7f8d18003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.012+0000 7f8d2fe00640 1 -- 172.21.15.28:0/3475087035 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d18003520 con 0x7f8d18004800 2024-04-26T20:57:33.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.012+0000 7f8d35800640 1 -- 172.21.15.28:0/3475087035 reap_dead start 2024-04-26T20:57:33.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.013+0000 7f8d2fe00640 1 --2- 172.21.15.28:0/3475087035 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d18004800 0x7f8d18004c80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8d20009ac0 tx=0x7f8d18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ad6cd636d8c4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:33.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.014+0000 7f8d2f400640 1 -- 172.21.15.28:0/3475087035 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d20036070 con 0x7f8d18004800 2024-04-26T20:57:33.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.014+0000 7f8d2f400640 1 -- 172.21.15.28:0/3475087035 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d20031400 con 0x7f8d18004800 2024-04-26T20:57:33.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.014+0000 7f8d2f400640 1 -- 172.21.15.28:0/3475087035 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d20009690 con 0x7f8d18004800 2024-04-26T20:57:33.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.014+0000 7f8d36200640 1 -- 172.21.15.28:0/3475087035 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d18004800 msgr2=0x7f8d18004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:33.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.014+0000 7f8d36200640 1 --2- 172.21.15.28:0/3475087035 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d18004800 0x7f8d18004c80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8d20009ac0 tx=0x7f8d18005830 comp rx=0 tx=0).stop 2024-04-26T20:57:33.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.014+0000 7f8d36200640 1 -- 172.21.15.28:0/3475087035 shutdown_connections 2024-04-26T20:57:33.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.014+0000 7f8d36200640 1 --2- 172.21.15.28:0/3475087035 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d18004800 0x7f8d18004c80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.014+0000 7f8d36200640 1 -- 172.21.15.28:0/3475087035 >> 172.21.15.28:0/3475087035 conn(0x7f8d30072f10 msgr2=0x7f8d30073320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:33.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.014+0000 7f8d36200640 1 -- 172.21.15.28:0/3475087035 shutdown_connections 2024-04-26T20:57:33.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.014+0000 7f8d36200640 1 -- 172.21.15.28:0/3475087035 wait complete. 2024-04-26T20:57:33.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.016+0000 7f8d36200640 1 Processor -- start 2024-04-26T20:57:33.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.016+0000 7f8d36200640 1 -- start start 2024-04-26T20:57:33.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.017+0000 7f8d36200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d18004800 0x7f8d301552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.017+0000 7f8d36200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8d300d0060 0x7f8d300d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.017+0000 7f8d36200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d300aeda0 0x7f8d300af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.017+0000 7f8d36200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3006d360 con 0x7f8d18004800 2024-04-26T20:57:33.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.017+0000 7f8d36200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8d30070a70 con 0x7f8d300aeda0 2024-04-26T20:57:33.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.017+0000 7f8d36200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f8d3006d4e0 con 0x7f8d300d0060 2024-04-26T20:57:33.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.018+0000 7f8d34e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d18004800 0x7f8d301552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:33.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.018+0000 7f8d34e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d18004800 0x7f8d301552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54768/0 (socket says 172.21.15.28:54768) 2024-04-26T20:57:33.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.018+0000 7f8d34e00640 1 -- 172.21.15.28:0/961472360 learned_addr learned my addr 172.21.15.28:0/961472360 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:33.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.018+0000 7f8d2fe00640 1 --2- 172.21.15.28:0/961472360 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8d300d0060 0x7f8d300d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:33.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.018+0000 7f8d35800640 1 --2- 172.21.15.28:0/961472360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d300aeda0 0x7f8d300af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:33.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.018+0000 7f8d34e00640 1 -- 172.21.15.28:0/961472360 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8d300d0060 msgr2=0x7f8d300d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:33.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.018+0000 7f8d34e00640 1 --2- 172.21.15.28:0/961472360 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8d300d0060 0x7f8d300d04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.018+0000 7f8d34e00640 1 -- 172.21.15.28:0/961472360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d300aeda0 msgr2=0x7f8d300af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:33.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.018+0000 7f8d34e00640 1 --2- 172.21.15.28:0/961472360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d300aeda0 0x7f8d300af220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.018+0000 7f8d34e00640 1 -- 172.21.15.28:0/961472360 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d300ceac0 con 0x7f8d18004800 2024-04-26T20:57:33.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.018+0000 7f8d34e00640 1 --2- 172.21.15.28:0/961472360 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d18004800 0x7f8d301552b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400e580 tx=0x7f8d2400ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:33.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.019+0000 7f8d2cc00640 1 -- 172.21.15.28:0/961472360 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d240052e0 con 0x7f8d18004800 2024-04-26T20:57:33.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.019+0000 7f8d2cc00640 1 -- 172.21.15.28:0/961472360 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d24018070 con 0x7f8d18004800 2024-04-26T20:57:33.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.019+0000 7f8d2cc00640 1 -- 172.21.15.28:0/961472360 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24003810 con 0x7f8d18004800 2024-04-26T20:57:33.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.019+0000 7f8d36200640 1 -- 172.21.15.28:0/961472360 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d300d0be0 con 0x7f8d18004800 2024-04-26T20:57:33.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.019+0000 7f8d36200640 1 -- 172.21.15.28:0/961472360 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d300ce2c0 con 0x7f8d18004800 2024-04-26T20:57:33.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.020+0000 7f8d36200640 1 -- 172.21.15.28:0/961472360 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d3006a360 con 0x7f8d18004800 2024-04-26T20:57:33.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.021+0000 7f8d2cc00640 1 -- 172.21.15.28:0/961472360 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8d240039b0 con 0x7f8d18004800 2024-04-26T20:57:33.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.023+0000 7f8d2cc00640 1 --2- 172.21.15.28:0/961472360 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8d1406c920 0x7f8d1406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.023+0000 7f8d2cc00640 1 -- 172.21.15.28:0/961472360 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8d2408b840 con 0x7f8d18004800 2024-04-26T20:57:33.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.023+0000 7f8d2fe00640 1 --2- 172.21.15.28:0/961472360 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8d1406c920 0x7f8d1406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:33.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.024+0000 7f8d2fe00640 1 --2- 172.21.15.28:0/961472360 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8d1406c920 0x7f8d1406ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8d20033000 tx=0x7f8d20008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:33.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.027+0000 7f8d2cc00640 1 -- 172.21.15.28:0/961472360 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8d24059be0 con 0x7f8d18004800 2024-04-26T20:57:33.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.235+0000 7f8d36200640 1 -- 172.21.15.28:0/961472360 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d3015f910 con 0x7f8d18004800 2024-04-26T20:57:33.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.237+0000 7f8d2cc00640 1 -- 172.21.15.28:0/961472360 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7f8d240594e0 con 0x7f8d18004800 2024-04-26T20:57:33.239 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:33.239 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":21,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:56:52.699663+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:57:33.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.240+0000 7f8d36200640 1 -- 172.21.15.28:0/961472360 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8d1406c920 msgr2=0x7f8d1406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:33.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.240+0000 7f8d36200640 1 --2- 172.21.15.28:0/961472360 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8d1406c920 0x7f8d1406ede0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8d20033000 tx=0x7f8d20008030 comp rx=0 tx=0).stop 2024-04-26T20:57:33.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.241+0000 7f8d36200640 1 -- 172.21.15.28:0/961472360 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d18004800 msgr2=0x7f8d301552b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:33.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.241+0000 7f8d36200640 1 --2- 172.21.15.28:0/961472360 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d18004800 0x7f8d301552b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400e580 tx=0x7f8d2400ea40 comp rx=0 tx=0).stop 2024-04-26T20:57:33.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.241+0000 7f8d36200640 1 -- 172.21.15.28:0/961472360 shutdown_connections 2024-04-26T20:57:33.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.241+0000 7f8d36200640 1 --2- 172.21.15.28:0/961472360 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8d1406c920 0x7f8d1406ede0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.241+0000 7f8d36200640 1 --2- 172.21.15.28:0/961472360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d300aeda0 0x7f8d300af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.241+0000 7f8d36200640 1 --2- 172.21.15.28:0/961472360 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8d300d0060 0x7f8d300d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.241+0000 7f8d36200640 1 --2- 172.21.15.28:0/961472360 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d18004800 0x7f8d301552b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.241+0000 7f8d36200640 1 -- 172.21.15.28:0/961472360 >> 172.21.15.28:0/961472360 conn(0x7f8d30072f10 msgr2=0x7f8d30070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:33.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.241+0000 7f8d36200640 1 -- 172.21.15.28:0/961472360 shutdown_connections 2024-04-26T20:57:33.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.241+0000 7f8d36200640 1 -- 172.21.15.28:0/961472360 wait complete. 2024-04-26T20:57:33.262 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:56:48.688498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:56:50.117541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:57:33.263 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:57:33.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.477+0000 7fb6a5000640 1 Processor -- start 2024-04-26T20:57:33.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.478+0000 7fb6a5000640 1 -- start start 2024-04-26T20:57:33.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.479+0000 7fb6a5000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6a0166f50 0x7fb6a00d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.479+0000 7fb6a5000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6a00721e0 con 0x7fb6a00d1250 2024-04-26T20:57:33.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.479+0000 7fb6a5000640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6a014d220 con 0x7fb6a014ce40 2024-04-26T20:57:33.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.479+0000 7fb6a5000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a00cd4e0 con 0x7fb6a0166f50 2024-04-26T20:57:33.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.480+0000 7fb69fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb6a00d1250 0x7fb6a0072690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58022/0 (socket says 172.21.15.28:58022) 2024-04-26T20:57:33.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.480+0000 7fb69fe00640 1 -- 172.21.15.28:0/582618415 learned_addr learned my addr 172.21.15.28:0/582618415 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:33.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.480+0000 7fb697e00640 1 --2- 172.21.15.28:0/582618415 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6a0166f50 0x7fb6a00d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:33.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.480+0000 7fb69ea00640 1 -- 172.21.15.28:0/582618415 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4289808171 0 0) 0x7fb6a00721e0 con 0x7fb6a00d1250 2024-04-26T20:57:33.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.480+0000 7fb69ea00640 1 -- 172.21.15.28:0/582618415 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb684003520 con 0x7fb6a00d1250 2024-04-26T20:57:33.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.481+0000 7fb69ea00640 1 -- 172.21.15.28:0/582618415 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2070006109 0 0) 0x7fb684003520 con 0x7fb6a00d1250 2024-04-26T20:57:33.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.481+0000 7fb69ea00640 1 -- 172.21.15.28:0/582618415 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6a0166f50 msgr2=0x7fb6a00d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:33.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.481+0000 7fb69ea00640 1 --2- 172.21.15.28:0/582618415 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6a0166f50 0x7fb6a00d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.481+0000 7fb69ea00640 1 -- 172.21.15.28:0/582618415 >> v1:172.21.15.103:6789/0 conn(0x7fb6a014ce40 legacy=0x7fb6a006f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:33.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.481+0000 7fb69ea00640 1 -- 172.21.15.28:0/582618415 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a01674c0 con 0x7fb6a00d1250 2024-04-26T20:57:33.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.481+0000 7fb69ea00640 1 -- 172.21.15.28:0/582618415 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fb6980019f0 con 0x7fb6a00d1250 2024-04-26T20:57:33.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.481+0000 7fb69ea00640 1 -- 172.21.15.28:0/582618415 >> v1:172.21.15.28:6789/0 conn(0x7fb6a00d1250 legacy=0x7fb6a0072690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:33.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.481+0000 7fb69ea00640 1 --2- 172.21.15.28:0/582618415 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6840039e0 0x7fb684003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.481+0000 7fb69ea00640 1 --2- 172.21.15.28:0/582618415 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6a0166f50 0x7fb684004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.481+0000 7fb69ea00640 1 --2- 172.21.15.28:0/582618415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb684004d40 0x7fb6840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.481+0000 7fb69ea00640 1 -- 172.21.15.28:0/582618415 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb68400ba70 con 0x7fb6840039e0 2024-04-26T20:57:33.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.481+0000 7fb69ea00640 1 -- 172.21.15.28:0/582618415 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb68400bc20 con 0x7fb684004d40 2024-04-26T20:57:33.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.481+0000 7fb69ea00640 1 -- 172.21.15.28:0/582618415 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb68400be20 con 0x7fb6a0166f50 2024-04-26T20:57:33.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.481+0000 7fb69f400640 1 --2- 172.21.15.28:0/582618415 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6840039e0 0x7fb684003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:33.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.481+0000 7fb69fe00640 1 --2- 172.21.15.28:0/582618415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb684004d40 0x7fb6840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:33.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.482+0000 7fb697e00640 1 --2- 172.21.15.28:0/582618415 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6a0166f50 0x7fb684004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:33.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.482+0000 7fb69f400640 1 -- 172.21.15.28:0/582618415 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6a0166f50 msgr2=0x7fb684004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:33.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.482+0000 7fb69f400640 1 --2- 172.21.15.28:0/582618415 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6a0166f50 0x7fb684004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.482+0000 7fb69f400640 1 -- 172.21.15.28:0/582618415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb684004d40 msgr2=0x7fb6840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:33.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.482+0000 7fb69f400640 1 --2- 172.21.15.28:0/582618415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb684004d40 0x7fb6840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.482+0000 7fb69f400640 1 -- 172.21.15.28:0/582618415 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb684003520 con 0x7fb6840039e0 2024-04-26T20:57:33.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.482+0000 7fb69fe00640 1 --2- 172.21.15.28:0/582618415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb684004d40 0x7fb6840051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:33.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.483+0000 7fb69f400640 1 --2- 172.21.15.28:0/582618415 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6840039e0 0x7fb684003e00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb690001720 tx=0x7fb684005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96257ee896ca0e43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:33.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.483+0000 7fb69ea00640 1 -- 172.21.15.28:0/582618415 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690001370 con 0x7fb6840039e0 2024-04-26T20:57:33.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.483+0000 7fb69ea00640 1 -- 172.21.15.28:0/582618415 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6900052d0 con 0x7fb6840039e0 2024-04-26T20:57:33.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.483+0000 7fb69ea00640 1 -- 172.21.15.28:0/582618415 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69002abc0 con 0x7fb6840039e0 2024-04-26T20:57:33.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.483+0000 7fb6a5000640 1 -- 172.21.15.28:0/582618415 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6840039e0 msgr2=0x7fb684003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:33.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.483+0000 7fb6a5000640 1 --2- 172.21.15.28:0/582618415 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6840039e0 0x7fb684003e00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb690001720 tx=0x7fb684005830 comp rx=0 tx=0).stop 2024-04-26T20:57:33.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.484+0000 7fb69fe00640 1 -- 172.21.15.28:0/582618415 reap_dead start 2024-04-26T20:57:33.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.484+0000 7fb6a5000640 1 -- 172.21.15.28:0/582618415 shutdown_connections 2024-04-26T20:57:33.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.484+0000 7fb6a5000640 1 -- 172.21.15.28:0/582618415 >> 172.21.15.28:0/582618415 conn(0x7fb6a006c530 msgr2=0x7fb6a006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:33.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.484+0000 7fb6a5000640 1 -- 172.21.15.28:0/582618415 shutdown_connections 2024-04-26T20:57:33.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.484+0000 7fb6a5000640 1 -- 172.21.15.28:0/582618415 wait complete. 2024-04-26T20:57:33.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.485+0000 7fb6a5000640 1 Processor -- start 2024-04-26T20:57:33.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.486+0000 7fb6a5000640 1 -- start start 2024-04-26T20:57:33.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.487+0000 7fb6a5000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6840039e0 0x7fb6a006f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.487+0000 7fb6a5000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6a01478c0 0x7fb6a0147d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.487+0000 7fb6a5000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6a00b1b50 0x7fb6a00b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.487+0000 7fb6a5000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a0167d60 con 0x7fb6840039e0 2024-04-26T20:57:33.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.487+0000 7fb6a5000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a0167a80 con 0x7fb6a00b1b50 2024-04-26T20:57:33.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.487+0000 7fb6a5000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb6a00d0d10 con 0x7fb6a01478c0 2024-04-26T20:57:33.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.487+0000 7fb69f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6840039e0 0x7fb6a006f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:33.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.487+0000 7fb69f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6840039e0 0x7fb6a006f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54776/0 (socket says 172.21.15.28:54776) 2024-04-26T20:57:33.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.487+0000 7fb69f400640 1 -- 172.21.15.28:0/3016802150 learned_addr learned my addr 172.21.15.28:0/3016802150 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:33.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.487+0000 7fb69f400640 1 -- 172.21.15.28:0/3016802150 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6a01478c0 msgr2=0x7fb6a0147d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:33.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.487+0000 7fb69f400640 1 --2- 172.21.15.28:0/3016802150 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6a01478c0 0x7fb6a0147d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.487+0000 7fb69f400640 1 -- 172.21.15.28:0/3016802150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6a00b1b50 msgr2=0x7fb6a00b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:33.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.487+0000 7fb69f400640 1 --2- 172.21.15.28:0/3016802150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6a00b1b50 0x7fb6a00b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.487+0000 7fb69f400640 1 -- 172.21.15.28:0/3016802150 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a0148280 con 0x7fb6840039e0 2024-04-26T20:57:33.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.488+0000 7fb69f400640 1 --2- 172.21.15.28:0/3016802150 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6840039e0 0x7fb6a006f6c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb690008fd0 tx=0x7fb690001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:33.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.488+0000 7fb697400640 1 -- 172.21.15.28:0/3016802150 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690037040 con 0x7fb6840039e0 2024-04-26T20:57:33.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.488+0000 7fb697400640 1 -- 172.21.15.28:0/3016802150 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb690005480 con 0x7fb6840039e0 2024-04-26T20:57:33.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.488+0000 7fb697400640 1 -- 172.21.15.28:0/3016802150 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb690007a30 con 0x7fb6840039e0 2024-04-26T20:57:33.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.488+0000 7fb6a5000640 1 -- 172.21.15.28:0/3016802150 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a00b28c0 con 0x7fb6840039e0 2024-04-26T20:57:33.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.488+0000 7fb6a5000640 1 -- 172.21.15.28:0/3016802150 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a00b2e80 con 0x7fb6840039e0 2024-04-26T20:57:33.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.489+0000 7fb6a5000640 1 -- 172.21.15.28:0/3016802150 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6a006a250 con 0x7fb6840039e0 2024-04-26T20:57:33.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.490+0000 7fb697400640 1 -- 172.21.15.28:0/3016802150 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb690044020 con 0x7fb6840039e0 2024-04-26T20:57:33.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.492+0000 7fb697400640 1 --2- 172.21.15.28:0/3016802150 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb67c06c920 0x7fb67c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.492+0000 7fb697400640 1 -- 172.21.15.28:0/3016802150 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb6900ab600 con 0x7fb6840039e0 2024-04-26T20:57:33.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.492+0000 7fb697e00640 1 --2- 172.21.15.28:0/3016802150 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb67c06c920 0x7fb67c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:33.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.493+0000 7fb697e00640 1 --2- 172.21.15.28:0/3016802150 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb67c06c920 0x7fb67c06ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0149990 tx=0x7fb688008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:33.497 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.496+0000 7fb697400640 1 -- 172.21.15.28:0/3016802150 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb69004b020 con 0x7fb6840039e0 2024-04-26T20:57:33.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.699+0000 7fb6a5000640 1 -- 172.21.15.28:0/3016802150 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb6a0166cc0 con 0x7fb6840039e0 2024-04-26T20:57:33.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.700+0000 7fb697400640 1 -- 172.21.15.28:0/3016802150 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fb690079a20 con 0x7fb6840039e0 2024-04-26T20:57:33.701 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:57:33.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.703+0000 7fb6a5000640 1 -- 172.21.15.28:0/3016802150 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb67c06c920 msgr2=0x7fb67c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:33.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.703+0000 7fb6a5000640 1 --2- 172.21.15.28:0/3016802150 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb67c06c920 0x7fb67c06ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb6a0149990 tx=0x7fb688008040 comp rx=0 tx=0).stop 2024-04-26T20:57:33.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.703+0000 7fb6a5000640 1 -- 172.21.15.28:0/3016802150 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6840039e0 msgr2=0x7fb6a006f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:33.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.703+0000 7fb6a5000640 1 --2- 172.21.15.28:0/3016802150 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6840039e0 0x7fb6a006f6c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb690008fd0 tx=0x7fb690001120 comp rx=0 tx=0).stop 2024-04-26T20:57:33.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.703+0000 7fb6a5000640 1 -- 172.21.15.28:0/3016802150 shutdown_connections 2024-04-26T20:57:33.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.703+0000 7fb6a5000640 1 --2- 172.21.15.28:0/3016802150 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb67c06c920 0x7fb67c06ede0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.703+0000 7fb6a5000640 1 --2- 172.21.15.28:0/3016802150 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb6a00b1b50 0x7fb6a00b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.703+0000 7fb6a5000640 1 --2- 172.21.15.28:0/3016802150 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb6a01478c0 0x7fb6a0147d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.703+0000 7fb6a5000640 1 --2- 172.21.15.28:0/3016802150 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6840039e0 0x7fb6a006f6c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.703+0000 7fb6a5000640 1 -- 172.21.15.28:0/3016802150 >> 172.21.15.28:0/3016802150 conn(0x7fb6a006c530 msgr2=0x7fb6a00cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:33.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.703+0000 7fb6a5000640 1 -- 172.21.15.28:0/3016802150 shutdown_connections 2024-04-26T20:57:33.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.703+0000 7fb6a5000640 1 -- 172.21.15.28:0/3016802150 wait complete. 2024-04-26T20:57:33.726 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:57:33.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.939+0000 7f293b200640 1 Processor -- start 2024-04-26T20:57:33.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.939+0000 7f293b200640 1 -- start start 2024-04-26T20:57:33.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.940+0000 7f293b200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f29340b5760 0x7f29340af3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.941+0000 7f293b200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29340b5be0 con 0x7f293415fc80 2024-04-26T20:57:33.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.941+0000 7f293b200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29340b6420 con 0x7f29340af910 2024-04-26T20:57:33.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.941+0000 7f293b200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f29340b5e60 con 0x7f29340b5760 2024-04-26T20:57:33.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.941+0000 7f2933e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f293415fc80 0x7f2934160060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58036/0 (socket says 172.21.15.28:58036) 2024-04-26T20:57:33.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.941+0000 7f2933e00640 1 -- 172.21.15.28:0/3945654139 learned_addr learned my addr 172.21.15.28:0/3945654139 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:33.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.941+0000 7f2933400640 1 --2- 172.21.15.28:0/3945654139 >> v2:172.21.15.28:3300/0 conn(0x7f29340b5760 0x7f29340af3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:33.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.941+0000 7f2932a00640 1 -- 172.21.15.28:0/3945654139 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 365412013 0 0) 0x7f2924001490 con 0x7f293415fc80 2024-04-26T20:57:33.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.941+0000 7f2932a00640 1 -- 172.21.15.28:0/3945654139 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2918003520 con 0x7f293415fc80 2024-04-26T20:57:33.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.942+0000 7f2932a00640 1 -- 172.21.15.28:0/3945654139 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1225520356 0 0) 0x7f2918003520 con 0x7f293415fc80 2024-04-26T20:57:33.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.942+0000 7f2933400640 1 -- 172.21.15.28:0/3945654139 >> v1:172.21.15.103:6789/0 conn(0x7f29340af910 legacy=0x7f293406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:33.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.942+0000 7f2933400640 1 -- 172.21.15.28:0/3945654139 >> v1:172.21.15.28:6789/0 conn(0x7f293415fc80 legacy=0x7f2934160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:33.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.942+0000 7f2933400640 1 -- 172.21.15.28:0/3945654139 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29340b6140 con 0x7f29340b5760 2024-04-26T20:57:33.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.943+0000 7f2933400640 1 --2- 172.21.15.28:0/3945654139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f29340b5760 0x7f29340af3d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f291c0097a0 tx=0x7f291c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c35b819c4bdfb1c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:33.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.943+0000 7f2932a00640 1 -- 172.21.15.28:0/3945654139 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291c037070 con 0x7f29340b5760 2024-04-26T20:57:33.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.943+0000 7f2932a00640 1 -- 172.21.15.28:0/3945654139 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f291c02ac50 con 0x7f29340b5760 2024-04-26T20:57:33.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.943+0000 7f2932a00640 1 -- 172.21.15.28:0/3945654139 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924001bc0 con 0x7f29340b5760 2024-04-26T20:57:33.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.944+0000 7f293b200640 1 -- 172.21.15.28:0/3945654139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f29340b5760 msgr2=0x7f29340af3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:33.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.944+0000 7f293b200640 1 --2- 172.21.15.28:0/3945654139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f29340b5760 0x7f29340af3d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f291c0097a0 tx=0x7f291c02a1c0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.944+0000 7f293b200640 1 -- 172.21.15.28:0/3945654139 shutdown_connections 2024-04-26T20:57:33.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.944+0000 7f293b200640 1 --2- 172.21.15.28:0/3945654139 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f29340b5760 0x7f29340af3d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.944+0000 7f293b200640 1 -- 172.21.15.28:0/3945654139 >> 172.21.15.28:0/3945654139 conn(0x7f29340b3360 msgr2=0x7f29340b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:33.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.944+0000 7f293b200640 1 -- 172.21.15.28:0/3945654139 shutdown_connections 2024-04-26T20:57:33.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.944+0000 7f293b200640 1 -- 172.21.15.28:0/3945654139 wait complete. 2024-04-26T20:57:33.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.946+0000 7f293b200640 1 Processor -- start 2024-04-26T20:57:33.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.946+0000 7f293b200640 1 -- start start 2024-04-26T20:57:33.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.948+0000 7f293b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29340af910 0x7f2934156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.948+0000 7f293b200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f29340b5760 0x7f2934158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.948+0000 7f293b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f293415fc80 0x7f29341f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.948+0000 7f293b200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f293406f990 con 0x7f293415fc80 2024-04-26T20:57:33.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.948+0000 7f293b200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f293406ed60 con 0x7f29340af910 2024-04-26T20:57:33.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.948+0000 7f293b200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f293406fb10 con 0x7f29340b5760 2024-04-26T20:57:33.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.948+0000 7f2939200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f293415fc80 0x7f29341f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:33.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.948+0000 7f2939200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f293415fc80 0x7f29341f42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54790/0 (socket says 172.21.15.28:54790) 2024-04-26T20:57:33.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.948+0000 7f2939200640 1 -- 172.21.15.28:0/1364042069 learned_addr learned my addr 172.21.15.28:0/1364042069 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:33.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.948+0000 7f2933e00640 1 --2- 172.21.15.28:0/1364042069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29340af910 0x7f2934156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:33.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.948+0000 7f2933400640 1 --2- 172.21.15.28:0/1364042069 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f29340b5760 0x7f2934158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:33.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.949+0000 7f2939200640 1 -- 172.21.15.28:0/1364042069 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f29340b5760 msgr2=0x7f2934158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:33.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.949+0000 7f2939200640 1 --2- 172.21.15.28:0/1364042069 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f29340b5760 0x7f2934158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.949+0000 7f2939200640 1 -- 172.21.15.28:0/1364042069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29340af910 msgr2=0x7f2934156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:33.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.949+0000 7f2939200640 1 --2- 172.21.15.28:0/1364042069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29340af910 0x7f2934156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:33.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.949+0000 7f2939200640 1 -- 172.21.15.28:0/1364042069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2934159dc0 con 0x7f293415fc80 2024-04-26T20:57:33.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.949+0000 7f2933e00640 1 --2- 172.21.15.28:0/1364042069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29340af910 0x7f2934156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:33.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.949+0000 7f2939200640 1 --2- 172.21.15.28:0/1364042069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f293415fc80 0x7f29341f42f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f292c00c400 tx=0x7f292c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:33.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.949+0000 7f292be00640 1 -- 172.21.15.28:0/1364042069 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c017070 con 0x7f293415fc80 2024-04-26T20:57:33.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.950+0000 7f292be00640 1 -- 172.21.15.28:0/1364042069 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f292c0052d0 con 0x7f293415fc80 2024-04-26T20:57:33.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.950+0000 7f292be00640 1 -- 172.21.15.28:0/1364042069 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c003810 con 0x7f293415fc80 2024-04-26T20:57:33.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.950+0000 7f293b200640 1 -- 172.21.15.28:0/1364042069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2934157920 con 0x7f293415fc80 2024-04-26T20:57:33.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.950+0000 7f293b200640 1 -- 172.21.15.28:0/1364042069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2934157ee0 con 0x7f293415fc80 2024-04-26T20:57:33.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.951+0000 7f293b200640 1 -- 172.21.15.28:0/1364042069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f293406a360 con 0x7f293415fc80 2024-04-26T20:57:33.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.952+0000 7f292be00640 1 -- 172.21.15.28:0/1364042069 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f292c01a070 con 0x7f293415fc80 2024-04-26T20:57:33.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.953+0000 7f292be00640 1 --2- 172.21.15.28:0/1364042069 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f291006c8f0 0x7f291006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:33.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.953+0000 7f292be00640 1 -- 172.21.15.28:0/1364042069 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f292c08af30 con 0x7f293415fc80 2024-04-26T20:57:33.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.953+0000 7f2933e00640 1 --2- 172.21.15.28:0/1364042069 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f291006c8f0 0x7f291006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:33.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.954+0000 7f2933e00640 1 --2- 172.21.15.28:0/1364042069 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f291006c8f0 0x7f291006edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2924002230 tx=0x7f2924009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:33.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:33.958+0000 7f292be00640 1 -- 172.21.15.28:0/1364042069 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f292c059350 con 0x7f293415fc80 2024-04-26T20:57:34.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.163+0000 7f293b200640 1 -- 172.21.15.28:0/1364042069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f293406d840 con 0x7f293415fc80 2024-04-26T20:57:34.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.164+0000 7f292be00640 1 -- 172.21.15.28:0/1364042069 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f292c058c50 con 0x7f293415fc80 2024-04-26T20:57:34.165 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-04-26T20:57:34.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.166+0000 7f293b200640 1 -- 172.21.15.28:0/1364042069 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f291006c8f0 msgr2=0x7f291006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:34.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.166+0000 7f293b200640 1 --2- 172.21.15.28:0/1364042069 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f291006c8f0 0x7f291006edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2924002230 tx=0x7f2924009040 comp rx=0 tx=0).stop 2024-04-26T20:57:34.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.166+0000 7f293b200640 1 -- 172.21.15.28:0/1364042069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f293415fc80 msgr2=0x7f29341f42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:34.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.166+0000 7f293b200640 1 --2- 172.21.15.28:0/1364042069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f293415fc80 0x7f29341f42f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f292c00c400 tx=0x7f292c00c8c0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.167+0000 7f293b200640 1 -- 172.21.15.28:0/1364042069 shutdown_connections 2024-04-26T20:57:34.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.167+0000 7f293b200640 1 --2- 172.21.15.28:0/1364042069 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f291006c8f0 0x7f291006edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.167+0000 7f293b200640 1 --2- 172.21.15.28:0/1364042069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f293415fc80 0x7f29341f42f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.167+0000 7f293b200640 1 --2- 172.21.15.28:0/1364042069 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f29340b5760 0x7f2934158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.167+0000 7f293b200640 1 --2- 172.21.15.28:0/1364042069 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29340af910 0x7f2934156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.167+0000 7f293b200640 1 -- 172.21.15.28:0/1364042069 >> 172.21.15.28:0/1364042069 conn(0x7f29340b3360 msgr2=0x7f29340b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:34.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.167+0000 7f293b200640 1 -- 172.21.15.28:0/1364042069 shutdown_connections 2024-04-26T20:57:34.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.167+0000 7f293b200640 1 -- 172.21.15.28:0/1364042069 wait complete. 2024-04-26T20:57:34.185 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:57:34.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.396+0000 7f5192c00640 1 Processor -- start 2024-04-26T20:57:34.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.397+0000 7f5192c00640 1 -- start start 2024-04-26T20:57:34.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.398+0000 7f5192c00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f518c0d0c70 0x7f518c161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:34.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.398+0000 7f5192c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f518c06ed40 con 0x7f518c0b4a30 2024-04-26T20:57:34.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.398+0000 7f5192c00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f518c161860 con 0x7f518c0d1050 2024-04-26T20:57:34.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.399+0000 7f5192c00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f518c0cf2f0 con 0x7f518c0d0c70 2024-04-26T20:57:34.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.399+0000 7f5192200640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f518c0b4a30 0x7f518c06e630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58042/0 (socket says 172.21.15.28:58042) 2024-04-26T20:57:34.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.399+0000 7f5192200640 1 -- 172.21.15.28:0/479497202 learned_addr learned my addr 172.21.15.28:0/479497202 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:34.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.399+0000 7f5191800640 1 --2- 172.21.15.28:0/479497202 >> v2:172.21.15.103:3300/0 conn(0x7f518c0d0c70 0x7f518c161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:34.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.399+0000 7f518be00640 1 -- 172.21.15.28:0/479497202 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 673408009 0 0) 0x7f518c06ed40 con 0x7f518c0b4a30 2024-04-26T20:57:34.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.399+0000 7f518be00640 1 -- 172.21.15.28:0/479497202 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5174003520 con 0x7f518c0b4a30 2024-04-26T20:57:34.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.400+0000 7f518be00640 1 -- 172.21.15.28:0/479497202 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1853846362 0 0) 0x7f5174003520 con 0x7f518c0b4a30 2024-04-26T20:57:34.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.400+0000 7f518be00640 1 -- 172.21.15.28:0/479497202 >> v2:172.21.15.103:3300/0 conn(0x7f518c0d0c70 msgr2=0x7f518c161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:34.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.400+0000 7f518be00640 1 --2- 172.21.15.28:0/479497202 >> v2:172.21.15.103:3300/0 conn(0x7f518c0d0c70 0x7f518c161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.400+0000 7f518be00640 1 -- 172.21.15.28:0/479497202 >> v1:172.21.15.103:6789/0 conn(0x7f518c0d1050 legacy=0x7f518c06df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:34.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.400+0000 7f518be00640 1 -- 172.21.15.28:0/479497202 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518c0cf5d0 con 0x7f518c0b4a30 2024-04-26T20:57:34.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.400+0000 7f518be00640 1 -- 172.21.15.28:0/479497202 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f51840019f0 con 0x7f518c0b4a30 2024-04-26T20:57:34.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.400+0000 7f518be00640 1 -- 172.21.15.28:0/479497202 >> v1:172.21.15.28:6789/0 conn(0x7f518c0b4a30 legacy=0x7f518c06e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:34.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.400+0000 7f518be00640 1 --2- 172.21.15.28:0/479497202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5174003a00 0x7f5174003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:34.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.400+0000 7f518be00640 1 --2- 172.21.15.28:0/479497202 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5174004800 0x7f5174004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:34.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.400+0000 7f518be00640 1 --2- 172.21.15.28:0/479497202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51740051c0 0x7f517400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:34.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.400+0000 7f518be00640 1 -- 172.21.15.28:0/479497202 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f518c0cf2f0 con 0x7f51740051c0 2024-04-26T20:57:34.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.400+0000 7f518be00640 1 -- 172.21.15.28:0/479497202 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f517400c0f0 con 0x7f5174003a00 2024-04-26T20:57:34.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.400+0000 7f518be00640 1 -- 172.21.15.28:0/479497202 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f517400c2f0 con 0x7f5174004800 2024-04-26T20:57:34.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.400+0000 7f5192200640 1 --2- 172.21.15.28:0/479497202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51740051c0 0x7f517400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:34.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.401+0000 7f5191800640 1 --2- 172.21.15.28:0/479497202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5174003a00 0x7f5174003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:34.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.401+0000 7f5190e00640 1 --2- 172.21.15.28:0/479497202 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5174004800 0x7f5174004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:34.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.401+0000 7f5192200640 1 -- 172.21.15.28:0/479497202 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5174004800 msgr2=0x7f5174004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:34.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.401+0000 7f5192200640 1 --2- 172.21.15.28:0/479497202 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5174004800 0x7f5174004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.401+0000 7f5192200640 1 -- 172.21.15.28:0/479497202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5174003a00 msgr2=0x7f5174003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:34.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.401+0000 7f5192200640 1 --2- 172.21.15.28:0/479497202 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5174003a00 0x7f5174003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.401+0000 7f5192200640 1 -- 172.21.15.28:0/479497202 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5174003520 con 0x7f51740051c0 2024-04-26T20:57:34.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.402+0000 7f5192200640 1 -- 172.21.15.28:0/479497202 reap_dead start 2024-04-26T20:57:34.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.402+0000 7f5192200640 1 --2- 172.21.15.28:0/479497202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51740051c0 0x7f517400b9c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f518400a770 tx=0x7f5174005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874f2a1c1f70d66e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:34.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.402+0000 7f518be00640 1 -- 172.21.15.28:0/479497202 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518400a390 con 0x7f51740051c0 2024-04-26T20:57:34.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.402+0000 7f518be00640 1 -- 172.21.15.28:0/479497202 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f518402f070 con 0x7f51740051c0 2024-04-26T20:57:34.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.402+0000 7f518be00640 1 -- 172.21.15.28:0/479497202 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518402bbe0 con 0x7f51740051c0 2024-04-26T20:57:34.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.403+0000 7f5192c00640 1 -- 172.21.15.28:0/479497202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51740051c0 msgr2=0x7f517400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:34.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.403+0000 7f5192c00640 1 --2- 172.21.15.28:0/479497202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51740051c0 0x7f517400b9c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f518400a770 tx=0x7f5174005830 comp rx=0 tx=0).stop 2024-04-26T20:57:34.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.403+0000 7f5192c00640 1 -- 172.21.15.28:0/479497202 shutdown_connections 2024-04-26T20:57:34.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.403+0000 7f5192c00640 1 --2- 172.21.15.28:0/479497202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51740051c0 0x7f517400b9c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.403+0000 7f5192c00640 1 -- 172.21.15.28:0/479497202 >> 172.21.15.28:0/479497202 conn(0x7f518c06c530 msgr2=0x7f518c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:34.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.403+0000 7f5192c00640 1 -- 172.21.15.28:0/479497202 shutdown_connections 2024-04-26T20:57:34.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.403+0000 7f5192c00640 1 -- 172.21.15.28:0/479497202 wait complete. 2024-04-26T20:57:34.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.405+0000 7f5192c00640 1 Processor -- start 2024-04-26T20:57:34.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.405+0000 7f5192c00640 1 -- start start 2024-04-26T20:57:34.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.406+0000 7f5192c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51740051c0 0x7f518c0c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:34.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.406+0000 7f5192c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f518c15bc00 0x7f518c0b2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:34.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.406+0000 7f5192c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f518c15c080 0x7f518c0b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:34.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.406+0000 7f5192c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f518c0cfe70 con 0x7f518c15c080 2024-04-26T20:57:34.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.406+0000 7f5192c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f518c0cfb90 con 0x7f51740051c0 2024-04-26T20:57:34.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.406+0000 7f5192c00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f518c146840 con 0x7f518c15bc00 2024-04-26T20:57:34.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.406+0000 7f5192200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f518c15c080 0x7f518c0b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:34.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.406+0000 7f5192200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f518c15c080 0x7f518c0b2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54820/0 (socket says 172.21.15.28:54820) 2024-04-26T20:57:34.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.406+0000 7f5192200640 1 -- 172.21.15.28:0/590976759 learned_addr learned my addr 172.21.15.28:0/590976759 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:34.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.406+0000 7f5190e00640 1 --2- 172.21.15.28:0/590976759 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f518c15bc00 0x7f518c0b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:34.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.407+0000 7f5191800640 1 --2- 172.21.15.28:0/590976759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51740051c0 0x7f518c0c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:34.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.407+0000 7f5192200640 1 -- 172.21.15.28:0/590976759 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f518c15bc00 msgr2=0x7f518c0b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:34.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.407+0000 7f5192200640 1 --2- 172.21.15.28:0/590976759 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f518c15bc00 0x7f518c0b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.407+0000 7f5192200640 1 -- 172.21.15.28:0/590976759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51740051c0 msgr2=0x7f518c0c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:34.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.407+0000 7f5192200640 1 --2- 172.21.15.28:0/590976759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51740051c0 0x7f518c0c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.407+0000 7f5192200640 1 -- 172.21.15.28:0/590976759 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518c1f4f50 con 0x7f518c15c080 2024-04-26T20:57:34.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.407+0000 7f5191800640 1 --2- 172.21.15.28:0/590976759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51740051c0 0x7f518c0c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:34.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.407+0000 7f5192200640 1 --2- 172.21.15.28:0/590976759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f518c15c080 0x7f518c0b2c40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f518402b430 tx=0x7f518c070020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:34.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.407+0000 7f5189600640 1 -- 172.21.15.28:0/590976759 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518402b780 con 0x7f518c15c080 2024-04-26T20:57:34.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.407+0000 7f5189600640 1 -- 172.21.15.28:0/590976759 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f518402b920 con 0x7f518c15c080 2024-04-26T20:57:34.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.407+0000 7f5189600640 1 -- 172.21.15.28:0/590976759 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518402bcd0 con 0x7f518c15c080 2024-04-26T20:57:34.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.407+0000 7f5192c00640 1 -- 172.21.15.28:0/590976759 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518c1f5230 con 0x7f518c15c080 2024-04-26T20:57:34.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.408+0000 7f5192c00640 1 -- 172.21.15.28:0/590976759 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f518c1f57f0 con 0x7f518c15c080 2024-04-26T20:57:34.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.409+0000 7f5192c00640 1 -- 172.21.15.28:0/590976759 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f518c06a2c0 con 0x7f518c15c080 2024-04-26T20:57:34.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.410+0000 7f5189600640 1 -- 172.21.15.28:0/590976759 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5184034470 con 0x7f518c15c080 2024-04-26T20:57:34.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.410+0000 7f5189600640 1 --2- 172.21.15.28:0/590976759 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f516c06c940 0x7f516c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:34.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.411+0000 7f5189600640 1 -- 172.21.15.28:0/590976759 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f51840ac160 con 0x7f518c15c080 2024-04-26T20:57:34.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.411+0000 7f5191800640 1 --2- 172.21.15.28:0/590976759 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f516c06c940 0x7f516c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:34.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.412+0000 7f5191800640 1 --2- 172.21.15.28:0/590976759 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f516c06c940 0x7f516c06ee00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5180009970 tx=0x7f5180008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:34.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.415+0000 7f5189600640 1 -- 172.21.15.28:0/590976759 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f518407a570 con 0x7f518c15c080 2024-04-26T20:57:34.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.620+0000 7f5192c00640 1 -- 172.21.15.28:0/590976759 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f518c0d07a0 con 0x7f518c15c080 2024-04-26T20:57:34.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.621+0000 7f5189600640 1 -- 172.21.15.28:0/590976759 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f5184079e70 con 0x7f518c15c080 2024-04-26T20:57:34.622 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-04-26T20:57:34.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.624+0000 7f5192c00640 1 -- 172.21.15.28:0/590976759 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f516c06c940 msgr2=0x7f516c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:34.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.624+0000 7f5192c00640 1 --2- 172.21.15.28:0/590976759 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f516c06c940 0x7f516c06ee00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5180009970 tx=0x7f5180008040 comp rx=0 tx=0).stop 2024-04-26T20:57:34.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.624+0000 7f5192c00640 1 -- 172.21.15.28:0/590976759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f518c15c080 msgr2=0x7f518c0b2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:34.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.624+0000 7f5192c00640 1 --2- 172.21.15.28:0/590976759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f518c15c080 0x7f518c0b2c40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f518402b430 tx=0x7f518c070020 comp rx=0 tx=0).stop 2024-04-26T20:57:34.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.624+0000 7f5192c00640 1 -- 172.21.15.28:0/590976759 shutdown_connections 2024-04-26T20:57:34.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.624+0000 7f5192c00640 1 --2- 172.21.15.28:0/590976759 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f516c06c940 0x7f516c06ee00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.624+0000 7f5192c00640 1 --2- 172.21.15.28:0/590976759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f518c15c080 0x7f518c0b2c40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.624+0000 7f5192c00640 1 --2- 172.21.15.28:0/590976759 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f518c15bc00 0x7f518c0b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.624+0000 7f5192c00640 1 --2- 172.21.15.28:0/590976759 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f51740051c0 0x7f518c0c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.624+0000 7f5192c00640 1 -- 172.21.15.28:0/590976759 >> 172.21.15.28:0/590976759 conn(0x7f518c06c530 msgr2=0x7f518c06e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:34.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.624+0000 7f5192c00640 1 -- 172.21.15.28:0/590976759 shutdown_connections 2024-04-26T20:57:34.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.624+0000 7f5192c00640 1 -- 172.21.15.28:0/590976759 wait complete. 2024-04-26T20:57:34.644 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:57:34.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.855+0000 7f8a6ae00640 1 Processor -- start 2024-04-26T20:57:34.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.856+0000 7f8a6ae00640 1 -- start start 2024-04-26T20:57:34.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.857+0000 7f8a6ae00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f8a6415f1b0 0x7f8a640d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:34.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.857+0000 7f8a6ae00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8a640cf700 0x7f8a640d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:34.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.857+0000 7f8a6ae00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a640d0710 con 0x7f8a640d5370 2024-04-26T20:57:34.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.857+0000 7f8a6ae00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f8a640d57e0 con 0x7f8a640cf700 2024-04-26T20:57:34.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.857+0000 7f8a6ae00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f8a640cff00 con 0x7f8a6415f1b0 2024-04-26T20:57:34.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.858+0000 7f8a69000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8a640cf700 0x7f8a640d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:34.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.858+0000 7f8a69000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8a640cf700 0x7f8a640d4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54826/0 (socket says 172.21.15.28:54826) 2024-04-26T20:57:34.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.858+0000 7f8a69000640 1 -- 172.21.15.28:0/614311173 learned_addr learned my addr 172.21.15.28:0/614311173 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:34.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.858+0000 7f8a69a00640 1 --2- 172.21.15.28:0/614311173 >> v2:172.21.15.103:3300/0 conn(0x7f8a6415f1b0 0x7f8a640d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:34.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.858+0000 7f8a69000640 1 -- 172.21.15.28:0/614311173 >> v2:172.21.15.103:3300/0 conn(0x7f8a6415f1b0 msgr2=0x7f8a640d4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:34.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.858+0000 7f8a69000640 1 --2- 172.21.15.28:0/614311173 >> v2:172.21.15.103:3300/0 conn(0x7f8a6415f1b0 0x7f8a640d4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.858+0000 7f8a69000640 1 -- 172.21.15.28:0/614311173 >> v1:172.21.15.103:6789/0 conn(0x7f8a640d5370 legacy=0x7f8a640b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:34.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.858+0000 7f8a69000640 1 -- 172.21.15.28:0/614311173 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a640d0150 con 0x7f8a640cf700 2024-04-26T20:57:34.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.859+0000 7f8a69000640 1 --2- 172.21.15.28:0/614311173 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a640cf700 0x7f8a640d4e30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8a640d0080 tx=0x7f8a5002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9472eb14b78616e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:34.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.860+0000 7f8a63e00640 1 -- 172.21.15.28:0/614311173 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50037070 con 0x7f8a640cf700 2024-04-26T20:57:34.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.860+0000 7f8a63e00640 1 -- 172.21.15.28:0/614311173 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a50032440 con 0x7f8a640cf700 2024-04-26T20:57:34.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.860+0000 7f8a63e00640 1 -- 172.21.15.28:0/614311173 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50032810 con 0x7f8a640cf700 2024-04-26T20:57:34.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.860+0000 7f8a6ae00640 1 -- 172.21.15.28:0/614311173 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a640cf700 msgr2=0x7f8a640d4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:34.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.860+0000 7f8a6ae00640 1 --2- 172.21.15.28:0/614311173 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a640cf700 0x7f8a640d4e30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8a640d0080 tx=0x7f8a5002a2c0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.861+0000 7f8a6ae00640 1 -- 172.21.15.28:0/614311173 shutdown_connections 2024-04-26T20:57:34.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.861+0000 7f8a6ae00640 1 --2- 172.21.15.28:0/614311173 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a640cf700 0x7f8a640d4e30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.861+0000 7f8a6ae00640 1 --2- 172.21.15.28:0/614311173 >> v2:172.21.15.103:3300/0 conn(0x7f8a6415f1b0 0x7f8a640d4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.861+0000 7f8a6ae00640 1 -- 172.21.15.28:0/614311173 >> 172.21.15.28:0/614311173 conn(0x7f8a6406c770 msgr2=0x7f8a6406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:34.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.861+0000 7f8a6ae00640 1 -- 172.21.15.28:0/614311173 shutdown_connections 2024-04-26T20:57:34.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.861+0000 7f8a6ae00640 1 -- 172.21.15.28:0/614311173 wait complete. 2024-04-26T20:57:34.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.862+0000 7f8a6ae00640 1 Processor -- start 2024-04-26T20:57:34.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.863+0000 7f8a6ae00640 1 -- start start 2024-04-26T20:57:34.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.864+0000 7f8a6ae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a640cf700 0x7f8a6415c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:34.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.864+0000 7f8a6ae00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8a640d5370 0x7f8a641f82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:34.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.864+0000 7f8a6ae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a6415f1b0 0x7f8a640c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:34.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.864+0000 7f8a6ae00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f8a640d3f00 con 0x7f8a6415f1b0 2024-04-26T20:57:34.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.864+0000 7f8a6ae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8a640d4270 con 0x7f8a640cf700 2024-04-26T20:57:34.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.864+0000 7f8a6ae00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f8a6415f590 con 0x7f8a640d5370 2024-04-26T20:57:34.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.864+0000 7f8a6a400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a6415f1b0 0x7f8a640c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:34.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.864+0000 7f8a6a400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a6415f1b0 0x7f8a640c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54840/0 (socket says 172.21.15.28:54840) 2024-04-26T20:57:34.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.864+0000 7f8a6a400640 1 -- 172.21.15.28:0/577763013 learned_addr learned my addr 172.21.15.28:0/577763013 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:34.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.864+0000 7f8a69a00640 1 --2- 172.21.15.28:0/577763013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a640cf700 0x7f8a6415c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:34.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.864+0000 7f8a69000640 1 --2- 172.21.15.28:0/577763013 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8a640d5370 0x7f8a641f82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:34.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.865+0000 7f8a6a400640 1 -- 172.21.15.28:0/577763013 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8a640d5370 msgr2=0x7f8a641f82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:34.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.865+0000 7f8a6a400640 1 --2- 172.21.15.28:0/577763013 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8a640d5370 0x7f8a641f82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.865+0000 7f8a6a400640 1 -- 172.21.15.28:0/577763013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a640cf700 msgr2=0x7f8a6415c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:34.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.865+0000 7f8a6a400640 1 --2- 172.21.15.28:0/577763013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a640cf700 0x7f8a6415c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:34.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.865+0000 7f8a6a400640 1 -- 172.21.15.28:0/577763013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a641f8c10 con 0x7f8a6415f1b0 2024-04-26T20:57:34.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.865+0000 7f8a69000640 1 --2- 172.21.15.28:0/577763013 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8a640d5370 0x7f8a641f82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:34.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.865+0000 7f8a6a400640 1 --2- 172.21.15.28:0/577763013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a6415f1b0 0x7f8a640c9300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00c780 tx=0x7f8a5c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:34.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.865+0000 7f8a61600640 1 -- 172.21.15.28:0/577763013 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c004b60 con 0x7f8a6415f1b0 2024-04-26T20:57:34.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.865+0000 7f8a61600640 1 -- 172.21.15.28:0/577763013 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a5c004d00 con 0x7f8a6415f1b0 2024-04-26T20:57:34.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.865+0000 7f8a61600640 1 -- 172.21.15.28:0/577763013 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c00aa90 con 0x7f8a6415f1b0 2024-04-26T20:57:34.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.865+0000 7f8a6ae00640 1 -- 172.21.15.28:0/577763013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a641f8ef0 con 0x7f8a6415f1b0 2024-04-26T20:57:34.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.865+0000 7f8a6ae00640 1 -- 172.21.15.28:0/577763013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a640c8de0 con 0x7f8a6415f1b0 2024-04-26T20:57:34.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.867+0000 7f8a6ae00640 1 -- 172.21.15.28:0/577763013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a6406a360 con 0x7f8a6415f1b0 2024-04-26T20:57:34.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.868+0000 7f8a61600640 1 -- 172.21.15.28:0/577763013 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a5c00ac30 con 0x7f8a6415f1b0 2024-04-26T20:57:34.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.868+0000 7f8a61600640 1 --2- 172.21.15.28:0/577763013 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8a4406c8f0 0x7f8a4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:34.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.869+0000 7f8a61600640 1 -- 172.21.15.28:0/577763013 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8a5c08bd60 con 0x7f8a6415f1b0 2024-04-26T20:57:34.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.869+0000 7f8a69a00640 1 --2- 172.21.15.28:0/577763013 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8a4406c8f0 0x7f8a4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:34.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.870+0000 7f8a69a00640 1 --2- 172.21.15.28:0/577763013 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8a4406c8f0 0x7f8a4406edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8a58006fd0 tx=0x7f8a58008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:34.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:34.873+0000 7f8a61600640 1 -- 172.21.15.28:0/577763013 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8a5c020070 con 0x7f8a6415f1b0 2024-04-26T20:57:35.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.093+0000 7f8a6ae00640 1 -- 172.21.15.28:0/577763013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8a640b8b60 con 0x7f8a6415f1b0 2024-04-26T20:57:35.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.094+0000 7f8a61600640 1 -- 172.21.15.28:0/577763013 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f8a5c05a220 con 0x7f8a6415f1b0 2024-04-26T20:57:35.095 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-04-26T20:57:35.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.097+0000 7f8a6ae00640 1 -- 172.21.15.28:0/577763013 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8a4406c8f0 msgr2=0x7f8a4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:35.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.097+0000 7f8a6ae00640 1 --2- 172.21.15.28:0/577763013 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8a4406c8f0 0x7f8a4406edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8a58006fd0 tx=0x7f8a58008040 comp rx=0 tx=0).stop 2024-04-26T20:57:35.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.097+0000 7f8a6ae00640 1 -- 172.21.15.28:0/577763013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a6415f1b0 msgr2=0x7f8a640c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:35.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.097+0000 7f8a6ae00640 1 --2- 172.21.15.28:0/577763013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a6415f1b0 0x7f8a640c9300 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00c780 tx=0x7f8a5c00cc40 comp rx=0 tx=0).stop 2024-04-26T20:57:35.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.097+0000 7f8a6ae00640 1 -- 172.21.15.28:0/577763013 shutdown_connections 2024-04-26T20:57:35.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.097+0000 7f8a6ae00640 1 --2- 172.21.15.28:0/577763013 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f8a4406c8f0 0x7f8a4406edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:35.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.097+0000 7f8a6ae00640 1 --2- 172.21.15.28:0/577763013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a6415f1b0 0x7f8a640c9300 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:35.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.097+0000 7f8a6ae00640 1 --2- 172.21.15.28:0/577763013 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8a640d5370 0x7f8a641f82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:35.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.097+0000 7f8a6ae00640 1 --2- 172.21.15.28:0/577763013 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8a640cf700 0x7f8a6415c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:35.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.097+0000 7f8a6ae00640 1 -- 172.21.15.28:0/577763013 >> 172.21.15.28:0/577763013 conn(0x7f8a6406c770 msgr2=0x7f8a6415e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:35.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.098+0000 7f8a6ae00640 1 -- 172.21.15.28:0/577763013 shutdown_connections 2024-04-26T20:57:35.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.098+0000 7f8a6ae00640 1 -- 172.21.15.28:0/577763013 wait complete. 2024-04-26T20:57:35.118 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:57:35.118 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:57:35.169 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:57:35.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.383+0000 7ff999400640 1 Processor -- start 2024-04-26T20:57:35.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.383+0000 7ff999400640 1 -- start start 2024-04-26T20:57:35.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.385+0000 7ff999400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff99415f2c0 0x7ff9940d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:35.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.385+0000 7ff999400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff9940b70a0 0x7ff9940d3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:35.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.385+0000 7ff999400640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9940b8030 con 0x7ff9940d2160 2024-04-26T20:57:35.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.385+0000 7ff999400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff99415f6a0 con 0x7ff9940b70a0 2024-04-26T20:57:35.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.385+0000 7ff999400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff9940d2a10 con 0x7ff99415f2c0 2024-04-26T20:57:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.385+0000 7ff998a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff9940b70a0 0x7ff9940d3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.385+0000 7ff998a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff9940b70a0 0x7ff9940d3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54852/0 (socket says 172.21.15.28:54852) 2024-04-26T20:57:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.385+0000 7ff998a00640 1 -- 172.21.15.28:0/1751426138 learned_addr learned my addr 172.21.15.28:0/1751426138 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.386+0000 7ff993e00640 1 --2- 172.21.15.28:0/1751426138 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff99415f2c0 0x7ff9940d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.386+0000 7ff998a00640 1 -- 172.21.15.28:0/1751426138 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff99415f2c0 msgr2=0x7ff9940d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.386+0000 7ff998a00640 1 --2- 172.21.15.28:0/1751426138 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff99415f2c0 0x7ff9940d0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.386+0000 7ff998a00640 1 -- 172.21.15.28:0/1751426138 >> v1:172.21.15.103:6789/0 conn(0x7ff9940d2160 legacy=0x7ff9940b6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:35.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.386+0000 7ff998a00640 1 -- 172.21.15.28:0/1751426138 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9940d2c60 con 0x7ff9940b70a0 2024-04-26T20:57:35.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.386+0000 7ff993e00640 1 --2- 172.21.15.28:0/1751426138 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff99415f2c0 0x7ff9940d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:57:35.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.388+0000 7ff998a00640 1 --2- 172.21.15.28:0/1751426138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9940b70a0 0x7ff9940d3a20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009870 tx=0x7ff98c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578296951ecf266e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:35.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.388+0000 7ff992a00640 1 -- 172.21.15.28:0/1751426138 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c037070 con 0x7ff9940b70a0 2024-04-26T20:57:35.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.388+0000 7ff992a00640 1 -- 172.21.15.28:0/1751426138 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff98c032440 con 0x7ff9940b70a0 2024-04-26T20:57:35.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.388+0000 7ff992a00640 1 -- 172.21.15.28:0/1751426138 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c032810 con 0x7ff9940b70a0 2024-04-26T20:57:35.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.389+0000 7ff999400640 1 -- 172.21.15.28:0/1751426138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9940b70a0 msgr2=0x7ff9940d3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:35.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.389+0000 7ff999400640 1 --2- 172.21.15.28:0/1751426138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9940b70a0 0x7ff9940d3a20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff98c009870 tx=0x7ff98c02a250 comp rx=0 tx=0).stop 2024-04-26T20:57:35.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.389+0000 7ff999400640 1 -- 172.21.15.28:0/1751426138 shutdown_connections 2024-04-26T20:57:35.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.389+0000 7ff999400640 1 --2- 172.21.15.28:0/1751426138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9940b70a0 0x7ff9940d3a20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:35.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.389+0000 7ff999400640 1 --2- 172.21.15.28:0/1751426138 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff99415f2c0 0x7ff9940d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:35.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.389+0000 7ff999400640 1 -- 172.21.15.28:0/1751426138 >> 172.21.15.28:0/1751426138 conn(0x7ff994072d90 msgr2=0x7ff9940731a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:35.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.389+0000 7ff999400640 1 -- 172.21.15.28:0/1751426138 shutdown_connections 2024-04-26T20:57:35.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.389+0000 7ff999400640 1 -- 172.21.15.28:0/1751426138 wait complete. 2024-04-26T20:57:35.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.391+0000 7ff999400640 1 Processor -- start 2024-04-26T20:57:35.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.391+0000 7ff999400640 1 -- start start 2024-04-26T20:57:35.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.392+0000 7ff999400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9940b70a0 0x7ff994155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:35.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.392+0000 7ff999400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9940d2160 0x7ff994153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:35.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.392+0000 7ff999400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff99415f2c0 0x7ff9941619f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:35.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.392+0000 7ff999400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff9940d3220 con 0x7ff9940d2160 2024-04-26T20:57:35.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.392+0000 7ff999400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff9940d3590 con 0x7ff9940b70a0 2024-04-26T20:57:35.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.392+0000 7ff999400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff9940d25c0 con 0x7ff99415f2c0 2024-04-26T20:57:35.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.393+0000 7ff993400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9940d2160 0x7ff994153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:35.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.393+0000 7ff993400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9940d2160 0x7ff994153350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54864/0 (socket says 172.21.15.28:54864) 2024-04-26T20:57:35.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.393+0000 7ff993400640 1 -- 172.21.15.28:0/2131665683 learned_addr learned my addr 172.21.15.28:0/2131665683 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:35.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.393+0000 7ff993e00640 1 --2- 172.21.15.28:0/2131665683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9940b70a0 0x7ff994155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:35.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.393+0000 7ff998a00640 1 --2- 172.21.15.28:0/2131665683 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff99415f2c0 0x7ff9941619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:35.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.393+0000 7ff993400640 1 -- 172.21.15.28:0/2131665683 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff99415f2c0 msgr2=0x7ff9941619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:35.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.393+0000 7ff993400640 1 --2- 172.21.15.28:0/2131665683 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff99415f2c0 0x7ff9941619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:35.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.393+0000 7ff993400640 1 -- 172.21.15.28:0/2131665683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9940b70a0 msgr2=0x7ff994155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:35.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.393+0000 7ff993400640 1 --2- 172.21.15.28:0/2131665683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9940b70a0 0x7ff994155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:35.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.393+0000 7ff993400640 1 -- 172.21.15.28:0/2131665683 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff994154210 con 0x7ff9940d2160 2024-04-26T20:57:35.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.393+0000 7ff993e00640 1 --2- 172.21.15.28:0/2131665683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9940b70a0 0x7ff994155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:35.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.393+0000 7ff993400640 1 --2- 172.21.15.28:0/2131665683 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9940d2160 0x7ff994153350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00c400 tx=0x7ff97c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:35.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.394+0000 7ff98be00640 1 -- 172.21.15.28:0/2131665683 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c0052d0 con 0x7ff9940d2160 2024-04-26T20:57:35.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.394+0000 7ff98be00640 1 -- 172.21.15.28:0/2131665683 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff97c005490 con 0x7ff9940d2160 2024-04-26T20:57:35.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.394+0000 7ff98be00640 1 -- 172.21.15.28:0/2131665683 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff97c007260 con 0x7ff9940d2160 2024-04-26T20:57:35.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.394+0000 7ff999400640 1 -- 172.21.15.28:0/2131665683 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9941544f0 con 0x7ff9940d2160 2024-04-26T20:57:35.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.394+0000 7ff999400640 1 -- 172.21.15.28:0/2131665683 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff994154ab0 con 0x7ff9940d2160 2024-04-26T20:57:35.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.395+0000 7ff999400640 1 -- 172.21.15.28:0/2131665683 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff99406a360 con 0x7ff9940d2160 2024-04-26T20:57:35.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.396+0000 7ff98be00640 1 -- 172.21.15.28:0/2131665683 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff97c007400 con 0x7ff9940d2160 2024-04-26T20:57:35.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.397+0000 7ff98be00640 1 --2- 172.21.15.28:0/2131665683 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff97006c940 0x7ff97006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:35.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.397+0000 7ff98be00640 1 -- 172.21.15.28:0/2131665683 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff97c08b030 con 0x7ff9940d2160 2024-04-26T20:57:35.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.397+0000 7ff993e00640 1 --2- 172.21.15.28:0/2131665683 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff97006c940 0x7ff97006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:35.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.398+0000 7ff993e00640 1 --2- 172.21.15.28:0/2131665683 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff97006c940 0x7ff97006ee00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff984009540 tx=0x7ff98400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:35.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.402+0000 7ff98be00640 1 -- 172.21.15.28:0/2131665683 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff97c0594d0 con 0x7ff9940d2160 2024-04-26T20:57:35.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.609+0000 7ff999400640 1 -- 172.21.15.28:0/2131665683 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9940b7840 con 0x7ff9940d2160 2024-04-26T20:57:35.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.612+0000 7ff98be00640 1 -- 172.21.15.28:0/2131665683 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16581 (secure 0 0 0) 0x7ff97c058dd0 con 0x7ff9940d2160 2024-04-26T20:57:35.614 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:35.614 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":21,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:56:52.699663+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:57:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.616+0000 7ff999400640 1 -- 172.21.15.28:0/2131665683 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff97006c940 msgr2=0x7ff97006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.616+0000 7ff999400640 1 --2- 172.21.15.28:0/2131665683 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff97006c940 0x7ff97006ee00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff984009540 tx=0x7ff98400b040 comp rx=0 tx=0).stop 2024-04-26T20:57:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.616+0000 7ff999400640 1 -- 172.21.15.28:0/2131665683 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9940d2160 msgr2=0x7ff994153350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.616+0000 7ff999400640 1 --2- 172.21.15.28:0/2131665683 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9940d2160 0x7ff994153350 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00c400 tx=0x7ff97c00c8c0 comp rx=0 tx=0).stop 2024-04-26T20:57:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.616+0000 7ff999400640 1 -- 172.21.15.28:0/2131665683 shutdown_connections 2024-04-26T20:57:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.616+0000 7ff999400640 1 --2- 172.21.15.28:0/2131665683 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff97006c940 0x7ff97006ee00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.616+0000 7ff999400640 1 --2- 172.21.15.28:0/2131665683 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff99415f2c0 0x7ff9941619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.616+0000 7ff999400640 1 --2- 172.21.15.28:0/2131665683 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9940d2160 0x7ff994153350 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.616+0000 7ff999400640 1 --2- 172.21.15.28:0/2131665683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9940b70a0 0x7ff994155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.616+0000 7ff999400640 1 -- 172.21.15.28:0/2131665683 >> 172.21.15.28:0/2131665683 conn(0x7ff994072d90 msgr2=0x7ff9940caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.616+0000 7ff999400640 1 -- 172.21.15.28:0/2131665683 shutdown_connections 2024-04-26T20:57:35.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.616+0000 7ff999400640 1 -- 172.21.15.28:0/2131665683 wait complete. 2024-04-26T20:57:35.636 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:56:48.688498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:56:50.117541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:57:35.636 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:57:35.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.853+0000 7fad96c00640 1 Processor -- start 2024-04-26T20:57:35.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.853+0000 7fad96c00640 1 -- start start 2024-04-26T20:57:35.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.854+0000 7fad96c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fad900d1de0 0x7fad900d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:35.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.854+0000 7fad96c00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fad900bac00 0x7fad900b8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:35.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.854+0000 7fad96c00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad900d2700 con 0x7fad9015f360 2024-04-26T20:57:35.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.854+0000 7fad96c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fad900ba350 con 0x7fad900d1de0 2024-04-26T20:57:35.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.854+0000 7fad96c00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fad900b9ab0 con 0x7fad900bac00 2024-04-26T20:57:35.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.855+0000 7fad95800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fad900d1de0 0x7fad900d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:35.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.855+0000 7fad95800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fad900d1de0 0x7fad900d21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54880/0 (socket says 172.21.15.28:54880) 2024-04-26T20:57:35.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.855+0000 7fad95800640 1 -- 172.21.15.28:0/1241939014 learned_addr learned my addr 172.21.15.28:0/1241939014 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:35.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.855+0000 7fad96200640 1 --2- 172.21.15.28:0/1241939014 >> v2:172.21.15.103:3300/0 conn(0x7fad900bac00 0x7fad900b8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:35.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.855+0000 7fad95800640 1 -- 172.21.15.28:0/1241939014 >> v2:172.21.15.103:3300/0 conn(0x7fad900bac00 msgr2=0x7fad900b8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:35.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.855+0000 7fad95800640 1 --2- 172.21.15.28:0/1241939014 >> v2:172.21.15.103:3300/0 conn(0x7fad900bac00 0x7fad900b8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:35.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.855+0000 7fad95800640 1 -- 172.21.15.28:0/1241939014 >> v1:172.21.15.103:6789/0 conn(0x7fad9015f360 legacy=0x7fad900d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:35.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.855+0000 7fad95800640 1 -- 172.21.15.28:0/1241939014 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad900b9d90 con 0x7fad900d1de0 2024-04-26T20:57:35.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.856+0000 7fad95800640 1 --2- 172.21.15.28:0/1241939014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad900d1de0 0x7fad900d21c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fad9008e350 tx=0x7fad8002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5931e758d1d0ccd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:35.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.857+0000 7fad8fe00640 1 -- 172.21.15.28:0/1241939014 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80037070 con 0x7fad900d1de0 2024-04-26T20:57:35.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.857+0000 7fad8fe00640 1 -- 172.21.15.28:0/1241939014 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad80032440 con 0x7fad900d1de0 2024-04-26T20:57:35.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.857+0000 7fad8fe00640 1 -- 172.21.15.28:0/1241939014 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad80032810 con 0x7fad900d1de0 2024-04-26T20:57:35.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.857+0000 7fad96c00640 1 -- 172.21.15.28:0/1241939014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad900d1de0 msgr2=0x7fad900d21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:35.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.857+0000 7fad96c00640 1 --2- 172.21.15.28:0/1241939014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad900d1de0 0x7fad900d21c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fad9008e350 tx=0x7fad8002a230 comp rx=0 tx=0).stop 2024-04-26T20:57:35.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.858+0000 7fad96c00640 1 -- 172.21.15.28:0/1241939014 shutdown_connections 2024-04-26T20:57:35.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.858+0000 7fad96c00640 1 --2- 172.21.15.28:0/1241939014 >> v2:172.21.15.103:3300/0 conn(0x7fad900bac00 0x7fad900b8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:35.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.858+0000 7fad96c00640 1 --2- 172.21.15.28:0/1241939014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad900d1de0 0x7fad900d21c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:35.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.858+0000 7fad96c00640 1 -- 172.21.15.28:0/1241939014 >> 172.21.15.28:0/1241939014 conn(0x7fad9006c770 msgr2=0x7fad9006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:35.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.858+0000 7fad96c00640 1 -- 172.21.15.28:0/1241939014 shutdown_connections 2024-04-26T20:57:35.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.858+0000 7fad96c00640 1 -- 172.21.15.28:0/1241939014 wait complete. 2024-04-26T20:57:35.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.859+0000 7fad96c00640 1 Processor -- start 2024-04-26T20:57:35.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.860+0000 7fad96c00640 1 -- start start 2024-04-26T20:57:35.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.860+0000 7fad96c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad900bac00 0x7fad90159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:35.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.861+0000 7fad96c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fad900d1de0 0x7fad900ce970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:35.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.861+0000 7fad96c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad9015f360 0x7fad90161d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:35.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.861+0000 7fad96c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fad900b9510 con 0x7fad9015f360 2024-04-26T20:57:35.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.861+0000 7fad96c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fad900d2f70 con 0x7fad900bac00 2024-04-26T20:57:35.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.861+0000 7fad96c00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fad900b9690 con 0x7fad900d1de0 2024-04-26T20:57:35.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.861+0000 7fad96200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad9015f360 0x7fad90161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:35.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.861+0000 7fad96200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad9015f360 0x7fad90161d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54890/0 (socket says 172.21.15.28:54890) 2024-04-26T20:57:35.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.861+0000 7fad96200640 1 -- 172.21.15.28:0/2285881583 learned_addr learned my addr 172.21.15.28:0/2285881583 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:35.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.861+0000 7fad94e00640 1 --2- 172.21.15.28:0/2285881583 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fad900d1de0 0x7fad900ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:35.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.861+0000 7fad95800640 1 --2- 172.21.15.28:0/2285881583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad900bac00 0x7fad90159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:35.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.861+0000 7fad96200640 1 -- 172.21.15.28:0/2285881583 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fad900d1de0 msgr2=0x7fad900ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:35.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.861+0000 7fad96200640 1 --2- 172.21.15.28:0/2285881583 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fad900d1de0 0x7fad900ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:35.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.861+0000 7fad96200640 1 -- 172.21.15.28:0/2285881583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad900bac00 msgr2=0x7fad90159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:35.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.861+0000 7fad96200640 1 --2- 172.21.15.28:0/2285881583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad900bac00 0x7fad90159500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:35.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.861+0000 7fad96200640 1 -- 172.21.15.28:0/2285881583 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad901590d0 con 0x7fad9015f360 2024-04-26T20:57:35.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.862+0000 7fad96200640 1 --2- 172.21.15.28:0/2285881583 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad9015f360 0x7fad90161d50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fad88009fd0 tx=0x7fad8800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:35.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.862+0000 7fad8d600640 1 -- 172.21.15.28:0/2285881583 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad88014420 con 0x7fad9015f360 2024-04-26T20:57:35.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.862+0000 7fad8d600640 1 -- 172.21.15.28:0/2285881583 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad88019070 con 0x7fad9015f360 2024-04-26T20:57:35.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.862+0000 7fad96c00640 1 -- 172.21.15.28:0/2285881583 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad90158830 con 0x7fad9015f360 2024-04-26T20:57:35.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.862+0000 7fad8d600640 1 -- 172.21.15.28:0/2285881583 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad88014800 con 0x7fad9015f360 2024-04-26T20:57:35.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.862+0000 7fad96c00640 1 -- 172.21.15.28:0/2285881583 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad90158df0 con 0x7fad9015f360 2024-04-26T20:57:35.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.863+0000 7fad96c00640 1 -- 172.21.15.28:0/2285881583 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad9006a250 con 0x7fad9015f360 2024-04-26T20:57:35.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.864+0000 7fad8d600640 1 -- 172.21.15.28:0/2285881583 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad880149a0 con 0x7fad9015f360 2024-04-26T20:57:35.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.865+0000 7fad8d600640 1 --2- 172.21.15.28:0/2285881583 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fad7006c8f0 0x7fad7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:35.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.865+0000 7fad8d600640 1 -- 172.21.15.28:0/2285881583 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fad8808c460 con 0x7fad9015f360 2024-04-26T20:57:35.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.866+0000 7fad95800640 1 --2- 172.21.15.28:0/2285881583 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fad7006c8f0 0x7fad7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:35.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.867+0000 7fad95800640 1 --2- 172.21.15.28:0/2285881583 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fad7006c8f0 0x7fad7006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fad80034000 tx=0x7fad8002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:35.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:35.870+0000 7fad8d600640 1 -- 172.21.15.28:0/2285881583 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fad8805a7f0 con 0x7fad9015f360 2024-04-26T20:57:36.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.078+0000 7fad96c00640 1 -- 172.21.15.28:0/2285881583 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fad9015e930 con 0x7fad9015f360 2024-04-26T20:57:36.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.079+0000 7fad8d600640 1 -- 172.21.15.28:0/2285881583 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fad8805a0f0 con 0x7fad9015f360 2024-04-26T20:57:36.080 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:57:36.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.082+0000 7fad96c00640 1 -- 172.21.15.28:0/2285881583 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fad7006c8f0 msgr2=0x7fad7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:36.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.082+0000 7fad96c00640 1 --2- 172.21.15.28:0/2285881583 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fad7006c8f0 0x7fad7006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fad80034000 tx=0x7fad8002e000 comp rx=0 tx=0).stop 2024-04-26T20:57:36.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.082+0000 7fad96c00640 1 -- 172.21.15.28:0/2285881583 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad9015f360 msgr2=0x7fad90161d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:36.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.082+0000 7fad96c00640 1 --2- 172.21.15.28:0/2285881583 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad9015f360 0x7fad90161d50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fad88009fd0 tx=0x7fad8800e4b0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.082+0000 7fad96c00640 1 -- 172.21.15.28:0/2285881583 shutdown_connections 2024-04-26T20:57:36.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.082+0000 7fad96c00640 1 --2- 172.21.15.28:0/2285881583 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fad7006c8f0 0x7fad7006edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.082+0000 7fad96c00640 1 --2- 172.21.15.28:0/2285881583 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad9015f360 0x7fad90161d50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.082+0000 7fad96c00640 1 --2- 172.21.15.28:0/2285881583 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fad900d1de0 0x7fad900ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.082+0000 7fad96c00640 1 --2- 172.21.15.28:0/2285881583 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fad900bac00 0x7fad90159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.082+0000 7fad96c00640 1 -- 172.21.15.28:0/2285881583 >> 172.21.15.28:0/2285881583 conn(0x7fad9006c770 msgr2=0x7fad900cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:36.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.082+0000 7fad96c00640 1 -- 172.21.15.28:0/2285881583 shutdown_connections 2024-04-26T20:57:36.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.082+0000 7fad96c00640 1 -- 172.21.15.28:0/2285881583 wait complete. 2024-04-26T20:57:36.105 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:57:36.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.320+0000 7fa3af200640 1 Processor -- start 2024-04-26T20:57:36.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.321+0000 7fa3af200640 1 -- start start 2024-04-26T20:57:36.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.322+0000 7fa3af200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa3b01465c0 0x7fa3b006e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:36.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.322+0000 7fa3af200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa3b00cf040 0x7fa3b00d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:36.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.322+0000 7fa3af200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3b00b5ba0 con 0x7fa3b00d1250 2024-04-26T20:57:36.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.322+0000 7fa3af200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa3b00d1660 con 0x7fa3b01465c0 2024-04-26T20:57:36.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.322+0000 7fa3af200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fa3b01469a0 con 0x7fa3b00cf040 2024-04-26T20:57:36.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.323+0000 7fa3ade00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa3b01465c0 0x7fa3b006e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:36.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.323+0000 7fa3ae800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa3b00d1250 0x7fa3b01609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58052/0 (socket says 172.21.15.28:58052) 2024-04-26T20:57:36.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.323+0000 7fa3ae800640 1 -- 172.21.15.28:0/3883650252 learned_addr learned my addr 172.21.15.28:0/3883650252 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:36.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.323+0000 7fa3ad400640 1 --2- 172.21.15.28:0/3883650252 >> v2:172.21.15.103:3300/0 conn(0x7fa3b00cf040 0x7fa3b00d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:36.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.323+0000 7fa3ade00640 1 -- 172.21.15.28:0/3883650252 >> v2:172.21.15.103:3300/0 conn(0x7fa3b00cf040 msgr2=0x7fa3b00d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:36.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.323+0000 7fa3ade00640 1 --2- 172.21.15.28:0/3883650252 >> v2:172.21.15.103:3300/0 conn(0x7fa3b00cf040 0x7fa3b00d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.323+0000 7fa3ade00640 1 -- 172.21.15.28:0/3883650252 >> v1:172.21.15.28:6789/0 conn(0x7fa3b00d1250 legacy=0x7fa3b01609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:36.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.323+0000 7fa3ade00640 1 -- 172.21.15.28:0/3883650252 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b00cf520 con 0x7fa3b01465c0 2024-04-26T20:57:36.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.325+0000 7fa3ade00640 1 --2- 172.21.15.28:0/3883650252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3b01465c0 0x7fa3b006e930 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0161a50 tx=0x7fa3a002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f6e4f05a8859f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:36.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.325+0000 7fa3aca00640 1 -- 172.21.15.28:0/3883650252 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0037070 con 0x7fa3b01465c0 2024-04-26T20:57:36.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.325+0000 7fa3aca00640 1 -- 172.21.15.28:0/3883650252 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a0032440 con 0x7fa3b01465c0 2024-04-26T20:57:36.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.325+0000 7fa3aca00640 1 -- 172.21.15.28:0/3883650252 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a002ae60 con 0x7fa3b01465c0 2024-04-26T20:57:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.325+0000 7fa3af200640 1 -- 172.21.15.28:0/3883650252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3b01465c0 msgr2=0x7fa3b006e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.325+0000 7fa3af200640 1 --2- 172.21.15.28:0/3883650252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3b01465c0 0x7fa3b006e930 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0161a50 tx=0x7fa3a002a110 comp rx=0 tx=0).stop 2024-04-26T20:57:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.326+0000 7fa3af200640 1 -- 172.21.15.28:0/3883650252 shutdown_connections 2024-04-26T20:57:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.326+0000 7fa3af200640 1 --2- 172.21.15.28:0/3883650252 >> v2:172.21.15.103:3300/0 conn(0x7fa3b00cf040 0x7fa3b00d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.326+0000 7fa3af200640 1 --2- 172.21.15.28:0/3883650252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3b01465c0 0x7fa3b006e930 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.326+0000 7fa3af200640 1 -- 172.21.15.28:0/3883650252 >> 172.21.15.28:0/3883650252 conn(0x7fa3b006c530 msgr2=0x7fa3b006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.326+0000 7fa3af200640 1 -- 172.21.15.28:0/3883650252 shutdown_connections 2024-04-26T20:57:36.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.326+0000 7fa3af200640 1 -- 172.21.15.28:0/3883650252 wait complete. 2024-04-26T20:57:36.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.327+0000 7fa3af200640 1 Processor -- start 2024-04-26T20:57:36.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.328+0000 7fa3af200640 1 -- start start 2024-04-26T20:57:36.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.328+0000 7fa3af200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3b00cf040 0x7fa3b0073490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:36.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.328+0000 7fa3af200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3b00d1250 0x7fa3b015e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:36.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.328+0000 7fa3af200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3b01465c0 0x7fa3b0160c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:36.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.328+0000 7fa3af200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b00cfdc0 con 0x7fa3b00d1250 2024-04-26T20:57:36.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.328+0000 7fa3af200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b00cfae0 con 0x7fa3b00cf040 2024-04-26T20:57:36.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.328+0000 7fa3af200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa3b00d00a0 con 0x7fa3b01465c0 2024-04-26T20:57:36.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.329+0000 7fa3ad400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3b00d1250 0x7fa3b015e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:36.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.329+0000 7fa3ad400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3b00d1250 0x7fa3b015e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54906/0 (socket says 172.21.15.28:54906) 2024-04-26T20:57:36.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.329+0000 7fa3ad400640 1 -- 172.21.15.28:0/1382170819 learned_addr learned my addr 172.21.15.28:0/1382170819 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:36.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.329+0000 7fa3ade00640 1 --2- 172.21.15.28:0/1382170819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3b00cf040 0x7fa3b0073490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:36.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.329+0000 7fa3ae800640 1 --2- 172.21.15.28:0/1382170819 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3b01465c0 0x7fa3b0160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:36.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.329+0000 7fa3ad400640 1 -- 172.21.15.28:0/1382170819 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3b01465c0 msgr2=0x7fa3b0160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:36.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.329+0000 7fa3ad400640 1 --2- 172.21.15.28:0/1382170819 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3b01465c0 0x7fa3b0160c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.329+0000 7fa3ad400640 1 -- 172.21.15.28:0/1382170819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3b00cf040 msgr2=0x7fa3b0073490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:36.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.329+0000 7fa3ad400640 1 --2- 172.21.15.28:0/1382170819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3b00cf040 0x7fa3b0073490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.329+0000 7fa3ad400640 1 -- 172.21.15.28:0/1382170819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b00723f0 con 0x7fa3b00d1250 2024-04-26T20:57:36.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.330+0000 7fa3ad400640 1 --2- 172.21.15.28:0/1382170819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3b00d1250 0x7fa3b015e1c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00d800 tx=0x7fa39c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:36.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.330+0000 7fa3a6000640 1 -- 172.21.15.28:0/1382170819 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c00b660 con 0x7fa3b00d1250 2024-04-26T20:57:36.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.330+0000 7fa3a6000640 1 -- 172.21.15.28:0/1382170819 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa39c019070 con 0x7fa3b00d1250 2024-04-26T20:57:36.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.330+0000 7fa3a6000640 1 -- 172.21.15.28:0/1382170819 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c00ba20 con 0x7fa3b00d1250 2024-04-26T20:57:36.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.330+0000 7fa3af200640 1 -- 172.21.15.28:0/1382170819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b00726d0 con 0x7fa3b00d1250 2024-04-26T20:57:36.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.330+0000 7fa3af200640 1 -- 172.21.15.28:0/1382170819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b0072c90 con 0x7fa3b00d1250 2024-04-26T20:57:36.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.332+0000 7fa3af200640 1 -- 172.21.15.28:0/1382170819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3b006a250 con 0x7fa3b00d1250 2024-04-26T20:57:36.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.332+0000 7fa3a6000640 1 -- 172.21.15.28:0/1382170819 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa39c00bbc0 con 0x7fa3b00d1250 2024-04-26T20:57:36.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.333+0000 7fa3a6000640 1 --2- 172.21.15.28:0/1382170819 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa39006c8f0 0x7fa39006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:36.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.333+0000 7fa3a6000640 1 -- 172.21.15.28:0/1382170819 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa39c08bf90 con 0x7fa3b00d1250 2024-04-26T20:57:36.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.334+0000 7fa3ade00640 1 --2- 172.21.15.28:0/1382170819 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa39006c8f0 0x7fa39006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:36.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.334+0000 7fa3ade00640 1 --2- 172.21.15.28:0/1382170819 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa39006c8f0 0x7fa39006edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0034000 tx=0x7fa3a002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:36.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.338+0000 7fa3a6000640 1 -- 172.21.15.28:0/1382170819 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa39c05a3c0 con 0x7fa3b00d1250 2024-04-26T20:57:36.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.546+0000 7fa3af200640 1 -- 172.21.15.28:0/1382170819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa3b006ea60 con 0x7fa3b00d1250 2024-04-26T20:57:36.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.546+0000 7fa3a6000640 1 -- 172.21.15.28:0/1382170819 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fa39c059cc0 con 0x7fa3b00d1250 2024-04-26T20:57:36.548 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-04-26T20:57:36.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.549+0000 7fa3af200640 1 -- 172.21.15.28:0/1382170819 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa39006c8f0 msgr2=0x7fa39006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:36.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.549+0000 7fa3af200640 1 --2- 172.21.15.28:0/1382170819 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa39006c8f0 0x7fa39006edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0034000 tx=0x7fa3a002e000 comp rx=0 tx=0).stop 2024-04-26T20:57:36.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.549+0000 7fa3af200640 1 -- 172.21.15.28:0/1382170819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3b00d1250 msgr2=0x7fa3b015e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:36.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.549+0000 7fa3af200640 1 --2- 172.21.15.28:0/1382170819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3b00d1250 0x7fa3b015e1c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00d800 tx=0x7fa39c00dcc0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.549+0000 7fa3af200640 1 -- 172.21.15.28:0/1382170819 shutdown_connections 2024-04-26T20:57:36.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.549+0000 7fa3af200640 1 --2- 172.21.15.28:0/1382170819 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa39006c8f0 0x7fa39006edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.549+0000 7fa3af200640 1 --2- 172.21.15.28:0/1382170819 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3b01465c0 0x7fa3b0160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.549+0000 7fa3af200640 1 --2- 172.21.15.28:0/1382170819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3b00d1250 0x7fa3b015e1c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.549+0000 7fa3af200640 1 --2- 172.21.15.28:0/1382170819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3b00cf040 0x7fa3b0073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.549+0000 7fa3af200640 1 -- 172.21.15.28:0/1382170819 >> 172.21.15.28:0/1382170819 conn(0x7fa3b006c530 msgr2=0x7fa3b006df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:36.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.549+0000 7fa3af200640 1 -- 172.21.15.28:0/1382170819 shutdown_connections 2024-04-26T20:57:36.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.550+0000 7fa3af200640 1 -- 172.21.15.28:0/1382170819 wait complete. 2024-04-26T20:57:36.571 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:57:36.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.786+0000 7f0a76000640 1 Processor -- start 2024-04-26T20:57:36.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.786+0000 7f0a76000640 1 -- start start 2024-04-26T20:57:36.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.788+0000 7f0a76000640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f0a70145900 0x7f0a70160500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:36.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.788+0000 7f0a76000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0a7006f9e0 0x7f0a70072030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:36.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.788+0000 7f0a76000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a700cf640 con 0x7f0a700cdea0 2024-04-26T20:57:36.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.788+0000 7f0a76000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f0a70145ce0 con 0x7f0a7006f9e0 2024-04-26T20:57:36.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.788+0000 7f0a76000640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f0a70160a40 con 0x7f0a70145900 2024-04-26T20:57:36.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.788+0000 7f0a6fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0a7006f9e0 0x7f0a70072030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:36.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.788+0000 7f0a6ea00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f0a700cdea0 0x7f0a7006f2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58066/0 (socket says 172.21.15.28:58066) 2024-04-26T20:57:36.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.788+0000 7f0a6ea00640 1 -- 172.21.15.28:0/2515790298 learned_addr learned my addr 172.21.15.28:0/2515790298 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:36.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.789+0000 7f0a6f400640 1 --2- 172.21.15.28:0/2515790298 >> v2:172.21.15.103:3300/0 conn(0x7f0a70145900 0x7f0a70160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:36.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.789+0000 7f0a6fe00640 1 -- 172.21.15.28:0/2515790298 >> v2:172.21.15.103:3300/0 conn(0x7f0a70145900 msgr2=0x7f0a70160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:36.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.789+0000 7f0a6fe00640 1 --2- 172.21.15.28:0/2515790298 >> v2:172.21.15.103:3300/0 conn(0x7f0a70145900 0x7f0a70160500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.789+0000 7f0a6fe00640 1 -- 172.21.15.28:0/2515790298 >> v1:172.21.15.28:6789/0 conn(0x7f0a700cdea0 legacy=0x7f0a7006f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:36.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.789+0000 7f0a6fe00640 1 -- 172.21.15.28:0/2515790298 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a700cf8c0 con 0x7f0a7006f9e0 2024-04-26T20:57:36.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.790+0000 7f0a6fe00640 1 --2- 172.21.15.28:0/2515790298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a7006f9e0 0x7f0a70072030 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0a70070020 tx=0x7f0a6802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af43116efb51980d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:36.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.790+0000 7f0a6e000640 1 -- 172.21.15.28:0/2515790298 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68037070 con 0x7f0a7006f9e0 2024-04-26T20:57:36.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.790+0000 7f0a6e000640 1 -- 172.21.15.28:0/2515790298 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a68032440 con 0x7f0a7006f9e0 2024-04-26T20:57:36.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.790+0000 7f0a6e000640 1 -- 172.21.15.28:0/2515790298 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a68032810 con 0x7f0a7006f9e0 2024-04-26T20:57:36.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.791+0000 7f0a76000640 1 -- 172.21.15.28:0/2515790298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a7006f9e0 msgr2=0x7f0a70072030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:36.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.791+0000 7f0a76000640 1 --2- 172.21.15.28:0/2515790298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a7006f9e0 0x7f0a70072030 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0a70070020 tx=0x7f0a6802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.791+0000 7f0a76000640 1 -- 172.21.15.28:0/2515790298 shutdown_connections 2024-04-26T20:57:36.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.791+0000 7f0a76000640 1 --2- 172.21.15.28:0/2515790298 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a7006f9e0 0x7f0a70072030 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.791+0000 7f0a76000640 1 --2- 172.21.15.28:0/2515790298 >> v2:172.21.15.103:3300/0 conn(0x7f0a70145900 0x7f0a70160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.791+0000 7f0a76000640 1 -- 172.21.15.28:0/2515790298 >> 172.21.15.28:0/2515790298 conn(0x7f0a700b2b60 msgr2=0x7f0a700b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:36.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.792+0000 7f0a76000640 1 -- 172.21.15.28:0/2515790298 shutdown_connections 2024-04-26T20:57:36.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.792+0000 7f0a76000640 1 -- 172.21.15.28:0/2515790298 wait complete. 2024-04-26T20:57:36.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.793+0000 7f0a76000640 1 Processor -- start 2024-04-26T20:57:36.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.793+0000 7f0a76000640 1 -- start start 2024-04-26T20:57:36.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.794+0000 7f0a76000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a7006f9e0 0x7f0a7015dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:36.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.794+0000 7f0a76000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a700cdea0 0x7f0a7015eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:36.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.794+0000 7f0a76000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a70145900 0x7f0a7015c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:36.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.794+0000 7f0a76000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0a700d0160 con 0x7f0a7006f9e0 2024-04-26T20:57:36.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.794+0000 7f0a76000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a700cfe80 con 0x7f0a70145900 2024-04-26T20:57:36.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.794+0000 7f0a76000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f0a700d0440 con 0x7f0a700cdea0 2024-04-26T20:57:36.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.795+0000 7f0a6f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a7006f9e0 0x7f0a7015dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:36.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.795+0000 7f0a6f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a7006f9e0 0x7f0a7015dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54920/0 (socket says 172.21.15.28:54920) 2024-04-26T20:57:36.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.795+0000 7f0a6f400640 1 -- 172.21.15.28:0/3019773042 learned_addr learned my addr 172.21.15.28:0/3019773042 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:36.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.795+0000 7f0a6ea00640 1 --2- 172.21.15.28:0/3019773042 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a700cdea0 0x7f0a7015eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:36.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.795+0000 7f0a6fe00640 1 --2- 172.21.15.28:0/3019773042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a70145900 0x7f0a7015c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:36.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.795+0000 7f0a6f400640 1 -- 172.21.15.28:0/3019773042 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a700cdea0 msgr2=0x7f0a7015eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:36.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.795+0000 7f0a6f400640 1 --2- 172.21.15.28:0/3019773042 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a700cdea0 0x7f0a7015eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.795+0000 7f0a6f400640 1 -- 172.21.15.28:0/3019773042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a70145900 msgr2=0x7f0a7015c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:36.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.795+0000 7f0a6f400640 1 --2- 172.21.15.28:0/3019773042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a70145900 0x7f0a7015c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:36.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.795+0000 7f0a6f400640 1 -- 172.21.15.28:0/3019773042 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a7015cd40 con 0x7f0a7006f9e0 2024-04-26T20:57:36.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.796+0000 7f0a6f400640 1 --2- 172.21.15.28:0/3019773042 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a7006f9e0 0x7f0a7015dde0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0a60007e70 tx=0x7f0a6000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:36.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.796+0000 7f0a67400640 1 -- 172.21.15.28:0/3019773042 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60014620 con 0x7f0a7006f9e0 2024-04-26T20:57:36.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.796+0000 7f0a67400640 1 -- 172.21.15.28:0/3019773042 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a60019070 con 0x7f0a7006f9e0 2024-04-26T20:57:36.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.796+0000 7f0a67400640 1 -- 172.21.15.28:0/3019773042 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a60014a00 con 0x7f0a7006f9e0 2024-04-26T20:57:36.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.796+0000 7f0a76000640 1 -- 172.21.15.28:0/3019773042 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a7015d020 con 0x7f0a7006f9e0 2024-04-26T20:57:36.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.796+0000 7f0a76000640 1 -- 172.21.15.28:0/3019773042 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a7015d5e0 con 0x7f0a7006f9e0 2024-04-26T20:57:36.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.797+0000 7f0a76000640 1 -- 172.21.15.28:0/3019773042 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a70076520 con 0x7f0a7006f9e0 2024-04-26T20:57:36.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.798+0000 7f0a67400640 1 -- 172.21.15.28:0/3019773042 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a60014ba0 con 0x7f0a7006f9e0 2024-04-26T20:57:36.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.799+0000 7f0a67400640 1 --2- 172.21.15.28:0/3019773042 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0a4c06c8f0 0x7f0a4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:36.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.799+0000 7f0a67400640 1 -- 172.21.15.28:0/3019773042 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0a6008c490 con 0x7f0a7006f9e0 2024-04-26T20:57:36.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.799+0000 7f0a6ea00640 1 --2- 172.21.15.28:0/3019773042 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0a4c06c8f0 0x7f0a4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:36.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.800+0000 7f0a6ea00640 1 --2- 172.21.15.28:0/3019773042 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0a4c06c8f0 0x7f0a4c06edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0a70073890 tx=0x7f0a58009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:36.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:36.804+0000 7f0a67400640 1 -- 172.21.15.28:0/3019773042 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0a60011260 con 0x7f0a7006f9e0 2024-04-26T20:57:37.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.014+0000 7f0a76000640 1 -- 172.21.15.28:0/3019773042 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0a70160820 con 0x7f0a7006f9e0 2024-04-26T20:57:37.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.015+0000 7f0a67400640 1 -- 172.21.15.28:0/3019773042 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f0a6005a8b0 con 0x7f0a7006f9e0 2024-04-26T20:57:37.016 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-04-26T20:57:37.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.018+0000 7f0a76000640 1 -- 172.21.15.28:0/3019773042 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0a4c06c8f0 msgr2=0x7f0a4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:37.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.018+0000 7f0a76000640 1 --2- 172.21.15.28:0/3019773042 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0a4c06c8f0 0x7f0a4c06edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0a70073890 tx=0x7f0a58009040 comp rx=0 tx=0).stop 2024-04-26T20:57:37.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.018+0000 7f0a76000640 1 -- 172.21.15.28:0/3019773042 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a7006f9e0 msgr2=0x7f0a7015dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:37.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.018+0000 7f0a76000640 1 --2- 172.21.15.28:0/3019773042 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a7006f9e0 0x7f0a7015dde0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0a60007e70 tx=0x7f0a6000e4b0 comp rx=0 tx=0).stop 2024-04-26T20:57:37.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.018+0000 7f0a76000640 1 -- 172.21.15.28:0/3019773042 shutdown_connections 2024-04-26T20:57:37.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.018+0000 7f0a76000640 1 --2- 172.21.15.28:0/3019773042 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0a4c06c8f0 0x7f0a4c06edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:37.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.018+0000 7f0a76000640 1 --2- 172.21.15.28:0/3019773042 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a70145900 0x7f0a7015c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:37.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.018+0000 7f0a76000640 1 --2- 172.21.15.28:0/3019773042 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a700cdea0 0x7f0a7015eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:37.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.018+0000 7f0a76000640 1 --2- 172.21.15.28:0/3019773042 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a7006f9e0 0x7f0a7015dde0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:37.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.018+0000 7f0a76000640 1 -- 172.21.15.28:0/3019773042 >> 172.21.15.28:0/3019773042 conn(0x7f0a700b2b60 msgr2=0x7f0a70072030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:37.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.018+0000 7f0a76000640 1 -- 172.21.15.28:0/3019773042 shutdown_connections 2024-04-26T20:57:37.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.018+0000 7f0a76000640 1 -- 172.21.15.28:0/3019773042 wait complete. 2024-04-26T20:57:37.039 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:57:37.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.245+0000 7f462e000640 1 Processor -- start 2024-04-26T20:57:37.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.245+0000 7f462e000640 1 -- start start 2024-04-26T20:57:37.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.246+0000 7f462e000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4628146980 0x7f46280b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:37.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.246+0000 7f462e000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46280d0a30 con 0x7f46280740e0 2024-04-26T20:57:37.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.246+0000 7f462e000640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f462815e230 con 0x7f462815dcc0 2024-04-26T20:57:37.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.246+0000 7f462e000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f462815e510 con 0x7f4628146980 2024-04-26T20:57:37.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.247+0000 7f4626a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f46280740e0 0x7f462815d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58080/0 (socket says 172.21.15.28:58080) 2024-04-26T20:57:37.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.247+0000 7f4626a00640 1 -- 172.21.15.28:0/4135275488 learned_addr learned my addr 172.21.15.28:0/4135275488 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:37.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.247+0000 7f4627400640 1 --2- 172.21.15.28:0/4135275488 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4628146980 0x7f46280b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:37.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.247+0000 7f4626000640 1 -- 172.21.15.28:0/4135275488 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4223493478 0 0) 0x7f4614001110 con 0x7f46280740e0 2024-04-26T20:57:37.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.247+0000 7f4626000640 1 -- 172.21.15.28:0/4135275488 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f460c003520 con 0x7f46280740e0 2024-04-26T20:57:37.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.248+0000 7f4626000640 1 -- 172.21.15.28:0/4135275488 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 18529591 0 0) 0x7f460c003520 con 0x7f46280740e0 2024-04-26T20:57:37.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.248+0000 7f4626000640 1 -- 172.21.15.28:0/4135275488 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4628146980 msgr2=0x7f46280b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:37.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.248+0000 7f4626000640 1 --2- 172.21.15.28:0/4135275488 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4628146980 0x7f46280b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:37.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.248+0000 7f4626000640 1 -- 172.21.15.28:0/4135275488 >> v1:172.21.15.103:6789/0 conn(0x7f462815dcc0 legacy=0x7f4628071ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:37.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.248+0000 7f4626000640 1 -- 172.21.15.28:0/4135275488 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4628073600 con 0x7f46280740e0 2024-04-26T20:57:37.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.248+0000 7f4626000640 1 -- 172.21.15.28:0/4135275488 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f4614001840 con 0x7f46280740e0 2024-04-26T20:57:37.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.248+0000 7f4626000640 1 -- 172.21.15.28:0/4135275488 >> v1:172.21.15.28:6789/0 conn(0x7f46280740e0 legacy=0x7f462815d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:37.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.248+0000 7f4626000640 1 --2- 172.21.15.28:0/4135275488 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f460c0039e0 0x7f460c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:37.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.248+0000 7f4626000640 1 --2- 172.21.15.28:0/4135275488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f460c004800 0x7f460c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:37.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.248+0000 7f4626000640 1 --2- 172.21.15.28:0/4135275488 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f460c0051c0 0x7f460c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:37.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.248+0000 7f4626000640 1 -- 172.21.15.28:0/4135275488 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f462815e510 con 0x7f460c0039e0 2024-04-26T20:57:37.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.248+0000 7f4626000640 1 -- 172.21.15.28:0/4135275488 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f460c00c060 con 0x7f460c004800 2024-04-26T20:57:37.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.248+0000 7f4626000640 1 -- 172.21.15.28:0/4135275488 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f460c00c260 con 0x7f460c0051c0 2024-04-26T20:57:37.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.248+0000 7f4627400640 1 --2- 172.21.15.28:0/4135275488 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f460c0039e0 0x7f460c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:37.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.248+0000 7f4627e00640 1 --2- 172.21.15.28:0/4135275488 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f460c0051c0 0x7f460c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:37.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.249+0000 7f4627400640 1 -- 172.21.15.28:0/4135275488 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f460c0051c0 msgr2=0x7f460c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:37.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.249+0000 7f4627400640 1 --2- 172.21.15.28:0/4135275488 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f460c0051c0 0x7f460c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:37.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.249+0000 7f4626a00640 1 --2- 172.21.15.28:0/4135275488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f460c004800 0x7f460c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:37.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.249+0000 7f4627400640 1 -- 172.21.15.28:0/4135275488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f460c004800 msgr2=0x7f460c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:37.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.249+0000 7f4627400640 1 --2- 172.21.15.28:0/4135275488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f460c004800 0x7f460c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:37.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.249+0000 7f4627400640 1 -- 172.21.15.28:0/4135275488 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460c003520 con 0x7f460c0039e0 2024-04-26T20:57:37.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.249+0000 7f4626a00640 1 --2- 172.21.15.28:0/4135275488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f460c004800 0x7f460c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:37.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.250+0000 7f4627400640 1 --2- 172.21.15.28:0/4135275488 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f460c0039e0 0x7f460c003e00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f46280b6dc0 tx=0x7f460c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9365d4923738cc96 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:37.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.250+0000 7f4626000640 1 -- 172.21.15.28:0/4135275488 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618037070 con 0x7f460c0039e0 2024-04-26T20:57:37.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.250+0000 7f4626000640 1 -- 172.21.15.28:0/4135275488 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f461802acd0 con 0x7f460c0039e0 2024-04-26T20:57:37.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.250+0000 7f4626000640 1 -- 172.21.15.28:0/4135275488 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618009630 con 0x7f460c0039e0 2024-04-26T20:57:37.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.251+0000 7f462e000640 1 -- 172.21.15.28:0/4135275488 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f460c0039e0 msgr2=0x7f460c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:37.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.251+0000 7f462e000640 1 --2- 172.21.15.28:0/4135275488 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f460c0039e0 0x7f460c003e00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f46280b6dc0 tx=0x7f460c005830 comp rx=0 tx=0).stop 2024-04-26T20:57:37.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.251+0000 7f4627e00640 1 -- 172.21.15.28:0/4135275488 reap_dead start 2024-04-26T20:57:37.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.251+0000 7f462e000640 1 -- 172.21.15.28:0/4135275488 shutdown_connections 2024-04-26T20:57:37.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.251+0000 7f462e000640 1 -- 172.21.15.28:0/4135275488 >> 172.21.15.28:0/4135275488 conn(0x7f462806e220 msgr2=0x7f462806e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:37.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.251+0000 7f462e000640 1 -- 172.21.15.28:0/4135275488 shutdown_connections 2024-04-26T20:57:37.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.251+0000 7f462e000640 1 -- 172.21.15.28:0/4135275488 wait complete. 2024-04-26T20:57:37.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.253+0000 7f462e000640 1 Processor -- start 2024-04-26T20:57:37.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.253+0000 7f462e000640 1 -- start start 2024-04-26T20:57:37.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.254+0000 7f462e000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f460c0039e0 0x7f4628159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:37.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.254+0000 7f462e000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46281397b0 0x7f46280b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:37.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.254+0000 7f462e000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46280b6440 0x7f462815baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:37.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.254+0000 7f462e000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4628072700 con 0x7f46281397b0 2024-04-26T20:57:37.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.254+0000 7f462e000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4628073bc0 con 0x7f46280b6440 2024-04-26T20:57:37.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.254+0000 7f462e000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f4628072880 con 0x7f460c0039e0 2024-04-26T20:57:37.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.254+0000 7f4626a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46281397b0 0x7f46280b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:37.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.254+0000 7f4626a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46281397b0 0x7f46280b5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54950/0 (socket says 172.21.15.28:54950) 2024-04-26T20:57:37.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.254+0000 7f4626a00640 1 -- 172.21.15.28:0/2481941638 learned_addr learned my addr 172.21.15.28:0/2481941638 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:37.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.255+0000 7f4627400640 1 --2- 172.21.15.28:0/2481941638 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f460c0039e0 0x7f4628159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:37.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.255+0000 7f4626a00640 1 -- 172.21.15.28:0/2481941638 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f460c0039e0 msgr2=0x7f4628159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:37.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.255+0000 7f4626a00640 1 --2- 172.21.15.28:0/2481941638 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f460c0039e0 0x7f4628159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:37.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.255+0000 7f4626a00640 1 -- 172.21.15.28:0/2481941638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46280b6440 msgr2=0x7f462815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:37.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.255+0000 7f4627e00640 1 --2- 172.21.15.28:0/2481941638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46280b6440 0x7f462815baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:37.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.255+0000 7f4626a00640 1 --2- 172.21.15.28:0/2481941638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46280b6440 0x7f462815baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:37.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.255+0000 7f4626a00640 1 -- 172.21.15.28:0/2481941638 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4628139c30 con 0x7f46281397b0 2024-04-26T20:57:37.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.255+0000 7f4627e00640 1 --2- 172.21.15.28:0/2481941638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46280b6440 0x7f462815baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:37.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.255+0000 7f4626a00640 1 --2- 172.21.15.28:0/2481941638 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46281397b0 0x7f46280b5f00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f461400d8e0 tx=0x7f461400dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:37.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.255+0000 7f461f400640 1 -- 172.21.15.28:0/2481941638 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4614019070 con 0x7f46281397b0 2024-04-26T20:57:37.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.255+0000 7f461f400640 1 -- 172.21.15.28:0/2481941638 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46140074e0 con 0x7f46281397b0 2024-04-26T20:57:37.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.255+0000 7f461f400640 1 -- 172.21.15.28:0/2481941638 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46140049e0 con 0x7f46281397b0 2024-04-26T20:57:37.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.255+0000 7f462e000640 1 -- 172.21.15.28:0/2481941638 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46281f5920 con 0x7f46281397b0 2024-04-26T20:57:37.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.256+0000 7f462e000640 1 -- 172.21.15.28:0/2481941638 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46281f5ee0 con 0x7f46281397b0 2024-04-26T20:57:37.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.257+0000 7f462e000640 1 -- 172.21.15.28:0/2481941638 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f462806a400 con 0x7f46281397b0 2024-04-26T20:57:37.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.258+0000 7f461f400640 1 -- 172.21.15.28:0/2481941638 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f461401c070 con 0x7f46281397b0 2024-04-26T20:57:37.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.259+0000 7f461f400640 1 --2- 172.21.15.28:0/2481941638 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f460806c940 0x7f460806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:37.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.259+0000 7f461f400640 1 -- 172.21.15.28:0/2481941638 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f461408b6f0 con 0x7f46281397b0 2024-04-26T20:57:37.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.259+0000 7f4627400640 1 --2- 172.21.15.28:0/2481941638 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f460806c940 0x7f460806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:37.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.260+0000 7f4627400640 1 --2- 172.21.15.28:0/2481941638 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f460806c940 0x7f460806ee00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4618034000 tx=0x7f461802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:37.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.264+0000 7f461f400640 1 -- 172.21.15.28:0/2481941638 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f46280d0a30 con 0x7f46281397b0 2024-04-26T20:57:37.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.481+0000 7f462e000640 1 -- 172.21.15.28:0/2481941638 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f462806ff00 con 0x7f46281397b0 2024-04-26T20:57:37.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.482+0000 7f461f400640 1 -- 172.21.15.28:0/2481941638 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7f46140594c0 con 0x7f46281397b0 2024-04-26T20:57:37.483 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-04-26T20:57:37.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.485+0000 7f462e000640 1 -- 172.21.15.28:0/2481941638 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f460806c940 msgr2=0x7f460806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:37.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.485+0000 7f462e000640 1 --2- 172.21.15.28:0/2481941638 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f460806c940 0x7f460806ee00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4618034000 tx=0x7f461802e000 comp rx=0 tx=0).stop 2024-04-26T20:57:37.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.485+0000 7f462e000640 1 -- 172.21.15.28:0/2481941638 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46281397b0 msgr2=0x7f46280b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:37.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.485+0000 7f462e000640 1 --2- 172.21.15.28:0/2481941638 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46281397b0 0x7f46280b5f00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f461400d8e0 tx=0x7f461400dda0 comp rx=0 tx=0).stop 2024-04-26T20:57:37.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.485+0000 7f462e000640 1 -- 172.21.15.28:0/2481941638 shutdown_connections 2024-04-26T20:57:37.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.485+0000 7f462e000640 1 --2- 172.21.15.28:0/2481941638 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f460806c940 0x7f460806ee00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:37.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.485+0000 7f462e000640 1 --2- 172.21.15.28:0/2481941638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46280b6440 0x7f462815baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:37.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.485+0000 7f462e000640 1 --2- 172.21.15.28:0/2481941638 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f46281397b0 0x7f46280b5f00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:37.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.485+0000 7f462e000640 1 --2- 172.21.15.28:0/2481941638 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f460c0039e0 0x7f4628159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:37.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.485+0000 7f462e000640 1 -- 172.21.15.28:0/2481941638 >> 172.21.15.28:0/2481941638 conn(0x7f462806e220 msgr2=0x7f462815d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:37.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.485+0000 7f462e000640 1 -- 172.21.15.28:0/2481941638 shutdown_connections 2024-04-26T20:57:37.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.485+0000 7f462e000640 1 -- 172.21.15.28:0/2481941638 wait complete. 2024-04-26T20:57:37.505 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi028.front.sepia.ceph.com... 2024-04-26T20:57:37.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:37.506 DEBUG:teuthology.orchestra.run.smithi028:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-26T20:57:37.583 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:57:37.584 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T20:57:37.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:37.584 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T20:57:37.644 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:57:37.644 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:57:37.703 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:57:37.703 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:57:37.759 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:57:37.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.975+0000 7fa103e00640 1 Processor -- start 2024-04-26T20:57:37.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.975+0000 7fa103e00640 1 -- start start 2024-04-26T20:57:37.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.976+0000 7fa103e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa0fc15f5e0 0x7fa0fc0bacc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:37.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.976+0000 7fa103e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa0fc0cfe70 0x7fa0fc0d0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:37.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.976+0000 7fa103e00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0fc0b8240 con 0x7fa0fc15f200 2024-04-26T20:57:37.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.976+0000 7fa103e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fa0fc0d3810 con 0x7fa0fc15f5e0 2024-04-26T20:57:37.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.976+0000 7fa103e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa0fc163760 con 0x7fa0fc0cfe70 2024-04-26T20:57:37.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.977+0000 7fa100a00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa0fc15f5e0 0x7fa0fc0bacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:37.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.977+0000 7fa101400640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7fa0fc15f200 0x7fa0fc162f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.28:50524/0 (socket says 172.21.15.28:50524) 2024-04-26T20:57:37.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.977+0000 7fa101400640 1 -- 172.21.15.28:0/3878368683 learned_addr learned my addr 172.21.15.28:0/3878368683 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:37.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.977+0000 7fa101e00640 1 --2- 172.21.15.28:0/3878368683 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa0fc0cfe70 0x7fa0fc0d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:37.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.978+0000 7fa0fbe00640 1 -- 172.21.15.28:0/3878368683 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2211183802 0 0) 0x7fa0ec001490 con 0x7fa0fc15f200 2024-04-26T20:57:37.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.978+0000 7fa0fbe00640 1 -- 172.21.15.28:0/3878368683 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0e0003520 con 0x7fa0fc15f200 2024-04-26T20:57:37.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.978+0000 7fa100a00640 1 -- 172.21.15.28:0/3878368683 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa0fc0cfe70 msgr2=0x7fa0fc0d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:37.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.978+0000 7fa100a00640 1 --2- 172.21.15.28:0/3878368683 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa0fc0cfe70 0x7fa0fc0d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:37.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.978+0000 7fa100a00640 1 -- 172.21.15.28:0/3878368683 >> v1:172.21.15.103:6789/0 conn(0x7fa0fc15f200 legacy=0x7fa0fc162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:37.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.978+0000 7fa100a00640 1 -- 172.21.15.28:0/3878368683 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc163a40 con 0x7fa0fc15f5e0 2024-04-26T20:57:37.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.980+0000 7fa100a00640 1 --2- 172.21.15.28:0/3878368683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0fc15f5e0 0x7fa0fc0bacc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80097a0 tx=0x7fa0e802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=472e104725a65922 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:37.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.980+0000 7fa0fbe00640 1 -- 172.21.15.28:0/3878368683 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8037070 con 0x7fa0fc15f5e0 2024-04-26T20:57:37.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.980+0000 7fa0fbe00640 1 -- 172.21.15.28:0/3878368683 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0e802ae50 con 0x7fa0fc15f5e0 2024-04-26T20:57:37.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.980+0000 7fa0fbe00640 1 -- 172.21.15.28:0/3878368683 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8032640 con 0x7fa0fc15f5e0 2024-04-26T20:57:37.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.981+0000 7fa103e00640 1 -- 172.21.15.28:0/3878368683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0fc15f5e0 msgr2=0x7fa0fc0bacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:37.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.981+0000 7fa103e00640 1 --2- 172.21.15.28:0/3878368683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0fc15f5e0 0x7fa0fc0bacc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa0e80097a0 tx=0x7fa0e802a100 comp rx=0 tx=0).stop 2024-04-26T20:57:37.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.981+0000 7fa103e00640 1 -- 172.21.15.28:0/3878368683 shutdown_connections 2024-04-26T20:57:37.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.981+0000 7fa103e00640 1 --2- 172.21.15.28:0/3878368683 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa0fc0cfe70 0x7fa0fc0d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:37.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.981+0000 7fa103e00640 1 --2- 172.21.15.28:0/3878368683 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0fc15f5e0 0x7fa0fc0bacc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:37.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.981+0000 7fa103e00640 1 -- 172.21.15.28:0/3878368683 >> 172.21.15.28:0/3878368683 conn(0x7fa0fc0b6bb0 msgr2=0x7fa0fc0b6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:37.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.981+0000 7fa103e00640 1 -- 172.21.15.28:0/3878368683 shutdown_connections 2024-04-26T20:57:37.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.981+0000 7fa103e00640 1 -- 172.21.15.28:0/3878368683 wait complete. 2024-04-26T20:57:37.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.983+0000 7fa103e00640 1 Processor -- start 2024-04-26T20:57:37.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.983+0000 7fa103e00640 1 -- start start 2024-04-26T20:57:37.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.984+0000 7fa103e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0fc0cfe70 0x7fa0fc15a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:37.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.984+0000 7fa103e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa0fc15f200 0x7fa0fc1f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:37.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.984+0000 7fa103e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0fc15f5e0 0x7fa0fc15afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:37.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.984+0000 7fa103e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc0d3ca0 con 0x7fa0fc0cfe70 2024-04-26T20:57:37.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.984+0000 7fa103e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc0d34a0 con 0x7fa0fc15f5e0 2024-04-26T20:57:37.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.984+0000 7fa103e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa0fc0d3e20 con 0x7fa0fc15f200 2024-04-26T20:57:37.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.985+0000 7fa101400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0fc0cfe70 0x7fa0fc15a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:37.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.985+0000 7fa101400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0fc0cfe70 0x7fa0fc15a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54956/0 (socket says 172.21.15.28:54956) 2024-04-26T20:57:37.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.985+0000 7fa101400640 1 -- 172.21.15.28:0/2142508244 learned_addr learned my addr 172.21.15.28:0/2142508244 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:37.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.985+0000 7fa100a00640 1 --2- 172.21.15.28:0/2142508244 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa0fc15f200 0x7fa0fc1f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:37.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.985+0000 7fa101e00640 1 --2- 172.21.15.28:0/2142508244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0fc15f5e0 0x7fa0fc15afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:37.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.985+0000 7fa101400640 1 -- 172.21.15.28:0/2142508244 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa0fc15f200 msgr2=0x7fa0fc1f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:37.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.985+0000 7fa101400640 1 --2- 172.21.15.28:0/2142508244 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa0fc15f200 0x7fa0fc1f4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:37.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.985+0000 7fa101400640 1 -- 172.21.15.28:0/2142508244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0fc15f5e0 msgr2=0x7fa0fc15afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:37.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.985+0000 7fa101400640 1 --2- 172.21.15.28:0/2142508244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0fc15f5e0 0x7fa0fc15afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:37.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.985+0000 7fa101400640 1 -- 172.21.15.28:0/2142508244 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc1f5cf0 con 0x7fa0fc0cfe70 2024-04-26T20:57:37.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.986+0000 7fa101400640 1 --2- 172.21.15.28:0/2142508244 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0fc0cfe70 0x7fa0fc15a7b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00cb80 tx=0x7fa0ec00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:37.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.986+0000 7fa0f9600640 1 -- 172.21.15.28:0/2142508244 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec017070 con 0x7fa0fc0cfe70 2024-04-26T20:57:37.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.986+0000 7fa0f9600640 1 -- 172.21.15.28:0/2142508244 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0ec00ad40 con 0x7fa0fc0cfe70 2024-04-26T20:57:37.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.986+0000 7fa103e00640 1 -- 172.21.15.28:0/2142508244 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0fc0b6910 con 0x7fa0fc0cfe70 2024-04-26T20:57:37.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.986+0000 7fa0f9600640 1 -- 172.21.15.28:0/2142508244 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec012640 con 0x7fa0fc0cfe70 2024-04-26T20:57:37.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.986+0000 7fa103e00640 1 -- 172.21.15.28:0/2142508244 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0fc1f5730 con 0x7fa0fc0cfe70 2024-04-26T20:57:37.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.987+0000 7fa103e00640 1 -- 172.21.15.28:0/2142508244 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0fc06a400 con 0x7fa0fc0cfe70 2024-04-26T20:57:37.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.988+0000 7fa0f9600640 1 -- 172.21.15.28:0/2142508244 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0ec01a070 con 0x7fa0fc0cfe70 2024-04-26T20:57:37.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.990+0000 7fa0f9600640 1 --2- 172.21.15.28:0/2142508244 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa0dc06c8f0 0x7fa0dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:37.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.990+0000 7fa0f9600640 1 -- 172.21.15.28:0/2142508244 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa0ec08ba90 con 0x7fa0fc0cfe70 2024-04-26T20:57:37.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.990+0000 7fa100a00640 1 --2- 172.21.15.28:0/2142508244 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa0dc06c8f0 0x7fa0dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:37.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.991+0000 7fa100a00640 1 --2- 172.21.15.28:0/2142508244 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa0dc06c8f0 0x7fa0dc06edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8034000 tx=0x7fa0e802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:37.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:37.995+0000 7fa0f9600640 1 -- 172.21.15.28:0/2142508244 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa0fc0b8240 con 0x7fa0fc0cfe70 2024-04-26T20:57:38.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.210+0000 7fa103e00640 1 -- 172.21.15.28:0/2142508244 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0fc0d2d70 con 0x7fa0fc0cfe70 2024-04-26T20:57:38.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.213+0000 7fa0f9600640 1 -- 172.21.15.28:0/2142508244 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16831 (secure 0 0 0) 0x7fa0ec059e30 con 0x7fa0fc0cfe70 2024-04-26T20:57:38.214 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:38.214 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":23,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:57:37.996297+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:57:38.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.216+0000 7fa103e00640 1 -- 172.21.15.28:0/2142508244 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa0dc06c8f0 msgr2=0x7fa0dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:38.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.216+0000 7fa103e00640 1 --2- 172.21.15.28:0/2142508244 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa0dc06c8f0 0x7fa0dc06edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8034000 tx=0x7fa0e802e000 comp rx=0 tx=0).stop 2024-04-26T20:57:38.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.216+0000 7fa103e00640 1 -- 172.21.15.28:0/2142508244 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0fc0cfe70 msgr2=0x7fa0fc15a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:38.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.216+0000 7fa103e00640 1 --2- 172.21.15.28:0/2142508244 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0fc0cfe70 0x7fa0fc15a7b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec00cb80 tx=0x7fa0ec00a430 comp rx=0 tx=0).stop 2024-04-26T20:57:38.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.216+0000 7fa103e00640 1 -- 172.21.15.28:0/2142508244 shutdown_connections 2024-04-26T20:57:38.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.216+0000 7fa103e00640 1 --2- 172.21.15.28:0/2142508244 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa0dc06c8f0 0x7fa0dc06edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:38.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.216+0000 7fa103e00640 1 --2- 172.21.15.28:0/2142508244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0fc15f5e0 0x7fa0fc15afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:38.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.216+0000 7fa103e00640 1 --2- 172.21.15.28:0/2142508244 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa0fc15f200 0x7fa0fc1f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:38.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.216+0000 7fa103e00640 1 --2- 172.21.15.28:0/2142508244 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0fc0cfe70 0x7fa0fc15a7b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:38.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.216+0000 7fa103e00640 1 -- 172.21.15.28:0/2142508244 >> 172.21.15.28:0/2142508244 conn(0x7fa0fc0b6bb0 msgr2=0x7fa0fc0caef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:38.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.216+0000 7fa103e00640 1 -- 172.21.15.28:0/2142508244 shutdown_connections 2024-04-26T20:57:38.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.216+0000 7fa103e00640 1 -- 172.21.15.28:0/2142508244 wait complete. 2024-04-26T20:57:38.237 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:56:48.688498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:56:50.117541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:57:38.238 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:57:38.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.444+0000 7f6b0ee00640 1 Processor -- start 2024-04-26T20:57:38.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.445+0000 7f6b0ee00640 1 -- start start 2024-04-26T20:57:38.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.446+0000 7f6b0ee00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6b080d1de0 0x7f6b080d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:38.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.446+0000 7f6b0ee00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f6b080bac00 0x7f6b080b8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:38.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.446+0000 7f6b0ee00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b080d2700 con 0x7f6b0815f360 2024-04-26T20:57:38.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.446+0000 7f6b0ee00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f6b080ba350 con 0x7f6b080d1de0 2024-04-26T20:57:38.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.446+0000 7f6b0ee00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f6b080b9ab0 con 0x7f6b080bac00 2024-04-26T20:57:38.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.447+0000 7f6b0da00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6b080d1de0 0x7f6b080d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:38.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.447+0000 7f6b0da00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6b080d1de0 0x7f6b080d21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54968/0 (socket says 172.21.15.28:54968) 2024-04-26T20:57:38.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.447+0000 7f6b0da00640 1 -- 172.21.15.28:0/917715884 learned_addr learned my addr 172.21.15.28:0/917715884 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:38.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.447+0000 7f6b0e400640 1 --2- 172.21.15.28:0/917715884 >> v2:172.21.15.103:3300/0 conn(0x7f6b080bac00 0x7f6b080b8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:38.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.447+0000 7f6b0da00640 1 -- 172.21.15.28:0/917715884 >> v2:172.21.15.103:3300/0 conn(0x7f6b080bac00 msgr2=0x7f6b080b8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:38.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.447+0000 7f6b0da00640 1 --2- 172.21.15.28:0/917715884 >> v2:172.21.15.103:3300/0 conn(0x7f6b080bac00 0x7f6b080b8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:38.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.447+0000 7f6b0da00640 1 -- 172.21.15.28:0/917715884 >> v1:172.21.15.103:6789/0 conn(0x7f6b0815f360 legacy=0x7f6b080d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:38.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.447+0000 7f6b0da00640 1 -- 172.21.15.28:0/917715884 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b080b9d90 con 0x7f6b080d1de0 2024-04-26T20:57:38.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.449+0000 7f6b0da00640 1 --2- 172.21.15.28:0/917715884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b080d1de0 0x7f6b080d21c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6b0808e350 tx=0x7f6afc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a68a50b8ee8b13 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:38.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.449+0000 7f6b07e00640 1 -- 172.21.15.28:0/917715884 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc037070 con 0x7f6b080d1de0 2024-04-26T20:57:38.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.449+0000 7f6b07e00640 1 -- 172.21.15.28:0/917715884 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6afc032440 con 0x7f6b080d1de0 2024-04-26T20:57:38.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.449+0000 7f6b07e00640 1 -- 172.21.15.28:0/917715884 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc032810 con 0x7f6b080d1de0 2024-04-26T20:57:38.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.450+0000 7f6b0ee00640 1 -- 172.21.15.28:0/917715884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b080d1de0 msgr2=0x7f6b080d21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:38.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.450+0000 7f6b0ee00640 1 --2- 172.21.15.28:0/917715884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b080d1de0 0x7f6b080d21c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6b0808e350 tx=0x7f6afc02a230 comp rx=0 tx=0).stop 2024-04-26T20:57:38.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.450+0000 7f6b0ee00640 1 -- 172.21.15.28:0/917715884 shutdown_connections 2024-04-26T20:57:38.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.450+0000 7f6b0ee00640 1 --2- 172.21.15.28:0/917715884 >> v2:172.21.15.103:3300/0 conn(0x7f6b080bac00 0x7f6b080b8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:38.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.450+0000 7f6b0ee00640 1 --2- 172.21.15.28:0/917715884 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b080d1de0 0x7f6b080d21c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:38.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.450+0000 7f6b0ee00640 1 -- 172.21.15.28:0/917715884 >> 172.21.15.28:0/917715884 conn(0x7f6b0806c770 msgr2=0x7f6b0806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:38.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.450+0000 7f6b0ee00640 1 -- 172.21.15.28:0/917715884 shutdown_connections 2024-04-26T20:57:38.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.450+0000 7f6b0ee00640 1 -- 172.21.15.28:0/917715884 wait complete. 2024-04-26T20:57:38.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.452+0000 7f6b0ee00640 1 Processor -- start 2024-04-26T20:57:38.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.452+0000 7f6b0ee00640 1 -- start start 2024-04-26T20:57:38.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.454+0000 7f6b0ee00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b080bac00 0x7f6b08159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.454+0000 7f6b0ee00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6b080d1de0 0x7f6b080ce970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.454+0000 7f6b0ee00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b0815f360 0x7f6b08161d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.454+0000 7f6b0ee00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6b080b9510 con 0x7f6b0815f360 2024-04-26T20:57:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.454+0000 7f6b0ee00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6b080d2f70 con 0x7f6b080bac00 2024-04-26T20:57:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.454+0000 7f6b0ee00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f6b080b9690 con 0x7f6b080d1de0 2024-04-26T20:57:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.454+0000 7f6b0e400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b0815f360 0x7f6b08161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.454+0000 7f6b0e400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b0815f360 0x7f6b08161d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54980/0 (socket says 172.21.15.28:54980) 2024-04-26T20:57:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.454+0000 7f6b0e400640 1 -- 172.21.15.28:0/886635529 learned_addr learned my addr 172.21.15.28:0/886635529 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.454+0000 7f6b0da00640 1 --2- 172.21.15.28:0/886635529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b080bac00 0x7f6b08159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.455+0000 7f6b0e400640 1 -- 172.21.15.28:0/886635529 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6b080d1de0 msgr2=0x7f6b080ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.455+0000 7f6b0e400640 1 --2- 172.21.15.28:0/886635529 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6b080d1de0 0x7f6b080ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.455+0000 7f6b0e400640 1 -- 172.21.15.28:0/886635529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b080bac00 msgr2=0x7f6b08159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.455+0000 7f6b0e400640 1 --2- 172.21.15.28:0/886635529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b080bac00 0x7f6b08159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:38.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.455+0000 7f6b0e400640 1 -- 172.21.15.28:0/886635529 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b081590d0 con 0x7f6b0815f360 2024-04-26T20:57:38.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.455+0000 7f6b0e400640 1 --2- 172.21.15.28:0/886635529 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b0815f360 0x7f6b08161d50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000d5a0 tx=0x7f6b0000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:38.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.455+0000 7f6b05600640 1 -- 172.21.15.28:0/886635529 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b000125f0 con 0x7f6b0815f360 2024-04-26T20:57:38.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.455+0000 7f6b05600640 1 -- 172.21.15.28:0/886635529 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b00019070 con 0x7f6b0815f360 2024-04-26T20:57:38.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.455+0000 7f6b05600640 1 -- 172.21.15.28:0/886635529 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b000129f0 con 0x7f6b0815f360 2024-04-26T20:57:38.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.455+0000 7f6b0ee00640 1 -- 172.21.15.28:0/886635529 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b08158830 con 0x7f6b0815f360 2024-04-26T20:57:38.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.456+0000 7f6b0ee00640 1 -- 172.21.15.28:0/886635529 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b08158df0 con 0x7f6b0815f360 2024-04-26T20:57:38.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.457+0000 7f6b0ee00640 1 -- 172.21.15.28:0/886635529 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b0806a250 con 0x7f6b0815f360 2024-04-26T20:57:38.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.458+0000 7f6b05600640 1 -- 172.21.15.28:0/886635529 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6b00012b90 con 0x7f6b0815f360 2024-04-26T20:57:38.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.459+0000 7f6b05600640 1 --2- 172.21.15.28:0/886635529 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6aec06c8f0 0x7f6aec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:38.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.459+0000 7f6b05600640 1 -- 172.21.15.28:0/886635529 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f6b0008bf60 con 0x7f6b0815f360 2024-04-26T20:57:38.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.459+0000 7f6b0da00640 1 --2- 172.21.15.28:0/886635529 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6aec06c8f0 0x7f6aec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:38.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.460+0000 7f6b0da00640 1 --2- 172.21.15.28:0/886635529 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6aec06c8f0 0x7f6aec06edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6afc034000 tx=0x7f6afc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:38.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.463+0000 7f6b05600640 1 -- 172.21.15.28:0/886635529 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6b0005a280 con 0x7f6b0815f360 2024-04-26T20:57:38.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.669+0000 7f6b0ee00640 1 -- 172.21.15.28:0/886635529 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6b08068c40 con 0x7f6b0815f360 2024-04-26T20:57:38.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.670+0000 7f6b05600640 1 -- 172.21.15.28:0/886635529 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f6b00059b80 con 0x7f6b0815f360 2024-04-26T20:57:38.671 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:57:38.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.672+0000 7f6b0ee00640 1 -- 172.21.15.28:0/886635529 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6aec06c8f0 msgr2=0x7f6aec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:38.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.672+0000 7f6b0ee00640 1 --2- 172.21.15.28:0/886635529 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6aec06c8f0 0x7f6aec06edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6afc034000 tx=0x7f6afc02e000 comp rx=0 tx=0).stop 2024-04-26T20:57:38.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.672+0000 7f6b0ee00640 1 -- 172.21.15.28:0/886635529 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b0815f360 msgr2=0x7f6b08161d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:38.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.672+0000 7f6b0ee00640 1 --2- 172.21.15.28:0/886635529 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b0815f360 0x7f6b08161d50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6b0000d5a0 tx=0x7f6b0000da60 comp rx=0 tx=0).stop 2024-04-26T20:57:38.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.673+0000 7f6b0ee00640 1 -- 172.21.15.28:0/886635529 shutdown_connections 2024-04-26T20:57:38.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.673+0000 7f6b0ee00640 1 --2- 172.21.15.28:0/886635529 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6aec06c8f0 0x7f6aec06edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:38.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.673+0000 7f6b0ee00640 1 --2- 172.21.15.28:0/886635529 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b0815f360 0x7f6b08161d50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:38.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.673+0000 7f6b0ee00640 1 --2- 172.21.15.28:0/886635529 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6b080d1de0 0x7f6b080ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:38.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.673+0000 7f6b0ee00640 1 --2- 172.21.15.28:0/886635529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6b080bac00 0x7f6b08159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:38.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.673+0000 7f6b0ee00640 1 -- 172.21.15.28:0/886635529 >> 172.21.15.28:0/886635529 conn(0x7f6b0806c770 msgr2=0x7f6b080cf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:38.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.673+0000 7f6b0ee00640 1 -- 172.21.15.28:0/886635529 shutdown_connections 2024-04-26T20:57:38.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.673+0000 7f6b0ee00640 1 -- 172.21.15.28:0/886635529 wait complete. 2024-04-26T20:57:38.696 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:57:38.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.907+0000 7fb2a9400640 1 Processor -- start 2024-04-26T20:57:38.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.908+0000 7fb2a9400640 1 -- start start 2024-04-26T20:57:38.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.909+0000 7fb2a9400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb2a40cf040 0x7fb2a40d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:38.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.909+0000 7fb2a9400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2a40d1250 0x7fb2a40b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:38.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.909+0000 7fb2a9400640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2a40b5ba0 con 0x7fb2a41465c0 2024-04-26T20:57:38.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.909+0000 7fb2a9400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb2a40d1680 con 0x7fb2a40cf040 2024-04-26T20:57:38.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.909+0000 7fb2a9400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a41469a0 con 0x7fb2a40d1250 2024-04-26T20:57:38.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.909+0000 7fb2a3400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb2a40cf040 0x7fb2a40d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:38.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.909+0000 7fb2a3400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb2a40cf040 0x7fb2a40d0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54984/0 (socket says 172.21.15.28:54984) 2024-04-26T20:57:38.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.909+0000 7fb2a3400640 1 -- 172.21.15.28:0/4080067252 learned_addr learned my addr 172.21.15.28:0/4080067252 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:38.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.910+0000 7fb2a8a00640 1 --2- 172.21.15.28:0/4080067252 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2a40d1250 0x7fb2a40b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:38.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.910+0000 7fb2a3400640 1 -- 172.21.15.28:0/4080067252 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2a40d1250 msgr2=0x7fb2a40b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:38.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.910+0000 7fb2a3400640 1 --2- 172.21.15.28:0/4080067252 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2a40d1250 0x7fb2a40b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:38.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.910+0000 7fb2a3400640 1 -- 172.21.15.28:0/4080067252 >> v1:172.21.15.103:6789/0 conn(0x7fb2a41465c0 legacy=0x7fb2a406e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:38.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.910+0000 7fb2a3400640 1 -- 172.21.15.28:0/4080067252 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a40cf520 con 0x7fb2a40cf040 2024-04-26T20:57:38.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.911+0000 7fb2a3400640 1 --2- 172.21.15.28:0/4080067252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a40cf040 0x7fb2a40d0d10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb290009870 tx=0x7fb29002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f4d8a45ba3d5a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:38.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.911+0000 7fb2a2a00640 1 -- 172.21.15.28:0/4080067252 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290037070 con 0x7fb2a40cf040 2024-04-26T20:57:38.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.911+0000 7fb2a2a00640 1 -- 172.21.15.28:0/4080067252 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb290032440 con 0x7fb2a40cf040 2024-04-26T20:57:38.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.911+0000 7fb2a2a00640 1 -- 172.21.15.28:0/4080067252 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb290032810 con 0x7fb2a40cf040 2024-04-26T20:57:38.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.912+0000 7fb2a9400640 1 -- 172.21.15.28:0/4080067252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a40cf040 msgr2=0x7fb2a40d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:38.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.912+0000 7fb2a9400640 1 --2- 172.21.15.28:0/4080067252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a40cf040 0x7fb2a40d0d10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb290009870 tx=0x7fb29002a180 comp rx=0 tx=0).stop 2024-04-26T20:57:38.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.912+0000 7fb2a9400640 1 -- 172.21.15.28:0/4080067252 shutdown_connections 2024-04-26T20:57:38.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.912+0000 7fb2a9400640 1 --2- 172.21.15.28:0/4080067252 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2a40d1250 0x7fb2a40b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:38.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.912+0000 7fb2a9400640 1 --2- 172.21.15.28:0/4080067252 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a40cf040 0x7fb2a40d0d10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:38.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.912+0000 7fb2a9400640 1 -- 172.21.15.28:0/4080067252 >> 172.21.15.28:0/4080067252 conn(0x7fb2a406c530 msgr2=0x7fb2a406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:38.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.912+0000 7fb2a9400640 1 -- 172.21.15.28:0/4080067252 shutdown_connections 2024-04-26T20:57:38.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.912+0000 7fb2a9400640 1 -- 172.21.15.28:0/4080067252 wait complete. 2024-04-26T20:57:38.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.914+0000 7fb2a9400640 1 Processor -- start 2024-04-26T20:57:38.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.914+0000 7fb2a9400640 1 -- start start 2024-04-26T20:57:38.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.915+0000 7fb2a9400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a40cf040 0x7fb2a415bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:38.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.915+0000 7fb2a9400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2a40d1250 0x7fb2a4072f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:38.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.915+0000 7fb2a9400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2a41465c0 0x7fb2a40734c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:38.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.915+0000 7fb2a9400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a40cfdc0 con 0x7fb2a40cf040 2024-04-26T20:57:38.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.915+0000 7fb2a9400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb2a40cfae0 con 0x7fb2a41465c0 2024-04-26T20:57:38.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.915+0000 7fb2a9400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb2a40d00a0 con 0x7fb2a40d1250 2024-04-26T20:57:38.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.915+0000 7fb2a3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a40cf040 0x7fb2a415bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:38.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.915+0000 7fb2a3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a40cf040 0x7fb2a415bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54996/0 (socket says 172.21.15.28:54996) 2024-04-26T20:57:38.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.915+0000 7fb2a3e00640 1 -- 172.21.15.28:0/3178321438 learned_addr learned my addr 172.21.15.28:0/3178321438 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:38.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.916+0000 7fb2a3400640 1 --2- 172.21.15.28:0/3178321438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2a40d1250 0x7fb2a4072f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:38.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.916+0000 7fb2a8a00640 1 --2- 172.21.15.28:0/3178321438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2a41465c0 0x7fb2a40734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:38.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.916+0000 7fb2a3e00640 1 -- 172.21.15.28:0/3178321438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2a40d1250 msgr2=0x7fb2a4072f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:38.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.916+0000 7fb2a3e00640 1 --2- 172.21.15.28:0/3178321438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2a40d1250 0x7fb2a4072f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:38.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.916+0000 7fb2a3e00640 1 -- 172.21.15.28:0/3178321438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2a41465c0 msgr2=0x7fb2a40734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:38.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.916+0000 7fb2a3e00640 1 --2- 172.21.15.28:0/3178321438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2a41465c0 0x7fb2a40734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:38.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.916+0000 7fb2a3e00640 1 -- 172.21.15.28:0/3178321438 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2a40ad470 con 0x7fb2a40cf040 2024-04-26T20:57:38.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.916+0000 7fb2a8a00640 1 --2- 172.21.15.28:0/3178321438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2a41465c0 0x7fb2a40734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:38.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.916+0000 7fb2a3e00640 1 --2- 172.21.15.28:0/3178321438 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a40cf040 0x7fb2a415bb80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb29400b420 tx=0x7fb29400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:38.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.917+0000 7fb29be00640 1 -- 172.21.15.28:0/3178321438 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294011830 con 0x7fb2a40cf040 2024-04-26T20:57:38.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.917+0000 7fb29be00640 1 -- 172.21.15.28:0/3178321438 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb294016070 con 0x7fb2a40cf040 2024-04-26T20:57:38.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.917+0000 7fb29be00640 1 -- 172.21.15.28:0/3178321438 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb294011c10 con 0x7fb2a40cf040 2024-04-26T20:57:38.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.917+0000 7fb2a9400640 1 -- 172.21.15.28:0/3178321438 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2a413f7e0 con 0x7fb2a40cf040 2024-04-26T20:57:38.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.917+0000 7fb2a9400640 1 -- 172.21.15.28:0/3178321438 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2a413fda0 con 0x7fb2a40cf040 2024-04-26T20:57:38.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.918+0000 7fb2a9400640 1 -- 172.21.15.28:0/3178321438 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2a406f3d0 con 0x7fb2a40cf040 2024-04-26T20:57:38.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.919+0000 7fb29be00640 1 -- 172.21.15.28:0/3178321438 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb29401b460 con 0x7fb2a40cf040 2024-04-26T20:57:38.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.920+0000 7fb29be00640 1 --2- 172.21.15.28:0/3178321438 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb28406c8f0 0x7fb28406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:38.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.920+0000 7fb29be00640 1 -- 172.21.15.28:0/3178321438 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb29408b720 con 0x7fb2a40cf040 2024-04-26T20:57:38.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.920+0000 7fb2a3400640 1 --2- 172.21.15.28:0/3178321438 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb28406c8f0 0x7fb28406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:38.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.921+0000 7fb2a3400640 1 --2- 172.21.15.28:0/3178321438 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb28406c8f0 0x7fb28406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb290034000 tx=0x7fb29002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:38.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:38.924+0000 7fb29be00640 1 -- 172.21.15.28:0/3178321438 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb294020020 con 0x7fb2a40cf040 2024-04-26T20:57:39.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.134+0000 7fb2a9400640 1 -- 172.21.15.28:0/3178321438 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb2a406a250 con 0x7fb2a40cf040 2024-04-26T20:57:39.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.134+0000 7fb29be00640 1 -- 172.21.15.28:0/3178321438 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fb294059a40 con 0x7fb2a40cf040 2024-04-26T20:57:39.138 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-04-26T20:57:39.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.137+0000 7fb2a9400640 1 -- 172.21.15.28:0/3178321438 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb28406c8f0 msgr2=0x7fb28406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:39.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.137+0000 7fb2a9400640 1 --2- 172.21.15.28:0/3178321438 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb28406c8f0 0x7fb28406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb290034000 tx=0x7fb29002e000 comp rx=0 tx=0).stop 2024-04-26T20:57:39.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.137+0000 7fb2a9400640 1 -- 172.21.15.28:0/3178321438 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a40cf040 msgr2=0x7fb2a415bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:39.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.137+0000 7fb2a9400640 1 --2- 172.21.15.28:0/3178321438 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a40cf040 0x7fb2a415bb80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb29400b420 tx=0x7fb29400b8e0 comp rx=0 tx=0).stop 2024-04-26T20:57:39.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.137+0000 7fb2a9400640 1 -- 172.21.15.28:0/3178321438 shutdown_connections 2024-04-26T20:57:39.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.137+0000 7fb2a9400640 1 --2- 172.21.15.28:0/3178321438 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb28406c8f0 0x7fb28406edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:39.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.137+0000 7fb2a9400640 1 --2- 172.21.15.28:0/3178321438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2a41465c0 0x7fb2a40734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:39.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.137+0000 7fb2a9400640 1 --2- 172.21.15.28:0/3178321438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2a40d1250 0x7fb2a4072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:39.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.137+0000 7fb2a9400640 1 --2- 172.21.15.28:0/3178321438 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2a40cf040 0x7fb2a415bb80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:39.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.137+0000 7fb2a9400640 1 -- 172.21.15.28:0/3178321438 >> 172.21.15.28:0/3178321438 conn(0x7fb2a406c530 msgr2=0x7fb2a40b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:39.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.138+0000 7fb2a9400640 1 -- 172.21.15.28:0/3178321438 shutdown_connections 2024-04-26T20:57:39.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.138+0000 7fb2a9400640 1 -- 172.21.15.28:0/3178321438 wait complete. 2024-04-26T20:57:39.157 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:57:39.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.374+0000 7f9b86600640 1 Processor -- start 2024-04-26T20:57:39.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.374+0000 7f9b86600640 1 -- start start 2024-04-26T20:57:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.376+0000 7f9b86600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9b800d34c0 0x7f9b800b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.376+0000 7f9b86600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b800cf520 0x7f9b800b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.376+0000 7f9b86600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b800b4450 con 0x7f9b8015d130 2024-04-26T20:57:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.376+0000 7f9b86600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f9b800d15a0 con 0x7f9b800d34c0 2024-04-26T20:57:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.376+0000 7f9b86600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8015d510 con 0x7f9b800cf520 2024-04-26T20:57:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.376+0000 7f9b7f400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f9b8015d130 0x7f9b800d2db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58096/0 (socket says 172.21.15.28:58096) 2024-04-26T20:57:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.376+0000 7f9b7f400640 1 -- 172.21.15.28:0/1237531224 learned_addr learned my addr 172.21.15.28:0/1237531224 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.376+0000 7f9b7ea00640 1 --2- 172.21.15.28:0/1237531224 >> v2:172.21.15.28:3300/0 conn(0x7f9b800d34c0 0x7f9b800b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:39.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.376+0000 7f9b7fe00640 1 --2- 172.21.15.28:0/1237531224 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b800cf520 0x7f9b800b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:39.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.377+0000 7f9b7ea00640 1 -- 172.21.15.28:0/1237531224 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b800cf520 msgr2=0x7f9b800b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:39.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.377+0000 7f9b7ea00640 1 --2- 172.21.15.28:0/1237531224 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b800cf520 0x7f9b800b3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:39.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.377+0000 7f9b7ea00640 1 -- 172.21.15.28:0/1237531224 >> v1:172.21.15.28:6789/0 conn(0x7f9b8015d130 legacy=0x7f9b800d2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:39.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.377+0000 7f9b7ea00640 1 -- 172.21.15.28:0/1237531224 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b800cda30 con 0x7f9b800d34c0 2024-04-26T20:57:39.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.377+0000 7f9b7e000640 1 -- 172.21.15.28:0/1237531224 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 607604683 0 0) 0x7f9b800b4450 con 0x7f9b8015d130 2024-04-26T20:57:39.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.378+0000 7f9b7ea00640 1 --2- 172.21.15.28:0/1237531224 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b800d34c0 0x7f9b800b3800 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0097a0 tx=0x7f9b6c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9eb843c42c1fae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:39.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.378+0000 7f9b7e000640 1 -- 172.21.15.28:0/1237531224 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c037070 con 0x7f9b800d34c0 2024-04-26T20:57:39.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.378+0000 7f9b7e000640 1 -- 172.21.15.28:0/1237531224 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b6c032440 con 0x7f9b800d34c0 2024-04-26T20:57:39.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.378+0000 7f9b7e000640 1 -- 172.21.15.28:0/1237531224 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c032810 con 0x7f9b800d34c0 2024-04-26T20:57:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.379+0000 7f9b86600640 1 -- 172.21.15.28:0/1237531224 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b800d34c0 msgr2=0x7f9b800b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.379+0000 7f9b86600640 1 --2- 172.21.15.28:0/1237531224 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b800d34c0 0x7f9b800b3800 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0097a0 tx=0x7f9b6c02a180 comp rx=0 tx=0).stop 2024-04-26T20:57:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.379+0000 7f9b86600640 1 -- 172.21.15.28:0/1237531224 shutdown_connections 2024-04-26T20:57:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.379+0000 7f9b86600640 1 --2- 172.21.15.28:0/1237531224 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b800cf520 0x7f9b800b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.379+0000 7f9b86600640 1 --2- 172.21.15.28:0/1237531224 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b800d34c0 0x7f9b800b3800 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.379+0000 7f9b86600640 1 -- 172.21.15.28:0/1237531224 >> 172.21.15.28:0/1237531224 conn(0x7f9b8009ba50 msgr2=0x7f9b8009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.379+0000 7f9b86600640 1 -- 172.21.15.28:0/1237531224 shutdown_connections 2024-04-26T20:57:39.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.379+0000 7f9b86600640 1 -- 172.21.15.28:0/1237531224 wait complete. 2024-04-26T20:57:39.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.381+0000 7f9b86600640 1 Processor -- start 2024-04-26T20:57:39.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.381+0000 7f9b86600640 1 -- start start 2024-04-26T20:57:39.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.382+0000 7f9b86600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b800cf520 0x7f9b800cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:39.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.382+0000 7f9b86600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b800d34c0 0x7f9b801f8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:39.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.382+0000 7f9b86600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b8015d130 0x7f9b8015bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:39.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.382+0000 7f9b86600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9b800b4730 con 0x7f9b800d34c0 2024-04-26T20:57:39.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.382+0000 7f9b86600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9b800cdff0 con 0x7f9b800cf520 2024-04-26T20:57:39.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.382+0000 7f9b86600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9b80166bc0 con 0x7f9b8015d130 2024-04-26T20:57:39.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.382+0000 7f9b7ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b800d34c0 0x7f9b801f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:39.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.382+0000 7f9b7ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b800d34c0 0x7f9b801f8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:55018/0 (socket says 172.21.15.28:55018) 2024-04-26T20:57:39.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.382+0000 7f9b7ea00640 1 -- 172.21.15.28:0/3561514289 learned_addr learned my addr 172.21.15.28:0/3561514289 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:39.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.382+0000 7f9b7ea00640 1 -- 172.21.15.28:0/3561514289 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b8015d130 msgr2=0x7f9b8015bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:39.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.382+0000 7f9b7ea00640 1 --2- 172.21.15.28:0/3561514289 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b8015d130 0x7f9b8015bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:39.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.382+0000 7f9b7ea00640 1 -- 172.21.15.28:0/3561514289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b800cf520 msgr2=0x7f9b800cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:39.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.382+0000 7f9b7ea00640 1 --2- 172.21.15.28:0/3561514289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b800cf520 0x7f9b800cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:39.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.382+0000 7f9b7ea00640 1 -- 172.21.15.28:0/3561514289 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b801f8d30 con 0x7f9b800d34c0 2024-04-26T20:57:39.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.383+0000 7f9b7ea00640 1 --2- 172.21.15.28:0/3561514289 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b800d34c0 0x7f9b801f8590 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c035040 tx=0x7f9b6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:39.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.383+0000 7f9b77400640 1 -- 172.21.15.28:0/3561514289 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c037040 con 0x7f9b800d34c0 2024-04-26T20:57:39.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.383+0000 7f9b77400640 1 -- 172.21.15.28:0/3561514289 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b6c030770 con 0x7f9b800d34c0 2024-04-26T20:57:39.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.383+0000 7f9b77400640 1 -- 172.21.15.28:0/3561514289 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c030b80 con 0x7f9b800d34c0 2024-04-26T20:57:39.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.383+0000 7f9b86600640 1 -- 172.21.15.28:0/3561514289 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b801f9010 con 0x7f9b800d34c0 2024-04-26T20:57:39.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.383+0000 7f9b86600640 1 -- 172.21.15.28:0/3561514289 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b8015c420 con 0x7f9b800d34c0 2024-04-26T20:57:39.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.385+0000 7f9b86600640 1 -- 172.21.15.28:0/3561514289 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b800693b0 con 0x7f9b800d34c0 2024-04-26T20:57:39.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.386+0000 7f9b77400640 1 -- 172.21.15.28:0/3561514289 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b6c044020 con 0x7f9b800d34c0 2024-04-26T20:57:39.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.386+0000 7f9b77400640 1 --2- 172.21.15.28:0/3561514289 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9b6006c8f0 0x7f9b6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:39.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.387+0000 7f9b77400640 1 -- 172.21.15.28:0/3561514289 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f9b6c0ac1e0 con 0x7f9b800d34c0 2024-04-26T20:57:39.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.387+0000 7f9b7f400640 1 --2- 172.21.15.28:0/3561514289 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9b6006c8f0 0x7f9b6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:39.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.387+0000 7f9b7f400640 1 --2- 172.21.15.28:0/3561514289 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9b6006c8f0 0x7f9b6006edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9b70007fd0 tx=0x7f9b70007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:39.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.391+0000 7f9b77400640 1 -- 172.21.15.28:0/3561514289 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9b6c032e20 con 0x7f9b800d34c0 2024-04-26T20:57:39.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.597+0000 7f9b86600640 1 -- 172.21.15.28:0/3561514289 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9b800b7930 con 0x7f9b800d34c0 2024-04-26T20:57:39.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.598+0000 7f9b77400640 1 -- 172.21.15.28:0/3561514289 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f9b6c07a4f0 con 0x7f9b800d34c0 2024-04-26T20:57:39.599 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-04-26T20:57:39.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.600+0000 7f9b86600640 1 -- 172.21.15.28:0/3561514289 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9b6006c8f0 msgr2=0x7f9b6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:39.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.600+0000 7f9b86600640 1 --2- 172.21.15.28:0/3561514289 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9b6006c8f0 0x7f9b6006edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f9b70007fd0 tx=0x7f9b70007920 comp rx=0 tx=0).stop 2024-04-26T20:57:39.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.600+0000 7f9b86600640 1 -- 172.21.15.28:0/3561514289 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b800d34c0 msgr2=0x7f9b801f8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:39.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.600+0000 7f9b86600640 1 --2- 172.21.15.28:0/3561514289 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b800d34c0 0x7f9b801f8590 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c035040 tx=0x7f9b6c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:57:39.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.601+0000 7f9b86600640 1 -- 172.21.15.28:0/3561514289 shutdown_connections 2024-04-26T20:57:39.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.601+0000 7f9b86600640 1 --2- 172.21.15.28:0/3561514289 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9b6006c8f0 0x7f9b6006edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:39.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.601+0000 7f9b86600640 1 --2- 172.21.15.28:0/3561514289 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b8015d130 0x7f9b8015bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:39.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.601+0000 7f9b86600640 1 --2- 172.21.15.28:0/3561514289 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b800d34c0 0x7f9b801f8590 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:39.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.601+0000 7f9b86600640 1 --2- 172.21.15.28:0/3561514289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b800cf520 0x7f9b800cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:39.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.601+0000 7f9b86600640 1 -- 172.21.15.28:0/3561514289 >> 172.21.15.28:0/3561514289 conn(0x7f9b8009ba50 msgr2=0x7f9b800701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:39.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.601+0000 7f9b86600640 1 -- 172.21.15.28:0/3561514289 shutdown_connections 2024-04-26T20:57:39.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.601+0000 7f9b86600640 1 -- 172.21.15.28:0/3561514289 wait complete. 2024-04-26T20:57:39.623 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:57:39.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.838+0000 7f8962600640 1 Processor -- start 2024-04-26T20:57:39.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.838+0000 7f8962600640 1 -- start start 2024-04-26T20:57:39.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.839+0000 7f8962600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f895c0d1f10 0x7f895c0b7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:39.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.839+0000 7f8962600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f895c0d4500 con 0x7f895c0ba710 2024-04-26T20:57:39.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.839+0000 7f8962600640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f895c15f450 con 0x7f895c15f070 2024-04-26T20:57:39.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.840+0000 7f8962600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f895c0d4a80 con 0x7f895c0d1f10 2024-04-26T20:57:39.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.840+0000 7f8961c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f895c0d1f10 0x7f895c0b7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:39.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.840+0000 7f8961c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f895c0d1f10 0x7f895c0b7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50980/0 (socket says 172.21.15.28:50980) 2024-04-26T20:57:39.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.840+0000 7f895be00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f895c0ba710 0x7f895c0d1800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58400/0 (socket says 172.21.15.28:58400) 2024-04-26T20:57:39.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.840+0000 7f8961c00640 1 -- 172.21.15.28:0/1286702187 learned_addr learned my addr 172.21.15.28:0/1286702187 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:39.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.840+0000 7f8961c00640 1 -- 172.21.15.28:0/1286702187 >> v1:172.21.15.103:6789/0 conn(0x7f895c15f070 legacy=0x7f895c0d36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:39.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.840+0000 7f8961c00640 1 -- 172.21.15.28:0/1286702187 >> v1:172.21.15.28:6789/0 conn(0x7f895c0ba710 legacy=0x7f895c0d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:39.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.840+0000 7f8961c00640 1 -- 172.21.15.28:0/1286702187 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895c0d01c0 con 0x7f895c0d1f10 2024-04-26T20:57:39.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.840+0000 7f895b400640 1 -- 172.21.15.28:0/1286702187 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4167232708 0 0) 0x7f895c0d4500 con 0x7f895c0ba710 2024-04-26T20:57:39.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.841+0000 7f8961c00640 1 --2- 172.21.15.28:0/1286702187 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f895c0d1f10 0x7f895c0b7f20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8954009e50 tx=0x7f895402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584c6b40a0fd358 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:39.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.842+0000 7f895b400640 1 -- 172.21.15.28:0/1286702187 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8954036070 con 0x7f895c0d1f10 2024-04-26T20:57:39.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.842+0000 7f895b400640 1 -- 172.21.15.28:0/1286702187 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8954031600 con 0x7f895c0d1f10 2024-04-26T20:57:39.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.842+0000 7f895b400640 1 -- 172.21.15.28:0/1286702187 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89540319d0 con 0x7f895c0d1f10 2024-04-26T20:57:39.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.842+0000 7f8962600640 1 -- 172.21.15.28:0/1286702187 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f895c0d1f10 msgr2=0x7f895c0b7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:39.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.842+0000 7f8962600640 1 --2- 172.21.15.28:0/1286702187 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f895c0d1f10 0x7f895c0b7f20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f8954009e50 tx=0x7f895402a6d0 comp rx=0 tx=0).stop 2024-04-26T20:57:39.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.843+0000 7f8962600640 1 -- 172.21.15.28:0/1286702187 shutdown_connections 2024-04-26T20:57:39.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.843+0000 7f8962600640 1 --2- 172.21.15.28:0/1286702187 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f895c0d1f10 0x7f895c0b7f20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:39.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.843+0000 7f8962600640 1 -- 172.21.15.28:0/1286702187 >> 172.21.15.28:0/1286702187 conn(0x7f895c06c980 msgr2=0x7f895c072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:39.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.843+0000 7f8962600640 1 -- 172.21.15.28:0/1286702187 shutdown_connections 2024-04-26T20:57:39.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.843+0000 7f8962600640 1 -- 172.21.15.28:0/1286702187 wait complete. 2024-04-26T20:57:39.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.844+0000 7f8962600640 1 Processor -- start 2024-04-26T20:57:39.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.845+0000 7f8962600640 1 -- start start 2024-04-26T20:57:39.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.846+0000 7f8962600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f895c0ba710 0x7f895c1f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:39.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.846+0000 7f8962600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f895c0d1f10 0x7f895c1f6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:39.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.846+0000 7f8962600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f895c15f070 0x7f895c1f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:39.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.846+0000 7f8962600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f895c0cfde0 con 0x7f895c0d1f10 2024-04-26T20:57:39.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.846+0000 7f8962600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f895c0d04a0 con 0x7f895c15f070 2024-04-26T20:57:39.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.846+0000 7f8962600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f895c0cff60 con 0x7f895c0ba710 2024-04-26T20:57:39.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.846+0000 7f895be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f895c0d1f10 0x7f895c1f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:39.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.846+0000 7f895be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f895c0d1f10 0x7f895c1f6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50986/0 (socket says 172.21.15.28:50986) 2024-04-26T20:57:39.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.846+0000 7f895be00640 1 -- 172.21.15.28:0/2192872163 learned_addr learned my addr 172.21.15.28:0/2192872163 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:39.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.846+0000 7f8961200640 1 --2- 172.21.15.28:0/2192872163 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f895c0ba710 0x7f895c1f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:39.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.846+0000 7f8961c00640 1 --2- 172.21.15.28:0/2192872163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f895c15f070 0x7f895c1f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:39.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.846+0000 7f895be00640 1 -- 172.21.15.28:0/2192872163 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f895c0ba710 msgr2=0x7f895c1f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:39.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.846+0000 7f895be00640 1 --2- 172.21.15.28:0/2192872163 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f895c0ba710 0x7f895c1f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:39.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.846+0000 7f895be00640 1 -- 172.21.15.28:0/2192872163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f895c15f070 msgr2=0x7f895c1f65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:39.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.846+0000 7f895be00640 1 --2- 172.21.15.28:0/2192872163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f895c15f070 0x7f895c1f65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:39.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.846+0000 7f895be00640 1 -- 172.21.15.28:0/2192872163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895c0b6790 con 0x7f895c0d1f10 2024-04-26T20:57:39.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.846+0000 7f8961c00640 1 --2- 172.21.15.28:0/2192872163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f895c15f070 0x7f895c1f65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:39.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.847+0000 7f895be00640 1 --2- 172.21.15.28:0/2192872163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f895c0d1f10 0x7f895c1f6070 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f894800c4b0 tx=0x7f894800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:39.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.847+0000 7f8958c00640 1 -- 172.21.15.28:0/2192872163 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948018070 con 0x7f895c0d1f10 2024-04-26T20:57:39.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.847+0000 7f8958c00640 1 -- 172.21.15.28:0/2192872163 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8948007480 con 0x7f895c0d1f10 2024-04-26T20:57:39.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.847+0000 7f8958c00640 1 -- 172.21.15.28:0/2192872163 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948004820 con 0x7f895c0d1f10 2024-04-26T20:57:39.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.847+0000 7f8962600640 1 -- 172.21.15.28:0/2192872163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895c0b4ff0 con 0x7f895c0d1f10 2024-04-26T20:57:39.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.847+0000 7f8962600640 1 -- 172.21.15.28:0/2192872163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f895c1569b0 con 0x7f895c0d1f10 2024-04-26T20:57:39.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.848+0000 7f8962600640 1 -- 172.21.15.28:0/2192872163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f895c06a360 con 0x7f895c0d1f10 2024-04-26T20:57:39.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.850+0000 7f8958c00640 1 -- 172.21.15.28:0/2192872163 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89480049e0 con 0x7f895c0d1f10 2024-04-26T20:57:39.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.851+0000 7f8958c00640 1 --2- 172.21.15.28:0/2192872163 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f893c06c8f0 0x7f893c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:39.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.851+0000 7f8958c00640 1 -- 172.21.15.28:0/2192872163 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f894808af20 con 0x7f895c0d1f10 2024-04-26T20:57:39.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.851+0000 7f8961200640 1 --2- 172.21.15.28:0/2192872163 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f893c06c8f0 0x7f893c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:39.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.852+0000 7f8961200640 1 --2- 172.21.15.28:0/2192872163 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f893c06c8f0 0x7f893c06edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f894c006fd0 tx=0x7f894c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:39.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:39.857+0000 7f8958c00640 1 -- 172.21.15.28:0/2192872163 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8948059380 con 0x7f895c0d1f10 2024-04-26T20:57:40.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.076+0000 7f8962600640 1 -- 172.21.15.28:0/2192872163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f895c15a050 con 0x7f895c0d1f10 2024-04-26T20:57:40.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.077+0000 7f8958c00640 1 -- 172.21.15.28:0/2192872163 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f8948058c80 con 0x7f895c0d1f10 2024-04-26T20:57:40.078 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-04-26T20:57:40.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.079+0000 7f8962600640 1 -- 172.21.15.28:0/2192872163 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f893c06c8f0 msgr2=0x7f893c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:40.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.079+0000 7f8962600640 1 --2- 172.21.15.28:0/2192872163 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f893c06c8f0 0x7f893c06edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f894c006fd0 tx=0x7f894c008040 comp rx=0 tx=0).stop 2024-04-26T20:57:40.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.079+0000 7f8962600640 1 -- 172.21.15.28:0/2192872163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f895c0d1f10 msgr2=0x7f895c1f6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:40.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.079+0000 7f8962600640 1 --2- 172.21.15.28:0/2192872163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f895c0d1f10 0x7f895c1f6070 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f894800c4b0 tx=0x7f894800c970 comp rx=0 tx=0).stop 2024-04-26T20:57:40.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.080+0000 7f8962600640 1 -- 172.21.15.28:0/2192872163 shutdown_connections 2024-04-26T20:57:40.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.080+0000 7f8962600640 1 --2- 172.21.15.28:0/2192872163 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f893c06c8f0 0x7f893c06edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.080+0000 7f8962600640 1 --2- 172.21.15.28:0/2192872163 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f895c15f070 0x7f895c1f65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.080+0000 7f8962600640 1 --2- 172.21.15.28:0/2192872163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f895c0d1f10 0x7f895c1f6070 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.080+0000 7f8962600640 1 --2- 172.21.15.28:0/2192872163 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f895c0ba710 0x7f895c1f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.080+0000 7f8962600640 1 -- 172.21.15.28:0/2192872163 >> 172.21.15.28:0/2192872163 conn(0x7f895c06c980 msgr2=0x7f895c0cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:40.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.080+0000 7f8962600640 1 -- 172.21.15.28:0/2192872163 shutdown_connections 2024-04-26T20:57:40.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.080+0000 7f8962600640 1 -- 172.21.15.28:0/2192872163 wait complete. 2024-04-26T20:57:40.101 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:57:40.102 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:57:40.155 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:57:40.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.375+0000 7f7487200640 1 Processor -- start 2024-04-26T20:57:40.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.375+0000 7f7487200640 1 -- start start 2024-04-26T20:57:40.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.376+0000 7f7487200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f7488073a00 0x7f7488073de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:40.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.376+0000 7f7487200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f748815c4d0 0x7f748815c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:40.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.376+0000 7f7487200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7488074320 con 0x7f74880cf180 2024-04-26T20:57:40.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.376+0000 7f7487200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f748815ce90 con 0x7f748815c4d0 2024-04-26T20:57:40.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.376+0000 7f7487200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f748815d0e0 con 0x7f7488073a00 2024-04-26T20:57:40.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.377+0000 7f7485400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f748815c4d0 0x7f748815c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:40.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.377+0000 7f7485e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f7488073a00 0x7f7488073de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:40.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.377+0000 7f7485400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f748815c4d0 0x7f748815c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51002/0 (socket says 172.21.15.28:51002) 2024-04-26T20:57:40.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.377+0000 7f7485400640 1 -- 172.21.15.28:0/3461360216 learned_addr learned my addr 172.21.15.28:0/3461360216 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:40.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.377+0000 7f7485400640 1 -- 172.21.15.28:0/3461360216 >> v2:172.21.15.103:3300/0 conn(0x7f7488073a00 msgr2=0x7f7488073de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:40.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.377+0000 7f7485400640 1 --2- 172.21.15.28:0/3461360216 >> v2:172.21.15.103:3300/0 conn(0x7f7488073a00 0x7f7488073de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.377+0000 7f7485400640 1 -- 172.21.15.28:0/3461360216 >> v1:172.21.15.103:6789/0 conn(0x7f74880cf180 legacy=0x7f74880cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:40.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.377+0000 7f7485400640 1 -- 172.21.15.28:0/3461360216 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748815d3c0 con 0x7f748815c4d0 2024-04-26T20:57:40.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.379+0000 7f7485400640 1 --2- 172.21.15.28:0/3461360216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f748815c4d0 0x7f748815c950 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7488072180 tx=0x7f747402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a3a2782771adbe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:40.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.379+0000 7f7484a00640 1 -- 172.21.15.28:0/3461360216 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474037070 con 0x7f748815c4d0 2024-04-26T20:57:40.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.379+0000 7f7484a00640 1 -- 172.21.15.28:0/3461360216 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7474032440 con 0x7f748815c4d0 2024-04-26T20:57:40.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.379+0000 7f7484a00640 1 -- 172.21.15.28:0/3461360216 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474032810 con 0x7f748815c4d0 2024-04-26T20:57:40.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.379+0000 7f7487200640 1 -- 172.21.15.28:0/3461360216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f748815c4d0 msgr2=0x7f748815c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:40.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.379+0000 7f7487200640 1 --2- 172.21.15.28:0/3461360216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f748815c4d0 0x7f748815c950 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7488072180 tx=0x7f747402a270 comp rx=0 tx=0).stop 2024-04-26T20:57:40.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.380+0000 7f7487200640 1 -- 172.21.15.28:0/3461360216 shutdown_connections 2024-04-26T20:57:40.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.380+0000 7f7487200640 1 --2- 172.21.15.28:0/3461360216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f748815c4d0 0x7f748815c950 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.380+0000 7f7487200640 1 --2- 172.21.15.28:0/3461360216 >> v2:172.21.15.103:3300/0 conn(0x7f7488073a00 0x7f7488073de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.380+0000 7f7487200640 1 -- 172.21.15.28:0/3461360216 >> 172.21.15.28:0/3461360216 conn(0x7f74880b2dd0 msgr2=0x7f74880b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:40.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.380+0000 7f7487200640 1 -- 172.21.15.28:0/3461360216 shutdown_connections 2024-04-26T20:57:40.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.380+0000 7f7487200640 1 -- 172.21.15.28:0/3461360216 wait complete. 2024-04-26T20:57:40.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.381+0000 7f7487200640 1 Processor -- start 2024-04-26T20:57:40.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.382+0000 7f7487200640 1 -- start start 2024-04-26T20:57:40.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.382+0000 7f7487200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7488073a00 0x7f7488158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:40.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.382+0000 7f7487200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74880cf180 0x7f74881f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:40.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.383+0000 7f7487200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f748815c4d0 0x7f74881f53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:40.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.383+0000 7f7487200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f74880cbae0 con 0x7f74880cf180 2024-04-26T20:57:40.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.383+0000 7f7487200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f74880cc1a0 con 0x7f748815c4d0 2024-04-26T20:57:40.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.383+0000 7f7487200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f74880cbc60 con 0x7f7488073a00 2024-04-26T20:57:40.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.383+0000 7f7485400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74880cf180 0x7f74881f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:40.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.383+0000 7f7485400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74880cf180 0x7f74881f4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51006/0 (socket says 172.21.15.28:51006) 2024-04-26T20:57:40.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.383+0000 7f7485400640 1 -- 172.21.15.28:0/2094227727 learned_addr learned my addr 172.21.15.28:0/2094227727 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:40.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.383+0000 7f7485e00640 1 --2- 172.21.15.28:0/2094227727 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7488073a00 0x7f7488158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:40.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.383+0000 7f7486800640 1 --2- 172.21.15.28:0/2094227727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f748815c4d0 0x7f74881f53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:40.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.383+0000 7f7485400640 1 -- 172.21.15.28:0/2094227727 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7488073a00 msgr2=0x7f7488158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:40.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.383+0000 7f7485400640 1 --2- 172.21.15.28:0/2094227727 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7488073a00 0x7f7488158dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.383+0000 7f7485400640 1 -- 172.21.15.28:0/2094227727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f748815c4d0 msgr2=0x7f74881f53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:40.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.383+0000 7f7485400640 1 --2- 172.21.15.28:0/2094227727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f748815c4d0 0x7f74881f53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.383+0000 7f7485400640 1 -- 172.21.15.28:0/2094227727 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7488153530 con 0x7f74880cf180 2024-04-26T20:57:40.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.384+0000 7f7485400640 1 --2- 172.21.15.28:0/2094227727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74880cf180 0x7f74881f4e60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f747402a2b0 tx=0x7f74740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:40.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.384+0000 7f747e000640 1 -- 172.21.15.28:0/2094227727 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474041020 con 0x7f74880cf180 2024-04-26T20:57:40.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.384+0000 7f747e000640 1 -- 172.21.15.28:0/2094227727 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74740306c0 con 0x7f74880cf180 2024-04-26T20:57:40.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.384+0000 7f747e000640 1 -- 172.21.15.28:0/2094227727 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7474037040 con 0x7f74880cf180 2024-04-26T20:57:40.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.384+0000 7f7487200640 1 -- 172.21.15.28:0/2094227727 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7488153810 con 0x7f74880cf180 2024-04-26T20:57:40.388 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.384+0000 7f7487200640 1 -- 172.21.15.28:0/2094227727 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7488153dd0 con 0x7f74880cf180 2024-04-26T20:57:40.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.385+0000 7f7487200640 1 -- 172.21.15.28:0/2094227727 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f748806a360 con 0x7f74880cf180 2024-04-26T20:57:40.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.386+0000 7f747e000640 1 -- 172.21.15.28:0/2094227727 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7474044020 con 0x7f74880cf180 2024-04-26T20:57:40.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.387+0000 7f747e000640 1 --2- 172.21.15.28:0/2094227727 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f746806c8f0 0x7f746806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:40.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.387+0000 7f747e000640 1 -- 172.21.15.28:0/2094227727 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f74740ac410 con 0x7f74880cf180 2024-04-26T20:57:40.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.388+0000 7f7485e00640 1 --2- 172.21.15.28:0/2094227727 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f746806c8f0 0x7f746806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:40.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.389+0000 7f7485e00640 1 --2- 172.21.15.28:0/2094227727 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f746806c8f0 0x7f746806edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7478006fd0 tx=0x7f7478008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:40.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.392+0000 7f747e000640 1 -- 172.21.15.28:0/2094227727 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f747404b030 con 0x7f74880cf180 2024-04-26T20:57:40.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.600+0000 7f7487200640 1 -- 172.21.15.28:0/2094227727 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7488073220 con 0x7f74880cf180 2024-04-26T20:57:40.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.602+0000 7f747e000640 1 -- 172.21.15.28:0/2094227727 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16831 (secure 0 0 0) 0x7f747407a7b0 con 0x7f74880cf180 2024-04-26T20:57:40.604 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:40.604 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":23,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:57:37.996297+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:57:40.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.605+0000 7f7487200640 1 -- 172.21.15.28:0/2094227727 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f746806c8f0 msgr2=0x7f746806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:40.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.605+0000 7f7487200640 1 --2- 172.21.15.28:0/2094227727 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f746806c8f0 0x7f746806edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7478006fd0 tx=0x7f7478008040 comp rx=0 tx=0).stop 2024-04-26T20:57:40.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.605+0000 7f7487200640 1 -- 172.21.15.28:0/2094227727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74880cf180 msgr2=0x7f74881f4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:40.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.605+0000 7f7487200640 1 --2- 172.21.15.28:0/2094227727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74880cf180 0x7f74881f4e60 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f747402a2b0 tx=0x7f74740046c0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.605+0000 7f7487200640 1 -- 172.21.15.28:0/2094227727 shutdown_connections 2024-04-26T20:57:40.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.605+0000 7f7487200640 1 --2- 172.21.15.28:0/2094227727 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f746806c8f0 0x7f746806edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.605+0000 7f7487200640 1 --2- 172.21.15.28:0/2094227727 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f748815c4d0 0x7f74881f53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.605+0000 7f7487200640 1 --2- 172.21.15.28:0/2094227727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f74880cf180 0x7f74881f4e60 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.605+0000 7f7487200640 1 --2- 172.21.15.28:0/2094227727 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7488073a00 0x7f7488158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.605+0000 7f7487200640 1 -- 172.21.15.28:0/2094227727 >> 172.21.15.28:0/2094227727 conn(0x7f74880b2dd0 msgr2=0x7f748806e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:40.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.605+0000 7f7487200640 1 -- 172.21.15.28:0/2094227727 shutdown_connections 2024-04-26T20:57:40.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.605+0000 7f7487200640 1 -- 172.21.15.28:0/2094227727 wait complete. 2024-04-26T20:57:40.625 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:56:48.688498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:56:50.117541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:57:40.626 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:57:40.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.837+0000 7f3720800640 1 Processor -- start 2024-04-26T20:57:40.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.837+0000 7f3720800640 1 -- start start 2024-04-26T20:57:40.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.839+0000 7f3720800640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3718161040 0x7f37180b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:40.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.839+0000 7f3720800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37181644e0 0x7f3718072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:40.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.839+0000 7f3720800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3718073500 0x7f3718167050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:40.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.839+0000 7f3720800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f3718073960 con 0x7f3718073500 2024-04-26T20:57:40.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.839+0000 7f3720800640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f37180b5090 con 0x7f3718161040 2024-04-26T20:57:40.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.839+0000 7f3720800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f37180b5370 con 0x7f37181644e0 2024-04-26T20:57:40.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.839+0000 7f371e800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3718073500 0x7f3718167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:40.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.839+0000 7f371e800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3718073500 0x7f3718167050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51018/0 (socket says 172.21.15.28:51018) 2024-04-26T20:57:40.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.839+0000 7f371e800640 1 -- 172.21.15.28:0/529690095 learned_addr learned my addr 172.21.15.28:0/529690095 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:40.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.840+0000 7f371d400640 1 --2- 172.21.15.28:0/529690095 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37181644e0 0x7f3718072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:40.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.840+0000 7f371de00640 1 --2- 172.21.15.28:0/529690095 >> v2:172.21.15.103:3300/0 conn(0x7f3718161040 0x7f37180b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:40.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.840+0000 7f371e800640 1 -- 172.21.15.28:0/529690095 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37181644e0 msgr2=0x7f3718072fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:40.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.840+0000 7f371e800640 1 --2- 172.21.15.28:0/529690095 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37181644e0 0x7f3718072fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.840+0000 7f371e800640 1 -- 172.21.15.28:0/529690095 >> v2:172.21.15.103:3300/0 conn(0x7f3718161040 msgr2=0x7f37180b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:40.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.840+0000 7f371e800640 1 --2- 172.21.15.28:0/529690095 >> v2:172.21.15.103:3300/0 conn(0x7f3718161040 0x7f37180b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.840+0000 7f371e800640 1 -- 172.21.15.28:0/529690095 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37180b5650 con 0x7f3718073500 2024-04-26T20:57:40.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.841+0000 7f371e800640 1 --2- 172.21.15.28:0/529690095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3718073500 0x7f3718167050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f37180704c0 tx=0x7f371402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1cf460d1702c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:40.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.842+0000 7f371ca00640 1 -- 172.21.15.28:0/529690095 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714036070 con 0x7f3718073500 2024-04-26T20:57:40.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.842+0000 7f371ca00640 1 -- 172.21.15.28:0/529690095 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3714031600 con 0x7f3718073500 2024-04-26T20:57:40.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.842+0000 7f371ca00640 1 -- 172.21.15.28:0/529690095 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37140319d0 con 0x7f3718073500 2024-04-26T20:57:40.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.842+0000 7f3720800640 1 -- 172.21.15.28:0/529690095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3718073500 msgr2=0x7f3718167050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:40.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.842+0000 7f3720800640 1 --2- 172.21.15.28:0/529690095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3718073500 0x7f3718167050 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f37180704c0 tx=0x7f371402a6a0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.843+0000 7f3720800640 1 -- 172.21.15.28:0/529690095 shutdown_connections 2024-04-26T20:57:40.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.843+0000 7f3720800640 1 --2- 172.21.15.28:0/529690095 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3718073500 0x7f3718167050 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.843+0000 7f3720800640 1 --2- 172.21.15.28:0/529690095 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37181644e0 0x7f3718072fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.843+0000 7f3720800640 1 --2- 172.21.15.28:0/529690095 >> v2:172.21.15.103:3300/0 conn(0x7f3718161040 0x7f37180b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.843+0000 7f3720800640 1 -- 172.21.15.28:0/529690095 >> 172.21.15.28:0/529690095 conn(0x7f371806c770 msgr2=0x7f371806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:40.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.843+0000 7f3720800640 1 -- 172.21.15.28:0/529690095 shutdown_connections 2024-04-26T20:57:40.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.843+0000 7f3720800640 1 -- 172.21.15.28:0/529690095 wait complete. 2024-04-26T20:57:40.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.845+0000 7f3720800640 1 Processor -- start 2024-04-26T20:57:40.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.845+0000 7f3720800640 1 -- start start 2024-04-26T20:57:40.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.846+0000 7f3720800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3718073500 0x7f37180af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:40.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.846+0000 7f3720800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3718161040 0x7f37180aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:40.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.846+0000 7f3720800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37181644e0 0x7f37180b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:40.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.846+0000 7f3720800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3718161420 con 0x7f3718073500 2024-04-26T20:57:40.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.846+0000 7f3720800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37180b5c10 con 0x7f3718161040 2024-04-26T20:57:40.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.846+0000 7f3720800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f3718164940 con 0x7f37181644e0 2024-04-26T20:57:40.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.847+0000 7f371de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3718073500 0x7f37180af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:40.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.847+0000 7f371de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3718073500 0x7f37180af930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51032/0 (socket says 172.21.15.28:51032) 2024-04-26T20:57:40.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.847+0000 7f371de00640 1 -- 172.21.15.28:0/2448963374 learned_addr learned my addr 172.21.15.28:0/2448963374 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:40.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.847+0000 7f371d400640 1 --2- 172.21.15.28:0/2448963374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3718161040 0x7f37180aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:40.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.847+0000 7f371e800640 1 --2- 172.21.15.28:0/2448963374 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37181644e0 0x7f37180b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:40.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.847+0000 7f371de00640 1 -- 172.21.15.28:0/2448963374 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37181644e0 msgr2=0x7f37180b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:40.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.847+0000 7f371de00640 1 --2- 172.21.15.28:0/2448963374 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37181644e0 0x7f37180b09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.847+0000 7f371de00640 1 -- 172.21.15.28:0/2448963374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3718161040 msgr2=0x7f37180aa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:40.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.847+0000 7f371de00640 1 --2- 172.21.15.28:0/2448963374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3718161040 0x7f37180aa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:40.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.847+0000 7f371de00640 1 -- 172.21.15.28:0/2448963374 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37181f9df0 con 0x7f3718073500 2024-04-26T20:57:40.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.848+0000 7f371de00640 1 --2- 172.21.15.28:0/2448963374 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3718073500 0x7f37180af930 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f370c00d800 tx=0x7f370c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:40.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.848+0000 7f3712000640 1 -- 172.21.15.28:0/2448963374 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c018020 con 0x7f3718073500 2024-04-26T20:57:40.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.848+0000 7f3712000640 1 -- 172.21.15.28:0/2448963374 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f370c0096a0 con 0x7f3718073500 2024-04-26T20:57:40.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.848+0000 7f3712000640 1 -- 172.21.15.28:0/2448963374 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f370c009a70 con 0x7f3718073500 2024-04-26T20:57:40.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.848+0000 7f3720800640 1 -- 172.21.15.28:0/2448963374 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37181fa0d0 con 0x7f3718073500 2024-04-26T20:57:40.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.848+0000 7f3720800640 1 -- 172.21.15.28:0/2448963374 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37181faac0 con 0x7f3718073500 2024-04-26T20:57:40.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.850+0000 7f3720800640 1 -- 172.21.15.28:0/2448963374 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f371806a360 con 0x7f3718073500 2024-04-26T20:57:40.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.851+0000 7f3712000640 1 -- 172.21.15.28:0/2448963374 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f370c01b020 con 0x7f3718073500 2024-04-26T20:57:40.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.851+0000 7f3712000640 1 --2- 172.21.15.28:0/2448963374 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f36fc06c8f0 0x7f36fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:40.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.852+0000 7f3712000640 1 -- 172.21.15.28:0/2448963374 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f370c08be60 con 0x7f3718073500 2024-04-26T20:57:40.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.852+0000 7f371d400640 1 --2- 172.21.15.28:0/2448963374 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f36fc06c8f0 0x7f36fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:40.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.853+0000 7f371d400640 1 --2- 172.21.15.28:0/2448963374 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f36fc06c8f0 0x7f36fc06edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3708006fd0 tx=0x7f3708008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:40.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:40.856+0000 7f3712000640 1 -- 172.21.15.28:0/2448963374 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f370c022030 con 0x7f3718073500 2024-04-26T20:57:41.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.063+0000 7f3720800640 1 -- 172.21.15.28:0/2448963374 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f37180b4c10 con 0x7f3718073500 2024-04-26T20:57:41.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.064+0000 7f3712000640 1 -- 172.21.15.28:0/2448963374 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f370c05a210 con 0x7f3718073500 2024-04-26T20:57:41.065 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:57:41.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.067+0000 7f3720800640 1 -- 172.21.15.28:0/2448963374 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f36fc06c8f0 msgr2=0x7f36fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:41.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.067+0000 7f3720800640 1 --2- 172.21.15.28:0/2448963374 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f36fc06c8f0 0x7f36fc06edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3708006fd0 tx=0x7f3708008040 comp rx=0 tx=0).stop 2024-04-26T20:57:41.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.067+0000 7f3720800640 1 -- 172.21.15.28:0/2448963374 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3718073500 msgr2=0x7f37180af930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:41.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.067+0000 7f3720800640 1 --2- 172.21.15.28:0/2448963374 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3718073500 0x7f37180af930 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f370c00d800 tx=0x7f370c00dcc0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.067+0000 7f3720800640 1 -- 172.21.15.28:0/2448963374 shutdown_connections 2024-04-26T20:57:41.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.067+0000 7f3720800640 1 --2- 172.21.15.28:0/2448963374 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f36fc06c8f0 0x7f36fc06edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.067+0000 7f3720800640 1 --2- 172.21.15.28:0/2448963374 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37181644e0 0x7f37180b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.067+0000 7f3720800640 1 --2- 172.21.15.28:0/2448963374 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3718161040 0x7f37180aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.067+0000 7f3720800640 1 --2- 172.21.15.28:0/2448963374 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3718073500 0x7f37180af930 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.067+0000 7f3720800640 1 -- 172.21.15.28:0/2448963374 >> 172.21.15.28:0/2448963374 conn(0x7f371806c770 msgr2=0x7f37180b1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:41.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.067+0000 7f3720800640 1 -- 172.21.15.28:0/2448963374 shutdown_connections 2024-04-26T20:57:41.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.068+0000 7f3720800640 1 -- 172.21.15.28:0/2448963374 wait complete. 2024-04-26T20:57:41.088 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:57:41.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.297+0000 7efcdda00640 1 Processor -- start 2024-04-26T20:57:41.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.297+0000 7efcdda00640 1 -- start start 2024-04-26T20:57:41.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.299+0000 7efcdda00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7efcd80d1420 0x7efcd806e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:41.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.299+0000 7efcdda00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcd80b3ea0 con 0x7efcd815efd0 2024-04-26T20:57:41.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.299+0000 7efcdda00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efcd80b4180 con 0x7efcd80b4f40 2024-04-26T20:57:41.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.299+0000 7efcdda00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7efcd80b4460 con 0x7efcd80d1420 2024-04-26T20:57:41.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.300+0000 7efcd7e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7efcd815efd0 0x7efcd80d0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58414/0 (socket says 172.21.15.28:58414) 2024-04-26T20:57:41.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.300+0000 7efcd7e00640 1 -- 172.21.15.28:0/270835284 learned_addr learned my addr 172.21.15.28:0/270835284 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:41.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.300+0000 7efccf400640 1 --2- 172.21.15.28:0/270835284 >> v2:172.21.15.103:3300/0 conn(0x7efcd80d1420 0x7efcd806e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:41.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.300+0000 7efcd7400640 1 -- 172.21.15.28:0/270835284 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 31434694 0 0) 0x7efcd80b3ea0 con 0x7efcd815efd0 2024-04-26T20:57:41.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.300+0000 7efcd7400640 1 -- 172.21.15.28:0/270835284 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efcbc003520 con 0x7efcd815efd0 2024-04-26T20:57:41.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.300+0000 7efcd7400640 1 -- 172.21.15.28:0/270835284 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4010274662 0 0) 0x7efcbc003520 con 0x7efcd815efd0 2024-04-26T20:57:41.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.300+0000 7efcd7400640 1 -- 172.21.15.28:0/270835284 >> v2:172.21.15.103:3300/0 conn(0x7efcd80d1420 msgr2=0x7efcd806e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:41.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.300+0000 7efcd7400640 1 --2- 172.21.15.28:0/270835284 >> v2:172.21.15.103:3300/0 conn(0x7efcd80d1420 0x7efcd806e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.301+0000 7efcd7400640 1 -- 172.21.15.28:0/270835284 >> v1:172.21.15.103:6789/0 conn(0x7efcd80b4f40 legacy=0x7efcd806e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:41.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.301+0000 7efcd7400640 1 -- 172.21.15.28:0/270835284 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd80b4740 con 0x7efcd815efd0 2024-04-26T20:57:41.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.301+0000 7efcd7400640 1 -- 172.21.15.28:0/270835284 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7efcc8001670 con 0x7efcd815efd0 2024-04-26T20:57:41.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.301+0000 7efcd7400640 1 -- 172.21.15.28:0/270835284 >> v1:172.21.15.28:6789/0 conn(0x7efcd815efd0 legacy=0x7efcd80d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:41.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.301+0000 7efcd7400640 1 --2- 172.21.15.28:0/270835284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcbc003a00 0x7efcbc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:41.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.301+0000 7efcd7400640 1 --2- 172.21.15.28:0/270835284 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcbc004800 0x7efcbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:41.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.301+0000 7efcd7400640 1 --2- 172.21.15.28:0/270835284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcbc0051c0 0x7efcbc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:41.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.301+0000 7efcd7400640 1 -- 172.21.15.28:0/270835284 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efcd80b4460 con 0x7efcbc0051c0 2024-04-26T20:57:41.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.301+0000 7efcd7400640 1 -- 172.21.15.28:0/270835284 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efcbc00c0c0 con 0x7efcbc003a00 2024-04-26T20:57:41.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.301+0000 7efcd7400640 1 -- 172.21.15.28:0/270835284 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7efcbc00c2c0 con 0x7efcbc004800 2024-04-26T20:57:41.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.301+0000 7efcdd000640 1 --2- 172.21.15.28:0/270835284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcbc0051c0 0x7efcbc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:41.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.301+0000 7efcd7e00640 1 --2- 172.21.15.28:0/270835284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcbc003a00 0x7efcbc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:41.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.301+0000 7efcdd000640 1 -- 172.21.15.28:0/270835284 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcbc004800 msgr2=0x7efcbc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:41.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.301+0000 7efcdd000640 1 --2- 172.21.15.28:0/270835284 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcbc004800 0x7efcbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.301+0000 7efcdd000640 1 -- 172.21.15.28:0/270835284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcbc003a00 msgr2=0x7efcbc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:41.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.301+0000 7efcdd000640 1 --2- 172.21.15.28:0/270835284 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcbc003a00 0x7efcbc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.301+0000 7efcdd000640 1 -- 172.21.15.28:0/270835284 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcbc003520 con 0x7efcbc0051c0 2024-04-26T20:57:41.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.302+0000 7efcdd000640 1 -- 172.21.15.28:0/270835284 reap_dead start 2024-04-26T20:57:41.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.303+0000 7efcdd000640 1 --2- 172.21.15.28:0/270835284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcbc0051c0 0x7efcbc00b9c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efcd0001e50 tx=0x7efcbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8700d05670dad287 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:41.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.303+0000 7efcd7400640 1 -- 172.21.15.28:0/270835284 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0001a70 con 0x7efcbc0051c0 2024-04-26T20:57:41.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.303+0000 7efcd7400640 1 -- 172.21.15.28:0/270835284 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcd0003970 con 0x7efcbc0051c0 2024-04-26T20:57:41.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.303+0000 7efcd7400640 1 -- 172.21.15.28:0/270835284 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0003d60 con 0x7efcbc0051c0 2024-04-26T20:57:41.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.303+0000 7efcdda00640 1 -- 172.21.15.28:0/270835284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcbc0051c0 msgr2=0x7efcbc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:41.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.303+0000 7efcdda00640 1 --2- 172.21.15.28:0/270835284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcbc0051c0 0x7efcbc00b9c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7efcd0001e50 tx=0x7efcbc005830 comp rx=0 tx=0).stop 2024-04-26T20:57:41.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.303+0000 7efcdda00640 1 -- 172.21.15.28:0/270835284 shutdown_connections 2024-04-26T20:57:41.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.303+0000 7efcdda00640 1 --2- 172.21.15.28:0/270835284 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcbc0051c0 0x7efcbc00b9c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.303+0000 7efcdda00640 1 -- 172.21.15.28:0/270835284 >> 172.21.15.28:0/270835284 conn(0x7efcd806c530 msgr2=0x7efcd806c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:41.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.304+0000 7efcdda00640 1 -- 172.21.15.28:0/270835284 shutdown_connections 2024-04-26T20:57:41.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.304+0000 7efcdda00640 1 -- 172.21.15.28:0/270835284 wait complete. 2024-04-26T20:57:41.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.305+0000 7efcdda00640 1 Processor -- start 2024-04-26T20:57:41.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.306+0000 7efcdda00640 1 -- start start 2024-04-26T20:57:41.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.306+0000 7efcdda00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcbc0051c0 0x7efcd8149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:41.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.306+0000 7efcdda00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcd80cf170 0x7efcd80cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:41.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.307+0000 7efcdda00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcd814c0f0 0x7efcd814c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:41.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.307+0000 7efcdda00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efcd80b56c0 con 0x7efcbc0051c0 2024-04-26T20:57:41.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.307+0000 7efcdda00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efcd815f3b0 con 0x7efcd814c0f0 2024-04-26T20:57:41.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.307+0000 7efcdda00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7efcd80b5840 con 0x7efcd80cf170 2024-04-26T20:57:41.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.307+0000 7efcd7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcbc0051c0 0x7efcd8149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:41.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.307+0000 7efcd7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcbc0051c0 0x7efcd8149530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51050/0 (socket says 172.21.15.28:51050) 2024-04-26T20:57:41.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.307+0000 7efcd7e00640 1 -- 172.21.15.28:0/3258575975 learned_addr learned my addr 172.21.15.28:0/3258575975 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:41.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.307+0000 7efccf400640 1 --2- 172.21.15.28:0/3258575975 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcd80cf170 0x7efcd80cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:41.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.307+0000 7efcdd000640 1 --2- 172.21.15.28:0/3258575975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcd814c0f0 0x7efcd814c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:41.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.307+0000 7efcd7e00640 1 -- 172.21.15.28:0/3258575975 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcd80cf170 msgr2=0x7efcd80cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:41.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.307+0000 7efcd7e00640 1 --2- 172.21.15.28:0/3258575975 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcd80cf170 0x7efcd80cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.307+0000 7efcd7e00640 1 -- 172.21.15.28:0/3258575975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcd814c0f0 msgr2=0x7efcd814c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:41.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.307+0000 7efcd7e00640 1 --2- 172.21.15.28:0/3258575975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcd814c0f0 0x7efcd814c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.307+0000 7efcd7e00640 1 -- 172.21.15.28:0/3258575975 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd814cfd0 con 0x7efcbc0051c0 2024-04-26T20:57:41.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.307+0000 7efcdd000640 1 --2- 172.21.15.28:0/3258575975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcd814c0f0 0x7efcd814c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:41.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.308+0000 7efcd7e00640 1 --2- 172.21.15.28:0/3258575975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcbc0051c0 0x7efcd8149530 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efcc8001050 tx=0x7efcc800eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:41.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.308+0000 7efcd4c00640 1 -- 172.21.15.28:0/3258575975 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc800c660 con 0x7efcbc0051c0 2024-04-26T20:57:41.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.308+0000 7efcd4c00640 1 -- 172.21.15.28:0/3258575975 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcc8018070 con 0x7efcbc0051c0 2024-04-26T20:57:41.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.308+0000 7efcd4c00640 1 -- 172.21.15.28:0/3258575975 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc800ca40 con 0x7efcbc0051c0 2024-04-26T20:57:41.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.308+0000 7efcdda00640 1 -- 172.21.15.28:0/3258575975 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd80cf710 con 0x7efcbc0051c0 2024-04-26T20:57:41.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.308+0000 7efcdda00640 1 -- 172.21.15.28:0/3258575975 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd80cfcd0 con 0x7efcbc0051c0 2024-04-26T20:57:41.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.310+0000 7efcdda00640 1 -- 172.21.15.28:0/3258575975 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd806f270 con 0x7efcbc0051c0 2024-04-26T20:57:41.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.311+0000 7efcd4c00640 1 -- 172.21.15.28:0/3258575975 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efcc800cbe0 con 0x7efcbc0051c0 2024-04-26T20:57:41.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.312+0000 7efcd4c00640 1 --2- 172.21.15.28:0/3258575975 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efcb406c920 0x7efcb406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:41.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.312+0000 7efcd4c00640 1 -- 172.21.15.28:0/3258575975 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7efcc808b9f0 con 0x7efcbc0051c0 2024-04-26T20:57:41.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.312+0000 7efccf400640 1 --2- 172.21.15.28:0/3258575975 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efcb406c920 0x7efcb406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:41.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.313+0000 7efccf400640 1 --2- 172.21.15.28:0/3258575975 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efcb406c920 0x7efcb406ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efcd8162350 tx=0x7efcc0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:41.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.316+0000 7efcd4c00640 1 -- 172.21.15.28:0/3258575975 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efcd806f270 con 0x7efcbc0051c0 2024-04-26T20:57:41.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.520+0000 7efcdda00640 1 -- 172.21.15.28:0/3258575975 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efcd806a250 con 0x7efcbc0051c0 2024-04-26T20:57:41.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.521+0000 7efcd4c00640 1 -- 172.21.15.28:0/3258575975 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7efcd814cfd0 con 0x7efcbc0051c0 2024-04-26T20:57:41.522 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-04-26T20:57:41.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.523+0000 7efcdda00640 1 -- 172.21.15.28:0/3258575975 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efcb406c920 msgr2=0x7efcb406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:41.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.523+0000 7efcdda00640 1 --2- 172.21.15.28:0/3258575975 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efcb406c920 0x7efcb406ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efcd8162350 tx=0x7efcc0008040 comp rx=0 tx=0).stop 2024-04-26T20:57:41.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.524+0000 7efcdda00640 1 -- 172.21.15.28:0/3258575975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcbc0051c0 msgr2=0x7efcd8149530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:41.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.524+0000 7efcdda00640 1 --2- 172.21.15.28:0/3258575975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcbc0051c0 0x7efcd8149530 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efcc8001050 tx=0x7efcc800eb50 comp rx=0 tx=0).stop 2024-04-26T20:57:41.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.524+0000 7efcdda00640 1 -- 172.21.15.28:0/3258575975 shutdown_connections 2024-04-26T20:57:41.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.524+0000 7efcdda00640 1 --2- 172.21.15.28:0/3258575975 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efcb406c920 0x7efcb406ede0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.524+0000 7efcdda00640 1 --2- 172.21.15.28:0/3258575975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efcd814c0f0 0x7efcd814c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.524+0000 7efcdda00640 1 --2- 172.21.15.28:0/3258575975 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efcd80cf170 0x7efcd80cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.524+0000 7efcdda00640 1 --2- 172.21.15.28:0/3258575975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcbc0051c0 0x7efcd8149530 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.524+0000 7efcdda00640 1 -- 172.21.15.28:0/3258575975 >> 172.21.15.28:0/3258575975 conn(0x7efcd806c530 msgr2=0x7efcd80b2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:41.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.524+0000 7efcdda00640 1 -- 172.21.15.28:0/3258575975 shutdown_connections 2024-04-26T20:57:41.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.524+0000 7efcdda00640 1 -- 172.21.15.28:0/3258575975 wait complete. 2024-04-26T20:57:41.544 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:57:41.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.759+0000 7f7c62400640 1 Processor -- start 2024-04-26T20:57:41.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.759+0000 7f7c62400640 1 -- start start 2024-04-26T20:57:41.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.761+0000 7f7c62400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c5c14cfe0 0x7f7c5c0b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:41.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.761+0000 7f7c62400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7c5c0b52d0 0x7f7c5c0ce570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:41.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.761+0000 7f7c62400640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c5c14d3c0 con 0x7f7c5c0ceab0 2024-04-26T20:57:41.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.761+0000 7f7c62400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f7c5c0b4f30 con 0x7f7c5c0b52d0 2024-04-26T20:57:41.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.761+0000 7f7c62400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7c5c0b4690 con 0x7f7c5c14cfe0 2024-04-26T20:57:41.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.762+0000 7f7c53e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7c5c0b52d0 0x7f7c5c0ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:41.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.762+0000 7f7c53e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7c5c0b52d0 0x7f7c5c0ce570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51058/0 (socket says 172.21.15.28:51058) 2024-04-26T20:57:41.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.762+0000 7f7c53e00640 1 -- 172.21.15.28:0/1996949281 learned_addr learned my addr 172.21.15.28:0/1996949281 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:41.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.762+0000 7f7c5b400640 1 --2- 172.21.15.28:0/1996949281 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c5c14cfe0 0x7f7c5c0b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:41.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.762+0000 7f7c53e00640 1 -- 172.21.15.28:0/1996949281 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c5c14cfe0 msgr2=0x7f7c5c0b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:41.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.762+0000 7f7c53e00640 1 --2- 172.21.15.28:0/1996949281 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c5c14cfe0 0x7f7c5c0b3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.762+0000 7f7c53e00640 1 -- 172.21.15.28:0/1996949281 >> v1:172.21.15.103:6789/0 conn(0x7f7c5c0ceab0 legacy=0x7f7c5c06f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:41.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.762+0000 7f7c53e00640 1 -- 172.21.15.28:0/1996949281 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5c0b4970 con 0x7f7c5c0b52d0 2024-04-26T20:57:41.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.763+0000 7f7c53e00640 1 --2- 172.21.15.28:0/1996949281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c5c0b52d0 0x7f7c5c0ce570 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009870 tx=0x7f7c4402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1db947ea6adb23 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:41.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.763+0000 7f7c5aa00640 1 -- 172.21.15.28:0/1996949281 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44037070 con 0x7f7c5c0b52d0 2024-04-26T20:57:41.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.764+0000 7f7c5aa00640 1 -- 172.21.15.28:0/1996949281 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c44032440 con 0x7f7c5c0b52d0 2024-04-26T20:57:41.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.764+0000 7f7c5aa00640 1 -- 172.21.15.28:0/1996949281 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44032810 con 0x7f7c5c0b52d0 2024-04-26T20:57:41.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.764+0000 7f7c62400640 1 -- 172.21.15.28:0/1996949281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c5c0b52d0 msgr2=0x7f7c5c0ce570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:41.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.764+0000 7f7c62400640 1 --2- 172.21.15.28:0/1996949281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c5c0b52d0 0x7f7c5c0ce570 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009870 tx=0x7f7c4402a210 comp rx=0 tx=0).stop 2024-04-26T20:57:41.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.764+0000 7f7c62400640 1 -- 172.21.15.28:0/1996949281 shutdown_connections 2024-04-26T20:57:41.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.764+0000 7f7c62400640 1 --2- 172.21.15.28:0/1996949281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c5c0b52d0 0x7f7c5c0ce570 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.764+0000 7f7c62400640 1 --2- 172.21.15.28:0/1996949281 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c5c14cfe0 0x7f7c5c0b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.764+0000 7f7c62400640 1 -- 172.21.15.28:0/1996949281 >> 172.21.15.28:0/1996949281 conn(0x7f7c5c0b2b60 msgr2=0x7f7c5c0b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:41.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.764+0000 7f7c62400640 1 -- 172.21.15.28:0/1996949281 shutdown_connections 2024-04-26T20:57:41.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.764+0000 7f7c62400640 1 -- 172.21.15.28:0/1996949281 wait complete. 2024-04-26T20:57:41.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.766+0000 7f7c62400640 1 Processor -- start 2024-04-26T20:57:41.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.766+0000 7f7c62400640 1 -- start start 2024-04-26T20:57:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.767+0000 7f7c62400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c5c0b52d0 0x7f7c5c13ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.767+0000 7f7c62400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c5c0ceab0 0x7f7c5c166ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.767+0000 7f7c62400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c5c14cfe0 0x7f7c5c167410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.767+0000 7f7c62400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c06f9e0 con 0x7f7c5c0ceab0 2024-04-26T20:57:41.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.767+0000 7f7c62400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7c5c0b43a0 con 0x7f7c5c0b52d0 2024-04-26T20:57:41.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.767+0000 7f7c62400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7c5c06fb60 con 0x7f7c5c14cfe0 2024-04-26T20:57:41.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.767+0000 7f7c53e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c5c0ceab0 0x7f7c5c166ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:41.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.767+0000 7f7c53e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c5c0ceab0 0x7f7c5c166ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51064/0 (socket says 172.21.15.28:51064) 2024-04-26T20:57:41.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.767+0000 7f7c53e00640 1 -- 172.21.15.28:0/1166223457 learned_addr learned my addr 172.21.15.28:0/1166223457 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:41.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.768+0000 7f7c53e00640 1 -- 172.21.15.28:0/1166223457 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c5c14cfe0 msgr2=0x7f7c5c167410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:41.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.768+0000 7f7c53e00640 1 --2- 172.21.15.28:0/1166223457 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c5c14cfe0 0x7f7c5c167410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.768+0000 7f7c53e00640 1 -- 172.21.15.28:0/1166223457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c5c0b52d0 msgr2=0x7f7c5c13ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:41.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.768+0000 7f7c53e00640 1 --2- 172.21.15.28:0/1166223457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c5c0b52d0 0x7f7c5c13ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.768+0000 7f7c53e00640 1 -- 172.21.15.28:0/1166223457 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5c1385b0 con 0x7f7c5c0ceab0 2024-04-26T20:57:41.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.768+0000 7f7c53e00640 1 --2- 172.21.15.28:0/1166223457 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c5c0ceab0 0x7f7c5c166ed0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009870 tx=0x7f7c440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:41.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.768+0000 7f7c53400640 1 -- 172.21.15.28:0/1166223457 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44041020 con 0x7f7c5c0ceab0 2024-04-26T20:57:41.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.768+0000 7f7c53400640 1 -- 172.21.15.28:0/1166223457 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c440306c0 con 0x7f7c5c0ceab0 2024-04-26T20:57:41.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.768+0000 7f7c53400640 1 -- 172.21.15.28:0/1166223457 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c44037040 con 0x7f7c5c0ceab0 2024-04-26T20:57:41.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.768+0000 7f7c62400640 1 -- 172.21.15.28:0/1166223457 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5c1646a0 con 0x7f7c5c0ceab0 2024-04-26T20:57:41.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.769+0000 7f7c62400640 1 -- 172.21.15.28:0/1166223457 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c5c164c60 con 0x7f7c5c0ceab0 2024-04-26T20:57:41.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.770+0000 7f7c62400640 1 -- 172.21.15.28:0/1166223457 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c5c076520 con 0x7f7c5c0ceab0 2024-04-26T20:57:41.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.771+0000 7f7c53400640 1 -- 172.21.15.28:0/1166223457 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c44044020 con 0x7f7c5c0ceab0 2024-04-26T20:57:41.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.772+0000 7f7c53400640 1 --2- 172.21.15.28:0/1166223457 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7c3806c920 0x7f7c3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:41.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.772+0000 7f7c53400640 1 -- 172.21.15.28:0/1166223457 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f7c440ac430 con 0x7f7c5c0ceab0 2024-04-26T20:57:41.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.772+0000 7f7c5b400640 1 --2- 172.21.15.28:0/1166223457 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7c3806c920 0x7f7c3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:41.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.773+0000 7f7c5b400640 1 --2- 172.21.15.28:0/1166223457 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7c3806c920 0x7f7c3806ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c006fd0 tx=0x7f7c4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:41.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.777+0000 7f7c53400640 1 -- 172.21.15.28:0/1166223457 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7c4404b030 con 0x7f7c5c0ceab0 2024-04-26T20:57:41.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.983+0000 7f7c62400640 1 -- 172.21.15.28:0/1166223457 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7c5c0b50b0 con 0x7f7c5c0ceab0 2024-04-26T20:57:41.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.984+0000 7f7c53400640 1 -- 172.21.15.28:0/1166223457 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f7c5c1385b0 con 0x7f7c5c0ceab0 2024-04-26T20:57:41.985 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-04-26T20:57:41.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.987+0000 7f7c62400640 1 -- 172.21.15.28:0/1166223457 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7c3806c920 msgr2=0x7f7c3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:41.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.987+0000 7f7c62400640 1 --2- 172.21.15.28:0/1166223457 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7c3806c920 0x7f7c3806ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c006fd0 tx=0x7f7c4c008040 comp rx=0 tx=0).stop 2024-04-26T20:57:41.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.987+0000 7f7c62400640 1 -- 172.21.15.28:0/1166223457 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c5c0ceab0 msgr2=0x7f7c5c166ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:41.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.987+0000 7f7c62400640 1 --2- 172.21.15.28:0/1166223457 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c5c0ceab0 0x7f7c5c166ed0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7c44009870 tx=0x7f7c440046c0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.987+0000 7f7c62400640 1 -- 172.21.15.28:0/1166223457 shutdown_connections 2024-04-26T20:57:41.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.987+0000 7f7c62400640 1 --2- 172.21.15.28:0/1166223457 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7c3806c920 0x7f7c3806ede0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.987+0000 7f7c62400640 1 --2- 172.21.15.28:0/1166223457 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7c5c14cfe0 0x7f7c5c167410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.987+0000 7f7c62400640 1 --2- 172.21.15.28:0/1166223457 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7c5c0ceab0 0x7f7c5c166ed0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.987+0000 7f7c62400640 1 --2- 172.21.15.28:0/1166223457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7c5c0b52d0 0x7f7c5c13ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:41.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.987+0000 7f7c62400640 1 -- 172.21.15.28:0/1166223457 >> 172.21.15.28:0/1166223457 conn(0x7f7c5c0b2b60 msgr2=0x7f7c5c0b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:41.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.987+0000 7f7c62400640 1 -- 172.21.15.28:0/1166223457 shutdown_connections 2024-04-26T20:57:41.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:41.987+0000 7f7c62400640 1 -- 172.21.15.28:0/1166223457 wait complete. 2024-04-26T20:57:42.008 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:57:42.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.217+0000 7feb9be00640 1 Processor -- start 2024-04-26T20:57:42.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.217+0000 7feb9be00640 1 -- start start 2024-04-26T20:57:42.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.219+0000 7feb9be00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb9c144b90 0x7feb9c0cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:42.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.219+0000 7feb9be00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7feb9c073800 0x7feb9c073c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:42.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.219+0000 7feb9be00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb9c072760 con 0x7feb9c145080 2024-04-26T20:57:42.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.219+0000 7feb9be00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7feb9c072a40 con 0x7feb9c073800 2024-04-26T20:57:42.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.219+0000 7feb9be00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7feb9c072d20 con 0x7feb9c144b90 2024-04-26T20:57:42.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.220+0000 7feb9aa00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb9c144b90 0x7feb9c0cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:42.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.220+0000 7feb9b400640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7feb9c073800 0x7feb9c073c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:42.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.220+0000 7feb9b400640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7feb9c073800 0x7feb9c073c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.28:33458/0 (socket says 172.21.15.28:33458) 2024-04-26T20:57:42.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.220+0000 7feb9aa00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb9c144b90 0x7feb9c0cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.28:42308/0 (socket says 172.21.15.28:42308) 2024-04-26T20:57:42.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.220+0000 7feb9b400640 1 -- 172.21.15.28:0/2179211965 learned_addr learned my addr 172.21.15.28:0/2179211965 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:42.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.220+0000 7feb9b400640 1 -- 172.21.15.28:0/2179211965 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb9c144b90 msgr2=0x7feb9c0cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:42.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.220+0000 7feb9b400640 1 --2- 172.21.15.28:0/2179211965 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb9c144b90 0x7feb9c0cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:42.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.220+0000 7feb9b400640 1 -- 172.21.15.28:0/2179211965 >> v1:172.21.15.103:6789/0 conn(0x7feb9c145080 legacy=0x7feb9c0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:42.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.220+0000 7feb9b400640 1 -- 172.21.15.28:0/2179211965 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9c073000 con 0x7feb9c073800 2024-04-26T20:57:42.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.222+0000 7feb9b400640 1 --2- 172.21.15.28:0/2179211965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb9c073800 0x7feb9c073c80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feb94009e50 tx=0x7feb9402a630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45f326c2a5200a12 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:42.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.222+0000 7feb99600640 1 -- 172.21.15.28:0/2179211965 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94036070 con 0x7feb9c073800 2024-04-26T20:57:42.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.222+0000 7feb99600640 1 -- 172.21.15.28:0/2179211965 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb94031600 con 0x7feb9c073800 2024-04-26T20:57:42.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.222+0000 7feb99600640 1 -- 172.21.15.28:0/2179211965 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb940319d0 con 0x7feb9c073800 2024-04-26T20:57:42.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.223+0000 7feb9be00640 1 -- 172.21.15.28:0/2179211965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb9c073800 msgr2=0x7feb9c073c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:42.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.223+0000 7feb9be00640 1 --2- 172.21.15.28:0/2179211965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb9c073800 0x7feb9c073c80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7feb94009e50 tx=0x7feb9402a630 comp rx=0 tx=0).stop 2024-04-26T20:57:42.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.223+0000 7feb9be00640 1 -- 172.21.15.28:0/2179211965 shutdown_connections 2024-04-26T20:57:42.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.223+0000 7feb9be00640 1 --2- 172.21.15.28:0/2179211965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb9c073800 0x7feb9c073c80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:42.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.223+0000 7feb9be00640 1 --2- 172.21.15.28:0/2179211965 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb9c144b90 0x7feb9c0cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:42.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.223+0000 7feb9be00640 1 -- 172.21.15.28:0/2179211965 >> 172.21.15.28:0/2179211965 conn(0x7feb9c0d0a50 msgr2=0x7feb9c0d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:42.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.223+0000 7feb9be00640 1 -- 172.21.15.28:0/2179211965 shutdown_connections 2024-04-26T20:57:42.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.223+0000 7feb9be00640 1 -- 172.21.15.28:0/2179211965 wait complete. 2024-04-26T20:57:42.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.226+0000 7feb9be00640 1 Processor -- start 2024-04-26T20:57:42.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.226+0000 7feb9be00640 1 -- start start 2024-04-26T20:57:42.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.227+0000 7feb9be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb9c073800 0x7feb9c15fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:42.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.227+0000 7feb9be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb9c144b90 0x7feb9c15dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:42.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.227+0000 7feb9be00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb9c145080 0x7feb9c142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:42.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.227+0000 7feb9be00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c0ced30 con 0x7feb9c073800 2024-04-26T20:57:42.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.227+0000 7feb9be00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c06f820 con 0x7feb9c144b90 2024-04-26T20:57:42.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.227+0000 7feb9be00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7feb9c15ee70 con 0x7feb9c145080 2024-04-26T20:57:42.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.227+0000 7feb9aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb9c073800 0x7feb9c15fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:42.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.227+0000 7feb9aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb9c073800 0x7feb9c15fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51066/0 (socket says 172.21.15.28:51066) 2024-04-26T20:57:42.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.227+0000 7feb9aa00640 1 -- 172.21.15.28:0/1312148247 learned_addr learned my addr 172.21.15.28:0/1312148247 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:42.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.227+0000 7feb9a000640 1 --2- 172.21.15.28:0/1312148247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb9c144b90 0x7feb9c15dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:42.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.228+0000 7feb9b400640 1 --2- 172.21.15.28:0/1312148247 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb9c145080 0x7feb9c142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:42.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.228+0000 7feb9aa00640 1 -- 172.21.15.28:0/1312148247 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb9c145080 msgr2=0x7feb9c142fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:42.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.228+0000 7feb9aa00640 1 --2- 172.21.15.28:0/1312148247 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb9c145080 0x7feb9c142fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:42.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.228+0000 7feb9aa00640 1 -- 172.21.15.28:0/1312148247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb9c144b90 msgr2=0x7feb9c15dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:42.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.228+0000 7feb9aa00640 1 --2- 172.21.15.28:0/1312148247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb9c144b90 0x7feb9c15dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:42.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.228+0000 7feb9aa00640 1 -- 172.21.15.28:0/1312148247 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9c0c7770 con 0x7feb9c073800 2024-04-26T20:57:42.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.228+0000 7feb9a000640 1 --2- 172.21.15.28:0/1312148247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb9c144b90 0x7feb9c15dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:42.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.228+0000 7feb9aa00640 1 --2- 172.21.15.28:0/1312148247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb9c073800 0x7feb9c15fc80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00e9e0 tx=0x7feb8c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:42.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.228+0000 7feb92a00640 1 -- 172.21.15.28:0/1312148247 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c017070 con 0x7feb9c073800 2024-04-26T20:57:42.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.228+0000 7feb92a00640 1 -- 172.21.15.28:0/1312148247 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb8c00c9d0 con 0x7feb9c073800 2024-04-26T20:57:42.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.228+0000 7feb9be00640 1 -- 172.21.15.28:0/1312148247 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9c15e5d0 con 0x7feb9c073800 2024-04-26T20:57:42.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.228+0000 7feb92a00640 1 -- 172.21.15.28:0/1312148247 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c00cdc0 con 0x7feb9c073800 2024-04-26T20:57:42.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.228+0000 7feb9be00640 1 -- 172.21.15.28:0/1312148247 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb9c15eb90 con 0x7feb9c073800 2024-04-26T20:57:42.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.229+0000 7feb9be00640 1 -- 172.21.15.28:0/1312148247 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb9c06a360 con 0x7feb9c073800 2024-04-26T20:57:42.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.230+0000 7feb92a00640 1 -- 172.21.15.28:0/1312148247 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb8c01a070 con 0x7feb9c073800 2024-04-26T20:57:42.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.231+0000 7feb92a00640 1 --2- 172.21.15.28:0/1312148247 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7feb7c06c8f0 0x7feb7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:42.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.231+0000 7feb92a00640 1 -- 172.21.15.28:0/1312148247 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7feb8c08c070 con 0x7feb9c073800 2024-04-26T20:57:42.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.232+0000 7feb9a000640 1 --2- 172.21.15.28:0/1312148247 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7feb7c06c8f0 0x7feb7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:42.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.232+0000 7feb9a000640 1 --2- 172.21.15.28:0/1312148247 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7feb7c06c8f0 0x7feb7c06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feb9c1f61f0 tx=0x7feb88008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:42.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.236+0000 7feb92a00640 1 -- 172.21.15.28:0/1312148247 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feb8c05a390 con 0x7feb9c073800 2024-04-26T20:57:42.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.443+0000 7feb9be00640 1 -- 172.21.15.28:0/1312148247 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7feb9c1442f0 con 0x7feb9c073800 2024-04-26T20:57:42.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.444+0000 7feb92a00640 1 -- 172.21.15.28:0/1312148247 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7feb8c059c90 con 0x7feb9c073800 2024-04-26T20:57:42.445 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-04-26T20:57:42.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.446+0000 7feb9be00640 1 -- 172.21.15.28:0/1312148247 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7feb7c06c8f0 msgr2=0x7feb7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:42.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.446+0000 7feb9be00640 1 --2- 172.21.15.28:0/1312148247 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7feb7c06c8f0 0x7feb7c06edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feb9c1f61f0 tx=0x7feb88008040 comp rx=0 tx=0).stop 2024-04-26T20:57:42.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.447+0000 7feb9be00640 1 -- 172.21.15.28:0/1312148247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb9c073800 msgr2=0x7feb9c15fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:42.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.447+0000 7feb9be00640 1 --2- 172.21.15.28:0/1312148247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb9c073800 0x7feb9c15fc80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00e9e0 tx=0x7feb8c00eea0 comp rx=0 tx=0).stop 2024-04-26T20:57:42.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.447+0000 7feb9be00640 1 -- 172.21.15.28:0/1312148247 shutdown_connections 2024-04-26T20:57:42.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.447+0000 7feb9be00640 1 --2- 172.21.15.28:0/1312148247 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7feb7c06c8f0 0x7feb7c06edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:42.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.447+0000 7feb9be00640 1 --2- 172.21.15.28:0/1312148247 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb9c145080 0x7feb9c142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:42.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.447+0000 7feb9be00640 1 --2- 172.21.15.28:0/1312148247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb9c144b90 0x7feb9c15dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:42.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.447+0000 7feb9be00640 1 --2- 172.21.15.28:0/1312148247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb9c073800 0x7feb9c15fc80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:42.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.447+0000 7feb9be00640 1 -- 172.21.15.28:0/1312148247 >> 172.21.15.28:0/1312148247 conn(0x7feb9c0d0a50 msgr2=0x7feb9c0d0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:42.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.447+0000 7feb9be00640 1 -- 172.21.15.28:0/1312148247 shutdown_connections 2024-04-26T20:57:42.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.447+0000 7feb9be00640 1 -- 172.21.15.28:0/1312148247 wait complete. 2024-04-26T20:57:42.467 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi103.front.sepia.ceph.com... 2024-04-26T20:57:42.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:42.467 DEBUG:teuthology.orchestra.run.smithi103:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-26T20:57:42.562 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:57:42.562 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi103.front.sepia.ceph.com 2024-04-26T20:57:42.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:42.563 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T20:57:42.625 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:57:42.625 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:57:42.684 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:57:42.685 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:57:42.741 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:57:42.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.950+0000 7f743c200640 1 Processor -- start 2024-04-26T20:57:42.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.950+0000 7f743c200640 1 -- start start 2024-04-26T20:57:42.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.952+0000 7f743c200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f74340b6810 0x7f743415d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:42.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.952+0000 7f743c200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f74340740e0 0x7f743415daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:42.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.952+0000 7f743c200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74340d0a30 con 0x7f7434146980 2024-04-26T20:57:42.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.952+0000 7f743c200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f743415e510 con 0x7f74340740e0 2024-04-26T20:57:42.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.952+0000 7f743c200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f743415e230 con 0x7f74340b6810 2024-04-26T20:57:42.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.952+0000 7f743a200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f74340740e0 0x7f743415daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:42.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.952+0000 7f7439800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7434146980 0x7f74340b6100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58416/0 (socket says 172.21.15.28:58416) 2024-04-26T20:57:42.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.953+0000 7f7439800640 1 -- 172.21.15.28:0/2041108963 learned_addr learned my addr 172.21.15.28:0/2041108963 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:42.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.953+0000 7f7438e00640 1 --2- 172.21.15.28:0/2041108963 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f74340b6810 0x7f743415d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:42.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.953+0000 7f7433e00640 1 -- 172.21.15.28:0/2041108963 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1547864516 0 0) 0x7f7428001110 con 0x7f7434146980 2024-04-26T20:57:42.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.953+0000 7f7433e00640 1 -- 172.21.15.28:0/2041108963 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f741c003520 con 0x7f7434146980 2024-04-26T20:57:42.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.954+0000 7f7438e00640 1 -- 172.21.15.28:0/2041108963 >> v2:172.21.15.28:3300/0 conn(0x7f74340740e0 msgr2=0x7f743415daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:42.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.954+0000 7f7438e00640 1 --2- 172.21.15.28:0/2041108963 >> v2:172.21.15.28:3300/0 conn(0x7f74340740e0 0x7f743415daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:42.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.954+0000 7f7438e00640 1 -- 172.21.15.28:0/2041108963 >> v1:172.21.15.28:6789/0 conn(0x7f7434146980 legacy=0x7f74340b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:42.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.954+0000 7f7438e00640 1 -- 172.21.15.28:0/2041108963 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7434073600 con 0x7f74340b6810 2024-04-26T20:57:42.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.955+0000 7f7438e00640 1 --2- 172.21.15.28:0/2041108963 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f74340b6810 0x7f743415d5b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f74200097a0 tx=0x7f742002a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87dae7c449e2a18d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:42.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.955+0000 7f7433e00640 1 -- 172.21.15.28:0/2041108963 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420037070 con 0x7f74340b6810 2024-04-26T20:57:42.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.955+0000 7f7433e00640 1 -- 172.21.15.28:0/2041108963 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f742002ae40 con 0x7f74340b6810 2024-04-26T20:57:42.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.956+0000 7f7433e00640 1 -- 172.21.15.28:0/2041108963 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7420032660 con 0x7f74340b6810 2024-04-26T20:57:42.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.956+0000 7f743c200640 1 -- 172.21.15.28:0/2041108963 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f74340b6810 msgr2=0x7f743415d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:42.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.956+0000 7f743c200640 1 --2- 172.21.15.28:0/2041108963 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f74340b6810 0x7f743415d5b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f74200097a0 tx=0x7f742002a0f0 comp rx=0 tx=0).stop 2024-04-26T20:57:42.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.957+0000 7f743c200640 1 -- 172.21.15.28:0/2041108963 shutdown_connections 2024-04-26T20:57:42.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.957+0000 7f743c200640 1 --2- 172.21.15.28:0/2041108963 >> v2:172.21.15.28:3300/0 conn(0x7f74340740e0 0x7f743415daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:42.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.957+0000 7f743c200640 1 --2- 172.21.15.28:0/2041108963 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f74340b6810 0x7f743415d5b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:42.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.957+0000 7f743c200640 1 -- 172.21.15.28:0/2041108963 >> 172.21.15.28:0/2041108963 conn(0x7f743406e220 msgr2=0x7f743406e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:42.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.957+0000 7f743c200640 1 -- 172.21.15.28:0/2041108963 shutdown_connections 2024-04-26T20:57:42.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.957+0000 7f743c200640 1 -- 172.21.15.28:0/2041108963 wait complete. 2024-04-26T20:57:42.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.958+0000 7f743c200640 1 Processor -- start 2024-04-26T20:57:42.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.959+0000 7f743c200640 1 -- start start 2024-04-26T20:57:42.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.960+0000 7f743c200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f74340740e0 0x7f743413b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:42.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.960+0000 7f743c200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74340b6810 0x7f74340afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:42.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.960+0000 7f743c200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7434146980 0x7f7434154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:42.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.960+0000 7f743c200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7434071ff0 con 0x7f7434146980 2024-04-26T20:57:42.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.960+0000 7f743c200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7434073bc0 con 0x7f74340b6810 2024-04-26T20:57:42.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.960+0000 7f743c200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7434072170 con 0x7f74340740e0 2024-04-26T20:57:42.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.960+0000 7f743a200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7434146980 0x7f7434154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:42.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.960+0000 7f743a200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7434146980 0x7f7434154f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51082/0 (socket says 172.21.15.28:51082) 2024-04-26T20:57:42.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.960+0000 7f743a200640 1 -- 172.21.15.28:0/143305742 learned_addr learned my addr 172.21.15.28:0/143305742 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:42.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.960+0000 7f7439800640 1 --2- 172.21.15.28:0/143305742 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f74340740e0 0x7f743413b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:42.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.960+0000 7f743a200640 1 -- 172.21.15.28:0/143305742 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f74340740e0 msgr2=0x7f743413b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:42.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.960+0000 7f7438e00640 1 --2- 172.21.15.28:0/143305742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74340b6810 0x7f74340afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:42.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.960+0000 7f743a200640 1 --2- 172.21.15.28:0/143305742 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f74340740e0 0x7f743413b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:42.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.960+0000 7f743a200640 1 -- 172.21.15.28:0/143305742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74340b6810 msgr2=0x7f74340afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:42.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.960+0000 7f743a200640 1 --2- 172.21.15.28:0/143305742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74340b6810 0x7f74340afbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:42.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.960+0000 7f743a200640 1 -- 172.21.15.28:0/143305742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743413c050 con 0x7f7434146980 2024-04-26T20:57:42.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.961+0000 7f743a200640 1 --2- 172.21.15.28:0/143305742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7434146980 0x7f7434154f50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f742c009dd0 tx=0x7f742c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:42.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.961+0000 7f7431600640 1 -- 172.21.15.28:0/143305742 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c018020 con 0x7f7434146980 2024-04-26T20:57:42.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.961+0000 7f7431600640 1 -- 172.21.15.28:0/143305742 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f742c014480 con 0x7f7434146980 2024-04-26T20:57:42.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.961+0000 7f7431600640 1 -- 172.21.15.28:0/143305742 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c014870 con 0x7f7434146980 2024-04-26T20:57:42.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.961+0000 7f743c200640 1 -- 172.21.15.28:0/143305742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743413c330 con 0x7f7434146980 2024-04-26T20:57:42.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.961+0000 7f743c200640 1 -- 172.21.15.28:0/143305742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7434155490 con 0x7f7434146980 2024-04-26T20:57:42.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.962+0000 7f743c200640 1 -- 172.21.15.28:0/143305742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f743406a400 con 0x7f7434146980 2024-04-26T20:57:42.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.963+0000 7f7431600640 1 -- 172.21.15.28:0/143305742 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f742c005d10 con 0x7f7434146980 2024-04-26T20:57:42.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.964+0000 7f7431600640 1 --2- 172.21.15.28:0/143305742 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f741406c8f0 0x7f741406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:42.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.965+0000 7f7431600640 1 -- 172.21.15.28:0/143305742 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f742c08c750 con 0x7f7434146980 2024-04-26T20:57:42.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.965+0000 7f7439800640 1 --2- 172.21.15.28:0/143305742 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f741406c8f0 0x7f741406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:42.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.965+0000 7f7439800640 1 --2- 172.21.15.28:0/143305742 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f741406c8f0 0x7f741406edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f74280012b0 tx=0x7f7428009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:42.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:42.969+0000 7f7431600640 1 -- 172.21.15.28:0/143305742 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f742c022070 con 0x7f7434146980 2024-04-26T20:57:43.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.174+0000 7f743c200640 1 -- 172.21.15.28:0/143305742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f743406ff00 con 0x7f7434146980 2024-04-26T20:57:43.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.177+0000 7f7431600640 1 -- 172.21.15.28:0/143305742 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16831 (secure 0 0 0) 0x7f742c010040 con 0x7f7434146980 2024-04-26T20:57:43.179 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:43.179 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":23,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:57:37.996297+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:57:43.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.180+0000 7f743c200640 1 -- 172.21.15.28:0/143305742 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f741406c8f0 msgr2=0x7f741406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:43.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.180+0000 7f743c200640 1 --2- 172.21.15.28:0/143305742 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f741406c8f0 0x7f741406edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f74280012b0 tx=0x7f7428009040 comp rx=0 tx=0).stop 2024-04-26T20:57:43.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.180+0000 7f743c200640 1 -- 172.21.15.28:0/143305742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7434146980 msgr2=0x7f7434154f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:43.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.180+0000 7f743c200640 1 --2- 172.21.15.28:0/143305742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7434146980 0x7f7434154f50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f742c009dd0 tx=0x7f742c00c490 comp rx=0 tx=0).stop 2024-04-26T20:57:43.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.180+0000 7f743c200640 1 -- 172.21.15.28:0/143305742 shutdown_connections 2024-04-26T20:57:43.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.180+0000 7f743c200640 1 --2- 172.21.15.28:0/143305742 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f741406c8f0 0x7f741406edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:43.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.180+0000 7f743c200640 1 --2- 172.21.15.28:0/143305742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7434146980 0x7f7434154f50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:43.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.180+0000 7f743c200640 1 --2- 172.21.15.28:0/143305742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74340b6810 0x7f74340afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:43.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.180+0000 7f743c200640 1 --2- 172.21.15.28:0/143305742 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f74340740e0 0x7f743413b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:43.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.180+0000 7f743c200640 1 -- 172.21.15.28:0/143305742 >> 172.21.15.28:0/143305742 conn(0x7f743406e220 msgr2=0x7f743415ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:43.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.180+0000 7f743c200640 1 -- 172.21.15.28:0/143305742 shutdown_connections 2024-04-26T20:57:43.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.180+0000 7f743c200640 1 -- 172.21.15.28:0/143305742 wait complete. 2024-04-26T20:57:43.202 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:56:48.688498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:56:50.117541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:57:43.202 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:57:43.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.413+0000 7fb382000640 1 Processor -- start 2024-04-26T20:57:43.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.413+0000 7fb382000640 1 -- start start 2024-04-26T20:57:43.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.415+0000 7fb382000640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fb37c0b52f0 0x7fb37c070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:43.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.415+0000 7fb382000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb37c0b56d0 con 0x7fb37c0d0c80 2024-04-26T20:57:43.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.415+0000 7fb382000640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb37c0ce350 con 0x7fb37c0d1770 2024-04-26T20:57:43.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.415+0000 7fb382000640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fb37c0ce5f0 con 0x7fb37c0b52f0 2024-04-26T20:57:43.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.415+0000 7fb380c00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb37c0d0c80 0x7fb37c0d1060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58426/0 (socket says 172.21.15.28:58426) 2024-04-26T20:57:43.416 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.415+0000 7fb380c00640 1 -- 172.21.15.28:0/1778570881 learned_addr learned my addr 172.21.15.28:0/1778570881 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:43.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.415+0000 7fb381600640 1 --2- 172.21.15.28:0/1778570881 >> v2:172.21.15.103:3300/0 conn(0x7fb37c0b52f0 0x7fb37c070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:43.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.415+0000 7fb37b400640 1 -- 172.21.15.28:0/1778570881 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3457923497 0 0) 0x7fb37c0b56d0 con 0x7fb37c0d0c80 2024-04-26T20:57:43.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.416+0000 7fb37b400640 1 -- 172.21.15.28:0/1778570881 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb360003520 con 0x7fb37c0d0c80 2024-04-26T20:57:43.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.416+0000 7fb37b400640 1 -- 172.21.15.28:0/1778570881 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 509429659 0 0) 0x7fb360003520 con 0x7fb37c0d0c80 2024-04-26T20:57:43.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.416+0000 7fb37b400640 1 -- 172.21.15.28:0/1778570881 >> v2:172.21.15.103:3300/0 conn(0x7fb37c0b52f0 msgr2=0x7fb37c070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:43.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.416+0000 7fb37b400640 1 --2- 172.21.15.28:0/1778570881 >> v2:172.21.15.103:3300/0 conn(0x7fb37c0b52f0 0x7fb37c070ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:43.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.416+0000 7fb37b400640 1 -- 172.21.15.28:0/1778570881 >> v1:172.21.15.103:6789/0 conn(0x7fb37c0d1770 legacy=0x7fb37c0707d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:43.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.416+0000 7fb37b400640 1 -- 172.21.15.28:0/1778570881 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37c0ce8d0 con 0x7fb37c0d0c80 2024-04-26T20:57:43.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.416+0000 7fb37b400640 1 -- 172.21.15.28:0/1778570881 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fb36c001670 con 0x7fb37c0d0c80 2024-04-26T20:57:43.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.416+0000 7fb37b400640 1 -- 172.21.15.28:0/1778570881 >> v1:172.21.15.28:6789/0 conn(0x7fb37c0d0c80 legacy=0x7fb37c0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:43.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.416+0000 7fb37b400640 1 --2- 172.21.15.28:0/1778570881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb360003a00 0x7fb360003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:43.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.416+0000 7fb37b400640 1 --2- 172.21.15.28:0/1778570881 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb360004800 0x7fb360004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:43.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.416+0000 7fb37b400640 1 --2- 172.21.15.28:0/1778570881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3600052e0 0x7fb36000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:43.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.416+0000 7fb37b400640 1 -- 172.21.15.28:0/1778570881 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c0ce5f0 con 0x7fb3600052e0 2024-04-26T20:57:43.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.416+0000 7fb37b400640 1 -- 172.21.15.28:0/1778570881 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb36000c1e0 con 0x7fb360003a00 2024-04-26T20:57:43.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.416+0000 7fb37b400640 1 -- 172.21.15.28:0/1778570881 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb36000c3e0 con 0x7fb360004800 2024-04-26T20:57:43.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.416+0000 7fb381600640 1 --2- 172.21.15.28:0/1778570881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3600052e0 0x7fb36000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:43.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.417+0000 7fb380c00640 1 --2- 172.21.15.28:0/1778570881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb360003a00 0x7fb360003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:43.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.417+0000 7fb37be00640 1 --2- 172.21.15.28:0/1778570881 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb360004800 0x7fb360004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:43.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.417+0000 7fb381600640 1 -- 172.21.15.28:0/1778570881 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb360004800 msgr2=0x7fb360004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:43.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.417+0000 7fb381600640 1 --2- 172.21.15.28:0/1778570881 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb360004800 0x7fb360004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:43.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.417+0000 7fb381600640 1 -- 172.21.15.28:0/1778570881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb360003a00 msgr2=0x7fb360003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:43.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.417+0000 7fb381600640 1 --2- 172.21.15.28:0/1778570881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb360003a00 0x7fb360003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:43.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.417+0000 7fb381600640 1 -- 172.21.15.28:0/1778570881 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb360003520 con 0x7fb3600052e0 2024-04-26T20:57:43.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.418+0000 7fb381600640 1 -- 172.21.15.28:0/1778570881 reap_dead start 2024-04-26T20:57:43.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.418+0000 7fb381600640 1 --2- 172.21.15.28:0/1778570881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3600052e0 0x7fb36000b9c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb374009ac0 tx=0x7fb360005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37384e20c642347b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:43.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.418+0000 7fb37b400640 1 -- 172.21.15.28:0/1778570881 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374009690 con 0x7fb3600052e0 2024-04-26T20:57:43.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.418+0000 7fb37b400640 1 -- 172.21.15.28:0/1778570881 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb374004020 con 0x7fb3600052e0 2024-04-26T20:57:43.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.418+0000 7fb37b400640 1 -- 172.21.15.28:0/1778570881 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374004410 con 0x7fb3600052e0 2024-04-26T20:57:43.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.419+0000 7fb382000640 1 -- 172.21.15.28:0/1778570881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3600052e0 msgr2=0x7fb36000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:43.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.419+0000 7fb382000640 1 --2- 172.21.15.28:0/1778570881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3600052e0 0x7fb36000b9c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb374009ac0 tx=0x7fb360005830 comp rx=0 tx=0).stop 2024-04-26T20:57:43.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.419+0000 7fb382000640 1 -- 172.21.15.28:0/1778570881 shutdown_connections 2024-04-26T20:57:43.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.419+0000 7fb382000640 1 --2- 172.21.15.28:0/1778570881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3600052e0 0x7fb36000b9c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:43.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.419+0000 7fb382000640 1 -- 172.21.15.28:0/1778570881 >> 172.21.15.28:0/1778570881 conn(0x7fb37c0b47d0 msgr2=0x7fb37c0b4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:43.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.419+0000 7fb382000640 1 -- 172.21.15.28:0/1778570881 shutdown_connections 2024-04-26T20:57:43.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.419+0000 7fb382000640 1 -- 172.21.15.28:0/1778570881 wait complete. 2024-04-26T20:57:43.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.421+0000 7fb382000640 1 Processor -- start 2024-04-26T20:57:43.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.421+0000 7fb382000640 1 -- start start 2024-04-26T20:57:43.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.422+0000 7fb382000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3600052e0 0x7fb37c148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:43.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.422+0000 7fb382000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb37c1584d0 0x7fb37c158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:43.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.422+0000 7fb382000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb37c06c3e0 0x7fb37c06c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:43.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.422+0000 7fb382000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c15d1d0 con 0x7fb3600052e0 2024-04-26T20:57:43.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.422+0000 7fb382000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c0d0a00 con 0x7fb37c06c3e0 2024-04-26T20:57:43.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.422+0000 7fb382000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb37c15d350 con 0x7fb37c1584d0 2024-04-26T20:57:43.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.422+0000 7fb380c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3600052e0 0x7fb37c148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:43.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.422+0000 7fb380c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3600052e0 0x7fb37c148eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51098/0 (socket says 172.21.15.28:51098) 2024-04-26T20:57:43.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.422+0000 7fb380c00640 1 -- 172.21.15.28:0/157836739 learned_addr learned my addr 172.21.15.28:0/157836739 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:43.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.422+0000 7fb381600640 1 --2- 172.21.15.28:0/157836739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb37c06c3e0 0x7fb37c06c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:43.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.423+0000 7fb37be00640 1 --2- 172.21.15.28:0/157836739 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb37c1584d0 0x7fb37c158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:43.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.423+0000 7fb380c00640 1 -- 172.21.15.28:0/157836739 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb37c1584d0 msgr2=0x7fb37c158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:43.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.423+0000 7fb380c00640 1 --2- 172.21.15.28:0/157836739 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb37c1584d0 0x7fb37c158950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:43.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.423+0000 7fb380c00640 1 -- 172.21.15.28:0/157836739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb37c06c3e0 msgr2=0x7fb37c06c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:43.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.423+0000 7fb380c00640 1 --2- 172.21.15.28:0/157836739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb37c06c3e0 0x7fb37c06c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:43.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.423+0000 7fb380c00640 1 -- 172.21.15.28:0/157836739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37c06d1a0 con 0x7fb3600052e0 2024-04-26T20:57:43.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.423+0000 7fb380c00640 1 --2- 172.21.15.28:0/157836739 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3600052e0 0x7fb37c148eb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb36c00e780 tx=0x7fb36c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:43.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.423+0000 7fb378c00640 1 -- 172.21.15.28:0/157836739 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c00c660 con 0x7fb3600052e0 2024-04-26T20:57:43.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.423+0000 7fb378c00640 1 -- 172.21.15.28:0/157836739 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36c018070 con 0x7fb3600052e0 2024-04-26T20:57:43.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.423+0000 7fb378c00640 1 -- 172.21.15.28:0/157836739 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c00ca00 con 0x7fb3600052e0 2024-04-26T20:57:43.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.423+0000 7fb382000640 1 -- 172.21.15.28:0/157836739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37c06d480 con 0x7fb3600052e0 2024-04-26T20:57:43.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.424+0000 7fb382000640 1 -- 172.21.15.28:0/157836739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb37c06da40 con 0x7fb3600052e0 2024-04-26T20:57:43.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.425+0000 7fb382000640 1 -- 172.21.15.28:0/157836739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb37c06a250 con 0x7fb3600052e0 2024-04-26T20:57:43.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.426+0000 7fb378c00640 1 -- 172.21.15.28:0/157836739 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb36c00cba0 con 0x7fb3600052e0 2024-04-26T20:57:43.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.427+0000 7fb378c00640 1 --2- 172.21.15.28:0/157836739 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb35806c8f0 0x7fb35806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:43.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.427+0000 7fb378c00640 1 -- 172.21.15.28:0/157836739 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fb36c08b920 con 0x7fb3600052e0 2024-04-26T20:57:43.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.427+0000 7fb37be00640 1 --2- 172.21.15.28:0/157836739 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb35806c8f0 0x7fb35806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:43.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.428+0000 7fb37be00640 1 --2- 172.21.15.28:0/157836739 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb35806c8f0 0x7fb35806edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb364007fd0 tx=0x7fb364009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:43.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.431+0000 7fb378c00640 1 -- 172.21.15.28:0/157836739 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb36c059cd0 con 0x7fb3600052e0 2024-04-26T20:57:43.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.625+0000 7fb382000640 1 -- 172.21.15.28:0/157836739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb37c0d0570 con 0x7fb3600052e0 2024-04-26T20:57:43.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.626+0000 7fb378c00640 1 -- 172.21.15.28:0/157836739 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7fb36c0595d0 con 0x7fb3600052e0 2024-04-26T20:57:43.627 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:57:43.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.629+0000 7fb382000640 1 -- 172.21.15.28:0/157836739 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb35806c8f0 msgr2=0x7fb35806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:43.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.629+0000 7fb382000640 1 --2- 172.21.15.28:0/157836739 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb35806c8f0 0x7fb35806edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb364007fd0 tx=0x7fb364009040 comp rx=0 tx=0).stop 2024-04-26T20:57:43.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.629+0000 7fb382000640 1 -- 172.21.15.28:0/157836739 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3600052e0 msgr2=0x7fb37c148eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:43.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.629+0000 7fb382000640 1 --2- 172.21.15.28:0/157836739 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3600052e0 0x7fb37c148eb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb36c00e780 tx=0x7fb36c00ec40 comp rx=0 tx=0).stop 2024-04-26T20:57:43.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.629+0000 7fb382000640 1 -- 172.21.15.28:0/157836739 shutdown_connections 2024-04-26T20:57:43.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.629+0000 7fb382000640 1 --2- 172.21.15.28:0/157836739 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb35806c8f0 0x7fb35806edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:43.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.629+0000 7fb382000640 1 --2- 172.21.15.28:0/157836739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb37c06c3e0 0x7fb37c06c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:43.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.629+0000 7fb382000640 1 --2- 172.21.15.28:0/157836739 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb37c1584d0 0x7fb37c158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:43.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.629+0000 7fb382000640 1 --2- 172.21.15.28:0/157836739 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3600052e0 0x7fb37c148eb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:43.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.629+0000 7fb382000640 1 -- 172.21.15.28:0/157836739 >> 172.21.15.28:0/157836739 conn(0x7fb37c0b47d0 msgr2=0x7fb37c0b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:43.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.629+0000 7fb382000640 1 -- 172.21.15.28:0/157836739 shutdown_connections 2024-04-26T20:57:43.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.629+0000 7fb382000640 1 -- 172.21.15.28:0/157836739 wait complete. 2024-04-26T20:57:43.647 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:57:43.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.853+0000 7f19e9a00640 1 Processor -- start 2024-04-26T20:57:43.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.853+0000 7f19e9a00640 1 -- start start 2024-04-26T20:57:43.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.854+0000 7f19e9a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19e40d1de0 0x7f19e40d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:43.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.854+0000 7f19e9a00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f19e415f360 0x7f19e40d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:43.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.854+0000 7f19e9a00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19e40d2700 con 0x7f19e40bac00 2024-04-26T20:57:43.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.854+0000 7f19e9a00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f19e40ba350 con 0x7f19e415f360 2024-04-26T20:57:43.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.854+0000 7f19e9a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f19e40b9ab0 con 0x7f19e40d1de0 2024-04-26T20:57:43.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.855+0000 7f19e9000640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f19e40bac00 0x7f19e40d39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.28:39268/0 (socket says 172.21.15.28:39268) 2024-04-26T20:57:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.855+0000 7f19e9000640 1 -- 172.21.15.28:0/2761830847 learned_addr learned my addr 172.21.15.28:0/2761830847 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.855+0000 7f19e3e00640 1 --2- 172.21.15.28:0/2761830847 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19e40d1de0 0x7f19e40d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.855+0000 7f19e3400640 1 --2- 172.21.15.28:0/2761830847 >> v2:172.21.15.103:3300/0 conn(0x7f19e415f360 0x7f19e40d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:43.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.856+0000 7f19e2a00640 1 -- 172.21.15.28:0/2761830847 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 687871320 0 0) 0x7f19e40d2700 con 0x7f19e40bac00 2024-04-26T20:57:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.856+0000 7f19e2a00640 1 -- 172.21.15.28:0/2761830847 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19c8003520 con 0x7f19e40bac00 2024-04-26T20:57:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.856+0000 7f19e3400640 1 -- 172.21.15.28:0/2761830847 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19e40d1de0 msgr2=0x7f19e40d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.856+0000 7f19e3400640 1 --2- 172.21.15.28:0/2761830847 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19e40d1de0 0x7f19e40d21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.856+0000 7f19e3400640 1 -- 172.21.15.28:0/2761830847 >> v1:172.21.15.103:6789/0 conn(0x7f19e40bac00 legacy=0x7f19e40d39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.856+0000 7f19e3400640 1 -- 172.21.15.28:0/2761830847 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e40b9d90 con 0x7f19e415f360 2024-04-26T20:57:43.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.858+0000 7f19e3400640 1 --2- 172.21.15.28:0/2761830847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e415f360 0x7f19e40d3490 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0097a0 tx=0x7f19cc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=656586c2434fea54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:43.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.858+0000 7f19e2a00640 1 -- 172.21.15.28:0/2761830847 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc037070 con 0x7f19e415f360 2024-04-26T20:57:43.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.858+0000 7f19e2a00640 1 -- 172.21.15.28:0/2761830847 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19cc032440 con 0x7f19e415f360 2024-04-26T20:57:43.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.858+0000 7f19e2a00640 1 -- 172.21.15.28:0/2761830847 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc032810 con 0x7f19e415f360 2024-04-26T20:57:43.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.858+0000 7f19e9a00640 1 -- 172.21.15.28:0/2761830847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e415f360 msgr2=0x7f19e40d3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:43.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.858+0000 7f19e9a00640 1 --2- 172.21.15.28:0/2761830847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e415f360 0x7f19e40d3490 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f19cc0097a0 tx=0x7f19cc02a180 comp rx=0 tx=0).stop 2024-04-26T20:57:43.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.859+0000 7f19e9a00640 1 -- 172.21.15.28:0/2761830847 shutdown_connections 2024-04-26T20:57:43.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.859+0000 7f19e9a00640 1 --2- 172.21.15.28:0/2761830847 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e415f360 0x7f19e40d3490 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:43.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.859+0000 7f19e9a00640 1 --2- 172.21.15.28:0/2761830847 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19e40d1de0 0x7f19e40d21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:43.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.859+0000 7f19e9a00640 1 -- 172.21.15.28:0/2761830847 >> 172.21.15.28:0/2761830847 conn(0x7f19e406c770 msgr2=0x7f19e406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:43.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.859+0000 7f19e9a00640 1 -- 172.21.15.28:0/2761830847 shutdown_connections 2024-04-26T20:57:43.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.859+0000 7f19e9a00640 1 -- 172.21.15.28:0/2761830847 wait complete. 2024-04-26T20:57:43.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.860+0000 7f19e9a00640 1 Processor -- start 2024-04-26T20:57:43.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.860+0000 7f19e9a00640 1 -- start start 2024-04-26T20:57:43.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.861+0000 7f19e9a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e40bac00 0x7f19e40b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:43.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.861+0000 7f19e9a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19e40d1de0 0x7f19e40b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:43.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.861+0000 7f19e9a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19e415f360 0x7f19e40b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.861+0000 7f19e9a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f19e40d40e0 con 0x7f19e415f360 2024-04-26T20:57:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.861+0000 7f19e9a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19e40d2f70 con 0x7f19e40bac00 2024-04-26T20:57:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.861+0000 7f19e9a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f19e40d4260 con 0x7f19e40d1de0 2024-04-26T20:57:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.862+0000 7f19e9000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19e415f360 0x7f19e40b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.862+0000 7f19e9000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19e415f360 0x7f19e40b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51114/0 (socket says 172.21.15.28:51114) 2024-04-26T20:57:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.862+0000 7f19e9000640 1 -- 172.21.15.28:0/1161305281 learned_addr learned my addr 172.21.15.28:0/1161305281 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.862+0000 7f19e3e00640 1 --2- 172.21.15.28:0/1161305281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e40bac00 0x7f19e40b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.862+0000 7f19e3400640 1 --2- 172.21.15.28:0/1161305281 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19e40d1de0 0x7f19e40b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.862+0000 7f19e9000640 1 -- 172.21.15.28:0/1161305281 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19e40d1de0 msgr2=0x7f19e40b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.862+0000 7f19e9000640 1 --2- 172.21.15.28:0/1161305281 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19e40d1de0 0x7f19e40b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.862+0000 7f19e9000640 1 -- 172.21.15.28:0/1161305281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e40bac00 msgr2=0x7f19e40b30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.862+0000 7f19e9000640 1 --2- 172.21.15.28:0/1161305281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e40bac00 0x7f19e40b30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.862+0000 7f19e9000640 1 -- 172.21.15.28:0/1161305281 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19e4160470 con 0x7f19e415f360 2024-04-26T20:57:43.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.863+0000 7f19e9000640 1 --2- 172.21.15.28:0/1161305281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19e415f360 0x7f19e40b4fd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00c3e0 tx=0x7f19dc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:43.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.863+0000 7f19dbe00640 1 -- 172.21.15.28:0/1161305281 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc017070 con 0x7f19e415f360 2024-04-26T20:57:43.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.863+0000 7f19dbe00640 1 -- 172.21.15.28:0/1161305281 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19dc005ce0 con 0x7f19e415f360 2024-04-26T20:57:43.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.863+0000 7f19dbe00640 1 -- 172.21.15.28:0/1161305281 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19dc0072d0 con 0x7f19e415f360 2024-04-26T20:57:43.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.863+0000 7f19e9a00640 1 -- 172.21.15.28:0/1161305281 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19e41f83d0 con 0x7f19e415f360 2024-04-26T20:57:43.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.863+0000 7f19e9a00640 1 -- 172.21.15.28:0/1161305281 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19e41f8990 con 0x7f19e415f360 2024-04-26T20:57:43.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.864+0000 7f19e9a00640 1 -- 172.21.15.28:0/1161305281 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19e40b6d30 con 0x7f19e415f360 2024-04-26T20:57:43.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.865+0000 7f19dbe00640 1 -- 172.21.15.28:0/1161305281 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19dc01a070 con 0x7f19e415f360 2024-04-26T20:57:43.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.866+0000 7f19dbe00640 1 --2- 172.21.15.28:0/1161305281 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f19c006c8f0 0x7f19c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:43.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.866+0000 7f19dbe00640 1 -- 172.21.15.28:0/1161305281 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f19dc08af20 con 0x7f19e415f360 2024-04-26T20:57:43.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.866+0000 7f19e3e00640 1 --2- 172.21.15.28:0/1161305281 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f19c006c8f0 0x7f19c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:43.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.867+0000 7f19e3e00640 1 --2- 172.21.15.28:0/1161305281 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f19c006c8f0 0x7f19c006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f19e40baa20 tx=0x7f19d400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:43.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:43.870+0000 7f19dbe00640 1 -- 172.21.15.28:0/1161305281 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f19e40b6d30 con 0x7f19e415f360 2024-04-26T20:57:44.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.071+0000 7f19e9a00640 1 -- 172.21.15.28:0/1161305281 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f19e40cf4a0 con 0x7f19e415f360 2024-04-26T20:57:44.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.072+0000 7f19dbe00640 1 -- 172.21.15.28:0/1161305281 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f19e40cf4a0 con 0x7f19e415f360 2024-04-26T20:57:44.073 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-04-26T20:57:44.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.074+0000 7f19e9a00640 1 -- 172.21.15.28:0/1161305281 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f19c006c8f0 msgr2=0x7f19c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:44.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.074+0000 7f19e9a00640 1 --2- 172.21.15.28:0/1161305281 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f19c006c8f0 0x7f19c006edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f19e40baa20 tx=0x7f19d400b040 comp rx=0 tx=0).stop 2024-04-26T20:57:44.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.074+0000 7f19e9a00640 1 -- 172.21.15.28:0/1161305281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19e415f360 msgr2=0x7f19e40b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:44.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.074+0000 7f19e9a00640 1 --2- 172.21.15.28:0/1161305281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19e415f360 0x7f19e40b4fd0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00c3e0 tx=0x7f19dc00c8a0 comp rx=0 tx=0).stop 2024-04-26T20:57:44.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.075+0000 7f19e9a00640 1 -- 172.21.15.28:0/1161305281 shutdown_connections 2024-04-26T20:57:44.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.075+0000 7f19e9a00640 1 --2- 172.21.15.28:0/1161305281 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f19c006c8f0 0x7f19c006edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:44.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.075+0000 7f19e9a00640 1 --2- 172.21.15.28:0/1161305281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19e415f360 0x7f19e40b4fd0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:44.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.075+0000 7f19e9a00640 1 --2- 172.21.15.28:0/1161305281 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19e40d1de0 0x7f19e40b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:44.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.075+0000 7f19e9a00640 1 --2- 172.21.15.28:0/1161305281 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19e40bac00 0x7f19e40b30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:44.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.075+0000 7f19e9a00640 1 -- 172.21.15.28:0/1161305281 >> 172.21.15.28:0/1161305281 conn(0x7f19e406c770 msgr2=0x7f19e406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:44.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.075+0000 7f19e9a00640 1 -- 172.21.15.28:0/1161305281 shutdown_connections 2024-04-26T20:57:44.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.075+0000 7f19e9a00640 1 -- 172.21.15.28:0/1161305281 wait complete. 2024-04-26T20:57:44.096 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:57:44.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.310+0000 7fe863200640 1 Processor -- start 2024-04-26T20:57:44.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.311+0000 7fe863200640 1 -- start start 2024-04-26T20:57:44.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.312+0000 7fe863200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe8640d0c70 0x7fe864161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:44.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.312+0000 7fe863200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe8640b4a30 0x7fe86406e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:44.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.312+0000 7fe863200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe864161860 con 0x7fe8640d1050 2024-04-26T20:57:44.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.312+0000 7fe863200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fe8640cf2f0 con 0x7fe8640d0c70 2024-04-26T20:57:44.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.312+0000 7fe863200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe8640cf5d0 con 0x7fe8640b4a30 2024-04-26T20:57:44.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.313+0000 7fe861e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe8640d0c70 0x7fe864161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:44.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.313+0000 7fe861e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe8640d0c70 0x7fe864161320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.28:33530/0 (socket says 172.21.15.28:33530) 2024-04-26T20:57:44.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.313+0000 7fe861e00640 1 -- 172.21.15.28:0/4084768599 learned_addr learned my addr 172.21.15.28:0/4084768599 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:44.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.313+0000 7fe862800640 1 --2- 172.21.15.28:0/4084768599 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe8640b4a30 0x7fe86406e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:44.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.314+0000 7fe861e00640 1 -- 172.21.15.28:0/4084768599 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe8640b4a30 msgr2=0x7fe86406e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:44.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.314+0000 7fe861e00640 1 --2- 172.21.15.28:0/4084768599 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe8640b4a30 0x7fe86406e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:44.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.314+0000 7fe861e00640 1 -- 172.21.15.28:0/4084768599 >> v1:172.21.15.103:6789/0 conn(0x7fe8640d1050 legacy=0x7fe86406df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:44.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.314+0000 7fe861e00640 1 -- 172.21.15.28:0/4084768599 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8640cf8b0 con 0x7fe8640d0c70 2024-04-26T20:57:44.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.315+0000 7fe861e00640 1 --2- 172.21.15.28:0/4084768599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8640d0c70 0x7fe864161320 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe854009870 tx=0x7fe85402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3843bb40ec9235c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:44.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.315+0000 7fe860a00640 1 -- 172.21.15.28:0/4084768599 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854037070 con 0x7fe8640d0c70 2024-04-26T20:57:44.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.315+0000 7fe860a00640 1 -- 172.21.15.28:0/4084768599 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe854032440 con 0x7fe8640d0c70 2024-04-26T20:57:44.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.315+0000 7fe860a00640 1 -- 172.21.15.28:0/4084768599 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854032810 con 0x7fe8640d0c70 2024-04-26T20:57:44.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.316+0000 7fe863200640 1 -- 172.21.15.28:0/4084768599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8640d0c70 msgr2=0x7fe864161320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:44.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.316+0000 7fe863200640 1 --2- 172.21.15.28:0/4084768599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8640d0c70 0x7fe864161320 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe854009870 tx=0x7fe85402a250 comp rx=0 tx=0).stop 2024-04-26T20:57:44.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.316+0000 7fe863200640 1 -- 172.21.15.28:0/4084768599 shutdown_connections 2024-04-26T20:57:44.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.316+0000 7fe863200640 1 --2- 172.21.15.28:0/4084768599 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe8640b4a30 0x7fe86406e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:44.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.316+0000 7fe863200640 1 --2- 172.21.15.28:0/4084768599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8640d0c70 0x7fe864161320 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:44.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.316+0000 7fe863200640 1 -- 172.21.15.28:0/4084768599 >> 172.21.15.28:0/4084768599 conn(0x7fe86406c530 msgr2=0x7fe86406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:44.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.316+0000 7fe863200640 1 -- 172.21.15.28:0/4084768599 shutdown_connections 2024-04-26T20:57:44.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.316+0000 7fe863200640 1 -- 172.21.15.28:0/4084768599 wait complete. 2024-04-26T20:57:44.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.318+0000 7fe863200640 1 Processor -- start 2024-04-26T20:57:44.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.318+0000 7fe863200640 1 -- start start 2024-04-26T20:57:44.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.319+0000 7fe863200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8640b4a30 0x7fe8641591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:44.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.319+0000 7fe863200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8640d0c70 0x7fe86413a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:44.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.319+0000 7fe863200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe8640d1050 0x7fe86413ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:44.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.319+0000 7fe863200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe864146840 con 0x7fe8640b4a30 2024-04-26T20:57:44.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.319+0000 7fe863200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe8640cfe70 con 0x7fe8640d0c70 2024-04-26T20:57:44.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.319+0000 7fe863200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe8640d14e0 con 0x7fe8640d1050 2024-04-26T20:57:44.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.319+0000 7fe861e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8640b4a30 0x7fe8641591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:44.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.320+0000 7fe861e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8640b4a30 0x7fe8641591b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51130/0 (socket says 172.21.15.28:51130) 2024-04-26T20:57:44.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.320+0000 7fe861e00640 1 -- 172.21.15.28:0/59895330 learned_addr learned my addr 172.21.15.28:0/59895330 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:44.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.320+0000 7fe861400640 1 --2- 172.21.15.28:0/59895330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8640d0c70 0x7fe86413a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:44.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.320+0000 7fe862800640 1 --2- 172.21.15.28:0/59895330 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe8640d1050 0x7fe86413ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:44.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.320+0000 7fe861e00640 1 -- 172.21.15.28:0/59895330 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe8640d1050 msgr2=0x7fe86413ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:44.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.320+0000 7fe861e00640 1 --2- 172.21.15.28:0/59895330 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe8640d1050 0x7fe86413ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:44.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.320+0000 7fe861e00640 1 -- 172.21.15.28:0/59895330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8640d0c70 msgr2=0x7fe86413a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:44.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.320+0000 7fe861e00640 1 --2- 172.21.15.28:0/59895330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8640d0c70 0x7fe86413a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:44.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.320+0000 7fe861e00640 1 -- 172.21.15.28:0/59895330 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8641596f0 con 0x7fe8640b4a30 2024-04-26T20:57:44.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.320+0000 7fe861e00640 1 --2- 172.21.15.28:0/59895330 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8640b4a30 0x7fe8641591b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe854000c00 tx=0x7fe8540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:44.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.321+0000 7fe85a000640 1 -- 172.21.15.28:0/59895330 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854037040 con 0x7fe8640b4a30 2024-04-26T20:57:44.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.321+0000 7fe85a000640 1 -- 172.21.15.28:0/59895330 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe854042070 con 0x7fe8640b4a30 2024-04-26T20:57:44.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.321+0000 7fe85a000640 1 -- 172.21.15.28:0/59895330 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe854030880 con 0x7fe8640b4a30 2024-04-26T20:57:44.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.321+0000 7fe863200640 1 -- 172.21.15.28:0/59895330 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe864159970 con 0x7fe8640b4a30 2024-04-26T20:57:44.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.321+0000 7fe863200640 1 -- 172.21.15.28:0/59895330 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe864159f30 con 0x7fe8640b4a30 2024-04-26T20:57:44.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.322+0000 7fe863200640 1 -- 172.21.15.28:0/59895330 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe86406a2c0 con 0x7fe8640b4a30 2024-04-26T20:57:44.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.323+0000 7fe85a000640 1 -- 172.21.15.28:0/59895330 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe854030a20 con 0x7fe8640b4a30 2024-04-26T20:57:44.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.324+0000 7fe85a000640 1 --2- 172.21.15.28:0/59895330 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe84406c8f0 0x7fe84406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:44.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.324+0000 7fe85a000640 1 -- 172.21.15.28:0/59895330 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fe8540ac550 con 0x7fe8640b4a30 2024-04-26T20:57:44.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.324+0000 7fe861400640 1 --2- 172.21.15.28:0/59895330 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe84406c8f0 0x7fe84406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:44.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.325+0000 7fe861400640 1 --2- 172.21.15.28:0/59895330 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe84406c8f0 0x7fe84406edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe850007fd0 tx=0x7fe850009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:44.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.329+0000 7fe85a000640 1 -- 172.21.15.28:0/59895330 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe8641596f0 con 0x7fe8640b4a30 2024-04-26T20:57:44.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.534+0000 7fe863200640 1 -- 172.21.15.28:0/59895330 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe86406f760 con 0x7fe8640b4a30 2024-04-26T20:57:44.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.534+0000 7fe85a000640 1 -- 172.21.15.28:0/59895330 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fe854045360 con 0x7fe8640b4a30 2024-04-26T20:57:44.536 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-04-26T20:57:44.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.537+0000 7fe863200640 1 -- 172.21.15.28:0/59895330 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe84406c8f0 msgr2=0x7fe84406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:44.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.537+0000 7fe863200640 1 --2- 172.21.15.28:0/59895330 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe84406c8f0 0x7fe84406edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe850007fd0 tx=0x7fe850009040 comp rx=0 tx=0).stop 2024-04-26T20:57:44.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.537+0000 7fe863200640 1 -- 172.21.15.28:0/59895330 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8640b4a30 msgr2=0x7fe8641591b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:44.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.537+0000 7fe863200640 1 --2- 172.21.15.28:0/59895330 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8640b4a30 0x7fe8641591b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe854000c00 tx=0x7fe8540046c0 comp rx=0 tx=0).stop 2024-04-26T20:57:44.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.537+0000 7fe863200640 1 -- 172.21.15.28:0/59895330 shutdown_connections 2024-04-26T20:57:44.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.538+0000 7fe863200640 1 --2- 172.21.15.28:0/59895330 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe84406c8f0 0x7fe84406edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:44.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.538+0000 7fe863200640 1 --2- 172.21.15.28:0/59895330 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe8640d1050 0x7fe86413ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:44.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.538+0000 7fe863200640 1 --2- 172.21.15.28:0/59895330 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8640d0c70 0x7fe86413a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:44.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.538+0000 7fe863200640 1 --2- 172.21.15.28:0/59895330 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8640b4a30 0x7fe8641591b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:44.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.538+0000 7fe863200640 1 -- 172.21.15.28:0/59895330 >> 172.21.15.28:0/59895330 conn(0x7fe86406c530 msgr2=0x7fe8640b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:44.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.538+0000 7fe863200640 1 -- 172.21.15.28:0/59895330 shutdown_connections 2024-04-26T20:57:44.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.538+0000 7fe863200640 1 -- 172.21.15.28:0/59895330 wait complete. 2024-04-26T20:57:44.559 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:57:44.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.769+0000 7f3436800640 1 Processor -- start 2024-04-26T20:57:44.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.770+0000 7f3436800640 1 -- start start 2024-04-26T20:57:44.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.771+0000 7f3436800640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f343014c300 0x7f34300ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:44.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.771+0000 7f3436800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f343014ce60 0x7f34300ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:44.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.771+0000 7f3436800640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3430070190 con 0x7f34300cf0a0 2024-04-26T20:57:44.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.771+0000 7f3436800640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f343014c6e0 con 0x7f343014c300 2024-04-26T20:57:44.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.771+0000 7f3436800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f343006f9e0 con 0x7f343014ce60 2024-04-26T20:57:44.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.772+0000 7f3434a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f343014ce60 0x7f34300ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:44.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.772+0000 7f3435400640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f343014c300 0x7f34300ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:44.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.772+0000 7f3435400640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f343014c300 0x7f34300ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.28:33554/0 (socket says 172.21.15.28:33554) 2024-04-26T20:57:44.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.772+0000 7f3435400640 1 -- 172.21.15.28:0/3547871944 learned_addr learned my addr 172.21.15.28:0/3547871944 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:44.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.773+0000 7f3435400640 1 -- 172.21.15.28:0/3547871944 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f343014ce60 msgr2=0x7f34300ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:44.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.773+0000 7f3435400640 1 --2- 172.21.15.28:0/3547871944 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f343014ce60 0x7f34300ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:44.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.773+0000 7f3435400640 1 -- 172.21.15.28:0/3547871944 >> v1:172.21.15.103:6789/0 conn(0x7f34300cf0a0 legacy=0x7f3430167080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:44.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.773+0000 7f3435400640 1 -- 172.21.15.28:0/3547871944 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343006fc10 con 0x7f343014c300 2024-04-26T20:57:44.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.774+0000 7f3435400640 1 --2- 172.21.15.28:0/3547871944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f343014c300 0x7f34300ce620 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3424009870 tx=0x7f342402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e028fcb0c52b1496 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:44.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.775+0000 7f342fe00640 1 -- 172.21.15.28:0/3547871944 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424037070 con 0x7f343014c300 2024-04-26T20:57:44.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.775+0000 7f342fe00640 1 -- 172.21.15.28:0/3547871944 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3424032440 con 0x7f343014c300 2024-04-26T20:57:44.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.775+0000 7f342fe00640 1 -- 172.21.15.28:0/3547871944 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424032810 con 0x7f343014c300 2024-04-26T20:57:44.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.775+0000 7f3436800640 1 -- 172.21.15.28:0/3547871944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f343014c300 msgr2=0x7f34300ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:44.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.775+0000 7f3436800640 1 --2- 172.21.15.28:0/3547871944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f343014c300 0x7f34300ce620 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3424009870 tx=0x7f342402a250 comp rx=0 tx=0).stop 2024-04-26T20:57:44.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.776+0000 7f3436800640 1 -- 172.21.15.28:0/3547871944 shutdown_connections 2024-04-26T20:57:44.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.776+0000 7f3436800640 1 --2- 172.21.15.28:0/3547871944 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f343014ce60 0x7f34300ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:44.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.776+0000 7f3436800640 1 --2- 172.21.15.28:0/3547871944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f343014c300 0x7f34300ce620 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:44.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.776+0000 7f3436800640 1 -- 172.21.15.28:0/3547871944 >> 172.21.15.28:0/3547871944 conn(0x7f34300d0a50 msgr2=0x7f34300d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:44.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.776+0000 7f3436800640 1 -- 172.21.15.28:0/3547871944 shutdown_connections 2024-04-26T20:57:44.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.776+0000 7f3436800640 1 -- 172.21.15.28:0/3547871944 wait complete. 2024-04-26T20:57:44.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.778+0000 7f3436800640 1 Processor -- start 2024-04-26T20:57:44.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.778+0000 7f3436800640 1 -- start start 2024-04-26T20:57:44.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.779+0000 7f3436800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34300cf0a0 0x7f3430162450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:44.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.779+0000 7f3436800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f343014c300 0x7f34301439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:44.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.779+0000 7f3436800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f343014ce60 0x7f3430143f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:44.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.779+0000 7f3436800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f343006e4d0 con 0x7f343014c300 2024-04-26T20:57:44.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.779+0000 7f3436800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f343006feb0 con 0x7f34300cf0a0 2024-04-26T20:57:44.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.779+0000 7f3436800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f343006de80 con 0x7f343014ce60 2024-04-26T20:57:44.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.779+0000 7f3434a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f343014c300 0x7f34301439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:44.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.779+0000 7f3434a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f343014c300 0x7f34301439c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51138/0 (socket says 172.21.15.28:51138) 2024-04-26T20:57:44.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.779+0000 7f3434a00640 1 -- 172.21.15.28:0/2332216464 learned_addr learned my addr 172.21.15.28:0/2332216464 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:44.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.779+0000 7f3434a00640 1 -- 172.21.15.28:0/2332216464 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f343014ce60 msgr2=0x7f3430143f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:44.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.779+0000 7f3434a00640 1 --2- 172.21.15.28:0/2332216464 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f343014ce60 0x7f3430143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:44.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.779+0000 7f3434a00640 1 -- 172.21.15.28:0/2332216464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34300cf0a0 msgr2=0x7f3430162450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:44.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.779+0000 7f3434a00640 1 --2- 172.21.15.28:0/2332216464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34300cf0a0 0x7f3430162450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:44.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.780+0000 7f3434a00640 1 -- 172.21.15.28:0/2332216464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3430162ee0 con 0x7f343014c300 2024-04-26T20:57:44.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.780+0000 7f3434a00640 1 --2- 172.21.15.28:0/2332216464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f343014c300 0x7f34301439c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f341c00e840 tx=0x7f341c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:44.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.780+0000 7f342d600640 1 -- 172.21.15.28:0/2332216464 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c017070 con 0x7f343014c300 2024-04-26T20:57:44.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.780+0000 7f342d600640 1 -- 172.21.15.28:0/2332216464 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f341c00c940 con 0x7f343014c300 2024-04-26T20:57:44.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.780+0000 7f342d600640 1 -- 172.21.15.28:0/2332216464 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341c00cd10 con 0x7f343014c300 2024-04-26T20:57:44.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.780+0000 7f3436800640 1 -- 172.21.15.28:0/2332216464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3430162990 con 0x7f343014c300 2024-04-26T20:57:44.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.780+0000 7f3436800640 1 -- 172.21.15.28:0/2332216464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34300b5b60 con 0x7f343014c300 2024-04-26T20:57:44.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.781+0000 7f3436800640 1 -- 172.21.15.28:0/2332216464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f343006a360 con 0x7f343014c300 2024-04-26T20:57:44.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.782+0000 7f342d600640 1 -- 172.21.15.28:0/2332216464 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f341c01a070 con 0x7f343014c300 2024-04-26T20:57:44.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.783+0000 7f342d600640 1 --2- 172.21.15.28:0/2332216464 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f341006c8f0 0x7f341006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:44.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.783+0000 7f342d600640 1 -- 172.21.15.28:0/2332216464 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f341c08bb30 con 0x7f343014c300 2024-04-26T20:57:44.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.784+0000 7f3435400640 1 --2- 172.21.15.28:0/2332216464 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f341006c8f0 0x7f341006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:44.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.785+0000 7f3435400640 1 --2- 172.21.15.28:0/2332216464 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f341006c8f0 0x7f341006edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3424034000 tx=0x7f342402c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:44.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:44.788+0000 7f342d600640 1 -- 172.21.15.28:0/2332216464 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f341c059e40 con 0x7f343014c300 2024-04-26T20:57:45.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.006+0000 7f3436800640 1 -- 172.21.15.28:0/2332216464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3430074550 con 0x7f343014c300 2024-04-26T20:57:45.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.007+0000 7f342d600640 1 -- 172.21.15.28:0/2332216464 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f341c059740 con 0x7f343014c300 2024-04-26T20:57:45.009 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-04-26T20:57:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.010+0000 7f3436800640 1 -- 172.21.15.28:0/2332216464 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f341006c8f0 msgr2=0x7f341006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.010+0000 7f3436800640 1 --2- 172.21.15.28:0/2332216464 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f341006c8f0 0x7f341006edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3424034000 tx=0x7f342402c000 comp rx=0 tx=0).stop 2024-04-26T20:57:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.010+0000 7f3436800640 1 -- 172.21.15.28:0/2332216464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f343014c300 msgr2=0x7f34301439c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.010+0000 7f3436800640 1 --2- 172.21.15.28:0/2332216464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f343014c300 0x7f34301439c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f341c00e840 tx=0x7f341c00ed00 comp rx=0 tx=0).stop 2024-04-26T20:57:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.011+0000 7f3436800640 1 -- 172.21.15.28:0/2332216464 shutdown_connections 2024-04-26T20:57:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.011+0000 7f3436800640 1 --2- 172.21.15.28:0/2332216464 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f341006c8f0 0x7f341006edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.011+0000 7f3436800640 1 --2- 172.21.15.28:0/2332216464 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f343014ce60 0x7f3430143f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.011+0000 7f3436800640 1 --2- 172.21.15.28:0/2332216464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f343014c300 0x7f34301439c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.011+0000 7f3436800640 1 --2- 172.21.15.28:0/2332216464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f34300cf0a0 0x7f3430162450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.011+0000 7f3436800640 1 -- 172.21.15.28:0/2332216464 >> 172.21.15.28:0/2332216464 conn(0x7f34300d0a50 msgr2=0x7f34300d0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.011+0000 7f3436800640 1 -- 172.21.15.28:0/2332216464 shutdown_connections 2024-04-26T20:57:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.011+0000 7f3436800640 1 -- 172.21.15.28:0/2332216464 wait complete. 2024-04-26T20:57:45.032 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:57:45.032 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:57:45.086 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:57:45.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.297+0000 7fcf06e00640 1 Processor -- start 2024-04-26T20:57:45.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.297+0000 7fcf06e00640 1 -- start start 2024-04-26T20:57:45.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.299+0000 7fcf06e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcf00071e20 0x7fcf001610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:45.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.299+0000 7fcf06e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf00146440 con 0x7fcf00146670 2024-04-26T20:57:45.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.299+0000 7fcf06e00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf0006f010 con 0x7fcf00146060 2024-04-26T20:57:45.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.299+0000 7fcf06e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fcf0006f510 con 0x7fcf00071e20 2024-04-26T20:57:45.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.299+0000 7fcf05000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fcf00146670 0x7fcf000cb0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58428/0 (socket says 172.21.15.28:58428) 2024-04-26T20:57:45.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.299+0000 7fcf05000640 1 -- 172.21.15.28:0/4168505994 learned_addr learned my addr 172.21.15.28:0/4168505994 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:45.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.299+0000 7fcf06400640 1 --2- 172.21.15.28:0/4168505994 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcf00071e20 0x7fcf001610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:45.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.299+0000 7fceffe00640 1 -- 172.21.15.28:0/4168505994 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3023402105 0 0) 0x7fcf00146440 con 0x7fcf00146670 2024-04-26T20:57:45.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.299+0000 7fceffe00640 1 -- 172.21.15.28:0/4168505994 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcee8003520 con 0x7fcf00146670 2024-04-26T20:57:45.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.299+0000 7fceffe00640 1 -- 172.21.15.28:0/4168505994 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3674384902 0 0) 0x7fcee8003520 con 0x7fcf00146670 2024-04-26T20:57:45.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.299+0000 7fceffe00640 1 -- 172.21.15.28:0/4168505994 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcf00071e20 msgr2=0x7fcf001610c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:45.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.299+0000 7fceffe00640 1 --2- 172.21.15.28:0/4168505994 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcf00071e20 0x7fcf001610c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.299+0000 7fceffe00640 1 -- 172.21.15.28:0/4168505994 >> v1:172.21.15.103:6789/0 conn(0x7fcf00146060 legacy=0x7fcf001609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:45.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.299+0000 7fceffe00640 1 -- 172.21.15.28:0/4168505994 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf001617a0 con 0x7fcf00146670 2024-04-26T20:57:45.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.299+0000 7fceffe00640 1 -- 172.21.15.28:0/4168505994 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fceec001690 con 0x7fcf00146670 2024-04-26T20:57:45.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.299+0000 7fceffe00640 1 -- 172.21.15.28:0/4168505994 >> v1:172.21.15.28:6789/0 conn(0x7fcf00146670 legacy=0x7fcf000cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:45.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.299+0000 7fceffe00640 1 --2- 172.21.15.28:0/4168505994 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcf00071e20 0x7fcee80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:45.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.299+0000 7fceffe00640 1 --2- 172.21.15.28:0/4168505994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee8003fb0 0x7fcee8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:45.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.300+0000 7fceffe00640 1 --2- 172.21.15.28:0/4168505994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcee8004d40 0x7fcee80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:45.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.300+0000 7fceffe00640 1 -- 172.21.15.28:0/4168505994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0006f510 con 0x7fcee8004d40 2024-04-26T20:57:45.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.300+0000 7fceffe00640 1 -- 172.21.15.28:0/4168505994 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcee800bbb0 con 0x7fcee8003fb0 2024-04-26T20:57:45.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.300+0000 7fceffe00640 1 -- 172.21.15.28:0/4168505994 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fcee800bdb0 con 0x7fcf00071e20 2024-04-26T20:57:45.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.300+0000 7fcf06400640 1 --2- 172.21.15.28:0/4168505994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcee8004d40 0x7fcee80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:45.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.300+0000 7fcf05000640 1 --2- 172.21.15.28:0/4168505994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee8003fb0 0x7fcee8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:45.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.300+0000 7fcf06400640 1 -- 172.21.15.28:0/4168505994 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcf00071e20 msgr2=0x7fcee80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:45.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.300+0000 7fcf06400640 1 --2- 172.21.15.28:0/4168505994 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcf00071e20 0x7fcee80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.300+0000 7fcf06400640 1 -- 172.21.15.28:0/4168505994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee8003fb0 msgr2=0x7fcee8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:45.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.300+0000 7fcf06400640 1 --2- 172.21.15.28:0/4168505994 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcee8003fb0 0x7fcee8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.300+0000 7fcf06400640 1 -- 172.21.15.28:0/4168505994 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee8003520 con 0x7fcee8004d40 2024-04-26T20:57:45.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.301+0000 7fcf06400640 1 --2- 172.21.15.28:0/4168505994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcee8004d40 0x7fcee80051c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009fc0 tx=0x7fcee8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7bff7a3b9709ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:45.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.302+0000 7fceffe00640 1 -- 172.21.15.28:0/4168505994 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef8009c10 con 0x7fcee8004d40 2024-04-26T20:57:45.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.302+0000 7fceffe00640 1 -- 172.21.15.28:0/4168505994 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcef8036070 con 0x7fcee8004d40 2024-04-26T20:57:45.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.302+0000 7fceffe00640 1 -- 172.21.15.28:0/4168505994 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef802d070 con 0x7fcee8004d40 2024-04-26T20:57:45.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.302+0000 7fcf06e00640 1 -- 172.21.15.28:0/4168505994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcee8004d40 msgr2=0x7fcee80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:45.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.302+0000 7fcf06e00640 1 --2- 172.21.15.28:0/4168505994 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcee8004d40 0x7fcee80051c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fcef8009fc0 tx=0x7fcee8005830 comp rx=0 tx=0).stop 2024-04-26T20:57:45.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.302+0000 7fcf06400640 1 -- 172.21.15.28:0/4168505994 reap_dead start 2024-04-26T20:57:45.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.302+0000 7fcf06e00640 1 -- 172.21.15.28:0/4168505994 shutdown_connections 2024-04-26T20:57:45.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.302+0000 7fcf06e00640 1 -- 172.21.15.28:0/4168505994 >> 172.21.15.28:0/4168505994 conn(0x7fcf000d0a50 msgr2=0x7fcf000d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:45.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.302+0000 7fcf06e00640 1 -- 172.21.15.28:0/4168505994 shutdown_connections 2024-04-26T20:57:45.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.302+0000 7fcf06e00640 1 -- 172.21.15.28:0/4168505994 wait complete. 2024-04-26T20:57:45.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.304+0000 7fcf06e00640 1 Processor -- start 2024-04-26T20:57:45.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.304+0000 7fcf06e00640 1 -- start start 2024-04-26T20:57:45.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.305+0000 7fcf06e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf001f48d0 0x7fcf001f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:45.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.305+0000 7fcf06e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcf000b6030 0x7fcf0015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:45.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.305+0000 7fcf06e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf0015aef0 0x7fcf000c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:45.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.305+0000 7fcf06e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcf001610c0 con 0x7fcf001f48d0 2024-04-26T20:57:45.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.305+0000 7fcf06e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcf0006ff60 con 0x7fcf0015aef0 2024-04-26T20:57:45.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.305+0000 7fcf06e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fcf00161240 con 0x7fcf000b6030 2024-04-26T20:57:45.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.306+0000 7fcf05a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf001f48d0 0x7fcf001f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:45.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.306+0000 7fcf05a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf001f48d0 0x7fcf001f4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51158/0 (socket says 172.21.15.28:51158) 2024-04-26T20:57:45.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.306+0000 7fcf05a00640 1 -- 172.21.15.28:0/765875783 learned_addr learned my addr 172.21.15.28:0/765875783 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:45.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.306+0000 7fcf05a00640 1 -- 172.21.15.28:0/765875783 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcf000b6030 msgr2=0x7fcf0015a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:45.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.306+0000 7fcf05a00640 1 --2- 172.21.15.28:0/765875783 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcf000b6030 0x7fcf0015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.306+0000 7fcf05a00640 1 -- 172.21.15.28:0/765875783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf0015aef0 msgr2=0x7fcf000c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:45.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.306+0000 7fcf05a00640 1 --2- 172.21.15.28:0/765875783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf0015aef0 0x7fcf000c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.306+0000 7fcf05a00640 1 -- 172.21.15.28:0/765875783 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf000ce6c0 con 0x7fcf001f48d0 2024-04-26T20:57:45.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.307+0000 7fcf05a00640 1 --2- 172.21.15.28:0/765875783 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf001f48d0 0x7fcf001f4cb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcef400c400 tx=0x7fcef400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:45.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.307+0000 7fcefd600640 1 -- 172.21.15.28:0/765875783 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4017070 con 0x7fcf001f48d0 2024-04-26T20:57:45.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.307+0000 7fcefd600640 1 -- 172.21.15.28:0/765875783 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcef40052f0 con 0x7fcf001f48d0 2024-04-26T20:57:45.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.307+0000 7fcefd600640 1 -- 172.21.15.28:0/765875783 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4003830 con 0x7fcf001f48d0 2024-04-26T20:57:45.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.307+0000 7fcf06e00640 1 -- 172.21.15.28:0/765875783 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf0015b3b0 con 0x7fcf001f48d0 2024-04-26T20:57:45.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.307+0000 7fcf06e00640 1 -- 172.21.15.28:0/765875783 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf0015b690 con 0x7fcf001f48d0 2024-04-26T20:57:45.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.308+0000 7fcf06e00640 1 -- 172.21.15.28:0/765875783 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf0006a360 con 0x7fcf001f48d0 2024-04-26T20:57:45.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.309+0000 7fcefd600640 1 -- 172.21.15.28:0/765875783 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcef40039d0 con 0x7fcf001f48d0 2024-04-26T20:57:45.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.310+0000 7fcefd600640 1 --2- 172.21.15.28:0/765875783 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcee006c920 0x7fcee006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:45.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.310+0000 7fcefd600640 1 -- 172.21.15.28:0/765875783 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fcef408ad70 con 0x7fcf001f48d0 2024-04-26T20:57:45.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.311+0000 7fcf05000640 1 --2- 172.21.15.28:0/765875783 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcee006c920 0x7fcee006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:45.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.311+0000 7fcf05000640 1 --2- 172.21.15.28:0/765875783 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcee006c920 0x7fcee006ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fceec000ff0 tx=0x7fceec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:45.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.315+0000 7fcefd600640 1 -- 172.21.15.28:0/765875783 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcef4058ac0 con 0x7fcf001f48d0 2024-04-26T20:57:45.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.520+0000 7fcf06e00640 1 -- 172.21.15.28:0/765875783 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf0006e4e0 con 0x7fcf001f48d0 2024-04-26T20:57:45.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.522+0000 7fcefd600640 1 -- 172.21.15.28:0/765875783 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16831 (secure 0 0 0) 0x7fcef405cdc0 con 0x7fcf001f48d0 2024-04-26T20:57:45.523 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:45.524 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":23,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:57:37.996297+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:57:45.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.525+0000 7fcf06e00640 1 -- 172.21.15.28:0/765875783 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcee006c920 msgr2=0x7fcee006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:45.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.525+0000 7fcf06e00640 1 --2- 172.21.15.28:0/765875783 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcee006c920 0x7fcee006ede0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fceec000ff0 tx=0x7fceec009040 comp rx=0 tx=0).stop 2024-04-26T20:57:45.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.525+0000 7fcf06e00640 1 -- 172.21.15.28:0/765875783 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf001f48d0 msgr2=0x7fcf001f4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:45.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.525+0000 7fcf06e00640 1 --2- 172.21.15.28:0/765875783 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf001f48d0 0x7fcf001f4cb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcef400c400 tx=0x7fcef400c8c0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.525+0000 7fcf06e00640 1 -- 172.21.15.28:0/765875783 shutdown_connections 2024-04-26T20:57:45.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.525+0000 7fcf06e00640 1 --2- 172.21.15.28:0/765875783 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcee006c920 0x7fcee006ede0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.525+0000 7fcf06e00640 1 --2- 172.21.15.28:0/765875783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcf0015aef0 0x7fcf000c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.525+0000 7fcf06e00640 1 --2- 172.21.15.28:0/765875783 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcf000b6030 0x7fcf0015a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.525+0000 7fcf06e00640 1 --2- 172.21.15.28:0/765875783 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf001f48d0 0x7fcf001f4cb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.525+0000 7fcf06e00640 1 -- 172.21.15.28:0/765875783 >> 172.21.15.28:0/765875783 conn(0x7fcf000d0a50 msgr2=0x7fcf0015f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:45.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.525+0000 7fcf06e00640 1 -- 172.21.15.28:0/765875783 shutdown_connections 2024-04-26T20:57:45.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.525+0000 7fcf06e00640 1 -- 172.21.15.28:0/765875783 wait complete. 2024-04-26T20:57:45.544 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:56:48.688498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:56:50.117541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:57:45.545 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:57:45.732 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:57:45.740 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:57:45.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.756+0000 7f9924400640 1 Processor -- start 2024-04-26T20:57:45.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.756+0000 7f9924400640 1 -- start start 2024-04-26T20:57:45.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.757+0000 7f9924400640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f991c06fc10 0x7f991c073f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:45.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.757+0000 7f9924400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f991c073580 0x7f991c0ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:45.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.757+0000 7f9924400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f991c0ced20 con 0x7f991c0ce8d0 2024-04-26T20:57:45.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.757+0000 7f9924400640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f991c074490 con 0x7f991c06fc10 2024-04-26T20:57:45.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.757+0000 7f9924400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f991c15e310 con 0x7f991c073580 2024-04-26T20:57:45.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.757+0000 7f9922400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f991c0ce8d0 0x7f991c06e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58434/0 (socket says 172.21.15.28:58434) 2024-04-26T20:57:45.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.757+0000 7f9922400640 1 -- 172.21.15.28:0/275471008 learned_addr learned my addr 172.21.15.28:0/275471008 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:45.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.758+0000 7f9921a00640 1 --2- 172.21.15.28:0/275471008 >> v2:172.21.15.103:3300/0 conn(0x7f991c06fc10 0x7f991c073f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:45.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.758+0000 7f9921000640 1 --2- 172.21.15.28:0/275471008 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f991c073580 0x7f991c0ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:45.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.758+0000 7f991be00640 1 -- 172.21.15.28:0/275471008 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4288484487 0 0) 0x7f991c0ced20 con 0x7f991c0ce8d0 2024-04-26T20:57:45.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.758+0000 7f991be00640 1 -- 172.21.15.28:0/275471008 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9904003520 con 0x7f991c0ce8d0 2024-04-26T20:57:45.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.758+0000 7f9921a00640 1 -- 172.21.15.28:0/275471008 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f991c073580 msgr2=0x7f991c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:45.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.758+0000 7f9921a00640 1 --2- 172.21.15.28:0/275471008 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f991c073580 0x7f991c0ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.758+0000 7f9921a00640 1 -- 172.21.15.28:0/275471008 >> v1:172.21.15.28:6789/0 conn(0x7f991c0ce8d0 legacy=0x7f991c06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:45.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.758+0000 7f9921a00640 1 -- 172.21.15.28:0/275471008 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991c072aa0 con 0x7f991c06fc10 2024-04-26T20:57:45.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.759+0000 7f9921a00640 1 --2- 172.21.15.28:0/275471008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f991c06fc10 0x7f991c073f50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f99100097a0 tx=0x7f991002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8f3ced819561386 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:45.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.760+0000 7f991be00640 1 -- 172.21.15.28:0/275471008 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9910037070 con 0x7f991c06fc10 2024-04-26T20:57:45.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.760+0000 7f991be00640 1 -- 172.21.15.28:0/275471008 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f991002ae50 con 0x7f991c06fc10 2024-04-26T20:57:45.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.760+0000 7f991be00640 1 -- 172.21.15.28:0/275471008 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9910032660 con 0x7f991c06fc10 2024-04-26T20:57:45.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.760+0000 7f9924400640 1 -- 172.21.15.28:0/275471008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f991c06fc10 msgr2=0x7f991c073f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:45.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.760+0000 7f9924400640 1 --2- 172.21.15.28:0/275471008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f991c06fc10 0x7f991c073f50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f99100097a0 tx=0x7f991002a100 comp rx=0 tx=0).stop 2024-04-26T20:57:45.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.761+0000 7f9924400640 1 -- 172.21.15.28:0/275471008 shutdown_connections 2024-04-26T20:57:45.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.761+0000 7f9924400640 1 --2- 172.21.15.28:0/275471008 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f991c073580 0x7f991c0ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.761+0000 7f9924400640 1 --2- 172.21.15.28:0/275471008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f991c06fc10 0x7f991c073f50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.761+0000 7f9924400640 1 -- 172.21.15.28:0/275471008 >> 172.21.15.28:0/275471008 conn(0x7f991c06c770 msgr2=0x7f991c06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:45.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.761+0000 7f9924400640 1 -- 172.21.15.28:0/275471008 shutdown_connections 2024-04-26T20:57:45.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.761+0000 7f9924400640 1 -- 172.21.15.28:0/275471008 wait complete. 2024-04-26T20:57:45.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.763+0000 7f9924400640 1 Processor -- start 2024-04-26T20:57:45.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.763+0000 7f9924400640 1 -- start start 2024-04-26T20:57:45.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.764+0000 7f9924400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f991c06fc10 0x7f991c149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:45.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.764+0000 7f9924400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f991c073580 0x7f991c0aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:45.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.764+0000 7f9924400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f991c0ce8d0 0x7f991c0b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:45.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.764+0000 7f9924400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f991c15dfa0 con 0x7f991c0ce8d0 2024-04-26T20:57:45.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.764+0000 7f9924400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f991c073060 con 0x7f991c06fc10 2024-04-26T20:57:45.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.764+0000 7f9924400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f991c073a00 con 0x7f991c073580 2024-04-26T20:57:45.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.764+0000 7f9922400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f991c0ce8d0 0x7f991c0b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:45.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.764+0000 7f9922400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f991c0ce8d0 0x7f991c0b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51160/0 (socket says 172.21.15.28:51160) 2024-04-26T20:57:45.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.764+0000 7f9922400640 1 -- 172.21.15.28:0/2414631538 learned_addr learned my addr 172.21.15.28:0/2414631538 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:45.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.765+0000 7f9921a00640 1 --2- 172.21.15.28:0/2414631538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f991c06fc10 0x7f991c149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:45.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.765+0000 7f9921000640 1 --2- 172.21.15.28:0/2414631538 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f991c073580 0x7f991c0aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:45.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.765+0000 7f9922400640 1 -- 172.21.15.28:0/2414631538 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f991c073580 msgr2=0x7f991c0aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:45.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.765+0000 7f9922400640 1 --2- 172.21.15.28:0/2414631538 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f991c073580 0x7f991c0aeae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.765+0000 7f9922400640 1 -- 172.21.15.28:0/2414631538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f991c06fc10 msgr2=0x7f991c149640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:45.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.765+0000 7f9922400640 1 --2- 172.21.15.28:0/2414631538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f991c06fc10 0x7f991c149640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.765+0000 7f9922400640 1 -- 172.21.15.28:0/2414631538 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991c149b80 con 0x7f991c0ce8d0 2024-04-26T20:57:45.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.765+0000 7f9922400640 1 --2- 172.21.15.28:0/2414631538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f991c0ce8d0 0x7f991c0b2a60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f991400cb10 tx=0x7f991400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:45.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.766+0000 7f9919600640 1 -- 172.21.15.28:0/2414631538 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914017070 con 0x7f991c0ce8d0 2024-04-26T20:57:45.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.766+0000 7f9919600640 1 -- 172.21.15.28:0/2414631538 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f991400ac10 con 0x7f991c0ce8d0 2024-04-26T20:57:45.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.766+0000 7f9919600640 1 -- 172.21.15.28:0/2414631538 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914012400 con 0x7f991c0ce8d0 2024-04-26T20:57:45.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.766+0000 7f9924400640 1 -- 172.21.15.28:0/2414631538 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991c0abe40 con 0x7f991c0ce8d0 2024-04-26T20:57:45.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.766+0000 7f9924400640 1 -- 172.21.15.28:0/2414631538 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f991c0ac3c0 con 0x7f991c0ce8d0 2024-04-26T20:57:45.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.767+0000 7f9924400640 1 -- 172.21.15.28:0/2414631538 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f991c06a360 con 0x7f991c0ce8d0 2024-04-26T20:57:45.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.768+0000 7f9919600640 1 -- 172.21.15.28:0/2414631538 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f991401a070 con 0x7f991c0ce8d0 2024-04-26T20:57:45.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.769+0000 7f9919600640 1 --2- 172.21.15.28:0/2414631538 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f98fc06c8f0 0x7f98fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:45.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.769+0000 7f9919600640 1 -- 172.21.15.28:0/2414631538 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f991408b750 con 0x7f991c0ce8d0 2024-04-26T20:57:45.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.769+0000 7f9921a00640 1 --2- 172.21.15.28:0/2414631538 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f98fc06c8f0 0x7f98fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:45.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.770+0000 7f9921a00640 1 --2- 172.21.15.28:0/2414631538 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f98fc06c8f0 0x7f98fc06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9910034000 tx=0x7f991002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:45.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.774+0000 7f9919600640 1 -- 172.21.15.28:0/2414631538 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f991c149b80 con 0x7f991c0ce8d0 2024-04-26T20:57:45.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.978+0000 7f9924400640 1 -- 172.21.15.28:0/2414631538 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f991c157380 con 0x7f991c0ce8d0 2024-04-26T20:57:45.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.979+0000 7f9919600640 1 -- 172.21.15.28:0/2414631538 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f99140594a0 con 0x7f991c0ce8d0 2024-04-26T20:57:45.979 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:57:45.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.982+0000 7f9924400640 1 -- 172.21.15.28:0/2414631538 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f98fc06c8f0 msgr2=0x7f98fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:45.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.982+0000 7f9924400640 1 --2- 172.21.15.28:0/2414631538 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f98fc06c8f0 0x7f98fc06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9910034000 tx=0x7f991002e000 comp rx=0 tx=0).stop 2024-04-26T20:57:45.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.982+0000 7f9924400640 1 -- 172.21.15.28:0/2414631538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f991c0ce8d0 msgr2=0x7f991c0b2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:45.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.982+0000 7f9924400640 1 --2- 172.21.15.28:0/2414631538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f991c0ce8d0 0x7f991c0b2a60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f991400cb10 tx=0x7f991400a060 comp rx=0 tx=0).stop 2024-04-26T20:57:45.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.982+0000 7f9924400640 1 -- 172.21.15.28:0/2414631538 shutdown_connections 2024-04-26T20:57:45.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.982+0000 7f9924400640 1 --2- 172.21.15.28:0/2414631538 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f98fc06c8f0 0x7f98fc06edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.982+0000 7f9924400640 1 --2- 172.21.15.28:0/2414631538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f991c0ce8d0 0x7f991c0b2a60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.982+0000 7f9924400640 1 --2- 172.21.15.28:0/2414631538 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f991c073580 0x7f991c0aeae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.982+0000 7f9924400640 1 --2- 172.21.15.28:0/2414631538 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f991c06fc10 0x7f991c149640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:45.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.982+0000 7f9924400640 1 -- 172.21.15.28:0/2414631538 >> 172.21.15.28:0/2414631538 conn(0x7f991c06c770 msgr2=0x7f991c06cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:45.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.983+0000 7f9924400640 1 -- 172.21.15.28:0/2414631538 shutdown_connections 2024-04-26T20:57:45.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:45.983+0000 7f9924400640 1 -- 172.21.15.28:0/2414631538 wait complete. 2024-04-26T20:57:46.002 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:57:46.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.219+0000 7fc00fe00640 1 Processor -- start 2024-04-26T20:57:46.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.220+0000 7fc00fe00640 1 -- start start 2024-04-26T20:57:46.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.221+0000 7fc00fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc0101465c0 0x7fc01006e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:46.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.221+0000 7fc00fe00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc0100cf040 0x7fc0100d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:46.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.221+0000 7fc00fe00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0100b5ba0 con 0x7fc0100d1250 2024-04-26T20:57:46.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.221+0000 7fc00fe00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fc0100d16c0 con 0x7fc0101465c0 2024-04-26T20:57:46.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.221+0000 7fc00fe00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc0101469a0 con 0x7fc0100cf040 2024-04-26T20:57:46.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.222+0000 7fc00ea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc0101465c0 0x7fc01006e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:46.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.222+0000 7fc00f400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc0100d1250 0x7fc0101609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58438/0 (socket says 172.21.15.28:58438) 2024-04-26T20:57:46.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.222+0000 7fc00f400640 1 -- 172.21.15.28:0/2301679956 learned_addr learned my addr 172.21.15.28:0/2301679956 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:46.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.222+0000 7fc00ea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc0101465c0 0x7fc01006e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51162/0 (socket says 172.21.15.28:51162) 2024-04-26T20:57:46.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.222+0000 7fc00e000640 1 --2- 172.21.15.28:0/2301679956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc0100cf040 0x7fc0100d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:46.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.222+0000 7fc00ea00640 1 -- 172.21.15.28:0/2301679956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc0100cf040 msgr2=0x7fc0100d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:46.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.222+0000 7fc00ea00640 1 --2- 172.21.15.28:0/2301679956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc0100cf040 0x7fc0100d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.222+0000 7fc00ea00640 1 -- 172.21.15.28:0/2301679956 >> v1:172.21.15.28:6789/0 conn(0x7fc0100d1250 legacy=0x7fc0101609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:46.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.222+0000 7fc00d600640 1 -- 172.21.15.28:0/2301679956 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4117004481 0 0) 0x7fc008001110 con 0x7fc0100d1250 2024-04-26T20:57:46.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.223+0000 7fc00ea00640 1 -- 172.21.15.28:0/2301679956 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0100cf520 con 0x7fc0101465c0 2024-04-26T20:57:46.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.224+0000 7fc00ea00640 1 --2- 172.21.15.28:0/2301679956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0101465c0 0x7fc01006e930 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc000009d90 tx=0x7fc00002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b833329bf1b53f3a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:46.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.224+0000 7fc00d600640 1 -- 172.21.15.28:0/2301679956 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000031600 con 0x7fc0101465c0 2024-04-26T20:57:46.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.224+0000 7fc00d600640 1 -- 172.21.15.28:0/2301679956 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0000317a0 con 0x7fc0101465c0 2024-04-26T20:57:46.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.224+0000 7fc00d600640 1 -- 172.21.15.28:0/2301679956 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000031b70 con 0x7fc0101465c0 2024-04-26T20:57:46.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.224+0000 7fc00fe00640 1 -- 172.21.15.28:0/2301679956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0101465c0 msgr2=0x7fc01006e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:46.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.224+0000 7fc00fe00640 1 --2- 172.21.15.28:0/2301679956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0101465c0 0x7fc01006e930 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc000009d90 tx=0x7fc00002a5d0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.225+0000 7fc00fe00640 1 -- 172.21.15.28:0/2301679956 shutdown_connections 2024-04-26T20:57:46.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.225+0000 7fc00fe00640 1 --2- 172.21.15.28:0/2301679956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc0100cf040 0x7fc0100d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.225+0000 7fc00fe00640 1 --2- 172.21.15.28:0/2301679956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0101465c0 0x7fc01006e930 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.225+0000 7fc00fe00640 1 -- 172.21.15.28:0/2301679956 >> 172.21.15.28:0/2301679956 conn(0x7fc01006c530 msgr2=0x7fc01006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:46.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.225+0000 7fc00fe00640 1 -- 172.21.15.28:0/2301679956 shutdown_connections 2024-04-26T20:57:46.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.225+0000 7fc00fe00640 1 -- 172.21.15.28:0/2301679956 wait complete. 2024-04-26T20:57:46.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.226+0000 7fc00fe00640 1 Processor -- start 2024-04-26T20:57:46.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.227+0000 7fc00fe00640 1 -- start start 2024-04-26T20:57:46.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.227+0000 7fc00fe00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc0100cf040 0x7fc010150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:46.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.227+0000 7fc00fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0100d1250 0x7fc0100b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:46.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.227+0000 7fc00fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0101465c0 0x7fc0100b2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:46.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.227+0000 7fc00fe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc0100cfdc0 con 0x7fc0101465c0 2024-04-26T20:57:46.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.227+0000 7fc00fe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc0100cfae0 con 0x7fc0100d1250 2024-04-26T20:57:46.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.227+0000 7fc00fe00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc0100d00a0 con 0x7fc0100cf040 2024-04-26T20:57:46.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.228+0000 7fc00f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0101465c0 0x7fc0100b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:46.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.228+0000 7fc00f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0101465c0 0x7fc0100b2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51178/0 (socket says 172.21.15.28:51178) 2024-04-26T20:57:46.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.228+0000 7fc00f400640 1 -- 172.21.15.28:0/3026126745 learned_addr learned my addr 172.21.15.28:0/3026126745 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:46.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.228+0000 7fc00e000640 1 --2- 172.21.15.28:0/3026126745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0100d1250 0x7fc0100b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:46.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.228+0000 7fc00ea00640 1 --2- 172.21.15.28:0/3026126745 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc0100cf040 0x7fc010150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:46.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.228+0000 7fc00f400640 1 -- 172.21.15.28:0/3026126745 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc0100cf040 msgr2=0x7fc010150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:46.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.228+0000 7fc00f400640 1 --2- 172.21.15.28:0/3026126745 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc0100cf040 0x7fc010150650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.228+0000 7fc00f400640 1 -- 172.21.15.28:0/3026126745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0100d1250 msgr2=0x7fc0100b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:46.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.228+0000 7fc00f400640 1 --2- 172.21.15.28:0/3026126745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0100d1250 0x7fc0100b4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.228+0000 7fc00f400640 1 -- 172.21.15.28:0/3026126745 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0100b31a0 con 0x7fc0101465c0 2024-04-26T20:57:46.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.229+0000 7fc00f400640 1 --2- 172.21.15.28:0/3026126745 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0101465c0 0x7fc0100b2700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc00800c7b0 tx=0x7fc00800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:46.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.229+0000 7fc007400640 1 -- 172.21.15.28:0/3026126745 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc008018070 con 0x7fc0101465c0 2024-04-26T20:57:46.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.229+0000 7fc007400640 1 -- 172.21.15.28:0/3026126745 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0080070d0 con 0x7fc0101465c0 2024-04-26T20:57:46.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.229+0000 7fc007400640 1 -- 172.21.15.28:0/3026126745 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0080074c0 con 0x7fc0101465c0 2024-04-26T20:57:46.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.229+0000 7fc00fe00640 1 -- 172.21.15.28:0/3026126745 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0100b3480 con 0x7fc0101465c0 2024-04-26T20:57:46.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.229+0000 7fc00fe00640 1 -- 172.21.15.28:0/3026126745 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0100b3a40 con 0x7fc0101465c0 2024-04-26T20:57:46.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.230+0000 7fc00fe00640 1 -- 172.21.15.28:0/3026126745 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc01006a250 con 0x7fc0101465c0 2024-04-26T20:57:46.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.231+0000 7fc007400640 1 -- 172.21.15.28:0/3026126745 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0080047e0 con 0x7fc0101465c0 2024-04-26T20:57:46.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.232+0000 7fc007400640 1 --2- 172.21.15.28:0/3026126745 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbfec06c8f0 0x7fbfec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:46.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.232+0000 7fc007400640 1 -- 172.21.15.28:0/3026126745 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fc00808afc0 con 0x7fc0101465c0 2024-04-26T20:57:46.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.233+0000 7fc00ea00640 1 --2- 172.21.15.28:0/3026126745 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbfec06c8f0 0x7fbfec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:46.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.233+0000 7fc00ea00640 1 --2- 172.21.15.28:0/3026126745 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbfec06c8f0 0x7fbfec06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc000034000 tx=0x7fc000033040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:46.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.237+0000 7fc007400640 1 -- 172.21.15.28:0/3026126745 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc0100b5ba0 con 0x7fc0101465c0 2024-04-26T20:57:46.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.444+0000 7fc00fe00640 1 -- 172.21.15.28:0/3026126745 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc01015d0b0 con 0x7fc0101465c0 2024-04-26T20:57:46.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.445+0000 7fc007400640 1 -- 172.21.15.28:0/3026126745 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fc008059240 con 0x7fc0101465c0 2024-04-26T20:57:46.445 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-04-26T20:57:46.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.448+0000 7fc00fe00640 1 -- 172.21.15.28:0/3026126745 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbfec06c8f0 msgr2=0x7fbfec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:46.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.448+0000 7fc00fe00640 1 --2- 172.21.15.28:0/3026126745 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbfec06c8f0 0x7fbfec06edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc000034000 tx=0x7fc000033040 comp rx=0 tx=0).stop 2024-04-26T20:57:46.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.448+0000 7fc00fe00640 1 -- 172.21.15.28:0/3026126745 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0101465c0 msgr2=0x7fc0100b2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:46.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.448+0000 7fc00fe00640 1 --2- 172.21.15.28:0/3026126745 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0101465c0 0x7fc0100b2700 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc00800c7b0 tx=0x7fc00800cc70 comp rx=0 tx=0).stop 2024-04-26T20:57:46.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.448+0000 7fc00fe00640 1 -- 172.21.15.28:0/3026126745 shutdown_connections 2024-04-26T20:57:46.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.448+0000 7fc00fe00640 1 --2- 172.21.15.28:0/3026126745 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbfec06c8f0 0x7fbfec06edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.448+0000 7fc00fe00640 1 --2- 172.21.15.28:0/3026126745 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc0101465c0 0x7fc0100b2700 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.448+0000 7fc00fe00640 1 --2- 172.21.15.28:0/3026126745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0100d1250 0x7fc0100b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.448+0000 7fc00fe00640 1 --2- 172.21.15.28:0/3026126745 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc0100cf040 0x7fc010150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.448+0000 7fc00fe00640 1 -- 172.21.15.28:0/3026126745 >> 172.21.15.28:0/3026126745 conn(0x7fc01006c530 msgr2=0x7fc0100b0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:46.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.448+0000 7fc00fe00640 1 -- 172.21.15.28:0/3026126745 shutdown_connections 2024-04-26T20:57:46.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.448+0000 7fc00fe00640 1 -- 172.21.15.28:0/3026126745 wait complete. 2024-04-26T20:57:46.467 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:57:46.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.671+0000 7f1233e00640 1 Processor -- start 2024-04-26T20:57:46.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.672+0000 7f1233e00640 1 -- start start 2024-04-26T20:57:46.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.673+0000 7f1233e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f12340b0210 0x7f123415f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:46.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.673+0000 7f1233e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12340aef80 con 0x7f123415c8a0 2024-04-26T20:57:46.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.673+0000 7f1233e00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12340af260 con 0x7f12340d1380 2024-04-26T20:57:46.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.673+0000 7f1233e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f12340aed60 con 0x7f12340b0210 2024-04-26T20:57:46.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.674+0000 7f1232a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f123415c8a0 0x7f12340d0c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58448/0 (socket says 172.21.15.28:58448) 2024-04-26T20:57:46.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.674+0000 7f1232a00640 1 -- 172.21.15.28:0/2668572975 learned_addr learned my addr 172.21.15.28:0/2668572975 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:46.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.674+0000 7f122be00640 1 --2- 172.21.15.28:0/2668572975 >> v2:172.21.15.103:3300/0 conn(0x7f12340b0210 0x7f123415f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:46.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.674+0000 7f1232000640 1 -- 172.21.15.28:0/2668572975 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3788537032 0 0) 0x7f12340aef80 con 0x7f123415c8a0 2024-04-26T20:57:46.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.674+0000 7f1232000640 1 -- 172.21.15.28:0/2668572975 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1218003520 con 0x7f123415c8a0 2024-04-26T20:57:46.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.675+0000 7f1232000640 1 -- 172.21.15.28:0/2668572975 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 864579459 0 0) 0x7f1218003520 con 0x7f123415c8a0 2024-04-26T20:57:46.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.675+0000 7f1232000640 1 -- 172.21.15.28:0/2668572975 >> v2:172.21.15.103:3300/0 conn(0x7f12340b0210 msgr2=0x7f123415f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:46.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.675+0000 7f1232000640 1 --2- 172.21.15.28:0/2668572975 >> v2:172.21.15.103:3300/0 conn(0x7f12340b0210 0x7f123415f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.675+0000 7f1232000640 1 -- 172.21.15.28:0/2668572975 >> v1:172.21.15.103:6789/0 conn(0x7f12340d1380 legacy=0x7f1234072210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:46.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.675+0000 7f1232000640 1 -- 172.21.15.28:0/2668572975 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12340af540 con 0x7f123415c8a0 2024-04-26T20:57:46.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.675+0000 7f1232000640 1 -- 172.21.15.28:0/2668572975 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f1224001670 con 0x7f123415c8a0 2024-04-26T20:57:46.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.675+0000 7f1232000640 1 -- 172.21.15.28:0/2668572975 >> v1:172.21.15.28:6789/0 conn(0x7f123415c8a0 legacy=0x7f12340d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:46.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.675+0000 7f1232000640 1 --2- 172.21.15.28:0/2668572975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1218003a00 0x7f1218003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:46.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.675+0000 7f1232000640 1 --2- 172.21.15.28:0/2668572975 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1218004800 0x7f1218004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:46.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.675+0000 7f1232000640 1 --2- 172.21.15.28:0/2668572975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12180051c0 0x7f121800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:46.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.675+0000 7f1232000640 1 -- 172.21.15.28:0/2668572975 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f12340aed60 con 0x7f1218003a00 2024-04-26T20:57:46.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.675+0000 7f1232000640 1 -- 172.21.15.28:0/2668572975 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f121800c0c0 con 0x7f12180051c0 2024-04-26T20:57:46.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.675+0000 7f1232000640 1 -- 172.21.15.28:0/2668572975 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f121800c2c0 con 0x7f1218004800 2024-04-26T20:57:46.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.675+0000 7f1232a00640 1 --2- 172.21.15.28:0/2668572975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1218003a00 0x7f1218003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:46.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.676+0000 7f1233400640 1 --2- 172.21.15.28:0/2668572975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12180051c0 0x7f121800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:46.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.676+0000 7f1232a00640 1 -- 172.21.15.28:0/2668572975 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1218004800 msgr2=0x7f1218004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:46.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.676+0000 7f1232a00640 1 --2- 172.21.15.28:0/2668572975 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1218004800 0x7f1218004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.676+0000 7f1232a00640 1 -- 172.21.15.28:0/2668572975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12180051c0 msgr2=0x7f121800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:46.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.676+0000 7f1232a00640 1 --2- 172.21.15.28:0/2668572975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12180051c0 0x7f121800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.676+0000 7f1232a00640 1 -- 172.21.15.28:0/2668572975 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1218003520 con 0x7f1218003a00 2024-04-26T20:57:46.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.676+0000 7f1233400640 1 --2- 172.21.15.28:0/2668572975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12180051c0 0x7f121800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:46.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.676+0000 7f1233400640 1 -- 172.21.15.28:0/2668572975 reap_dead start 2024-04-26T20:57:46.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.677+0000 7f1232a00640 1 --2- 172.21.15.28:0/2668572975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1218003a00 0x7f1218003e20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f122400c0e0 tx=0x7f1218005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8009f5351b5d4f33 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:46.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.677+0000 7f1232000640 1 -- 172.21.15.28:0/2668572975 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224007c20 con 0x7f1218003a00 2024-04-26T20:57:46.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.677+0000 7f1232000640 1 -- 172.21.15.28:0/2668572975 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1224005af0 con 0x7f1218003a00 2024-04-26T20:57:46.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.677+0000 7f1232000640 1 -- 172.21.15.28:0/2668572975 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224007070 con 0x7f1218003a00 2024-04-26T20:57:46.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.678+0000 7f1233e00640 1 -- 172.21.15.28:0/2668572975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1218003a00 msgr2=0x7f1218003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:46.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.678+0000 7f1233e00640 1 --2- 172.21.15.28:0/2668572975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1218003a00 0x7f1218003e20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f122400c0e0 tx=0x7f1218005830 comp rx=0 tx=0).stop 2024-04-26T20:57:46.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.678+0000 7f1233e00640 1 -- 172.21.15.28:0/2668572975 shutdown_connections 2024-04-26T20:57:46.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.678+0000 7f1233e00640 1 --2- 172.21.15.28:0/2668572975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1218003a00 0x7f1218003e20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.678+0000 7f1233e00640 1 -- 172.21.15.28:0/2668572975 >> 172.21.15.28:0/2668572975 conn(0x7f123406c530 msgr2=0x7f123406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:46.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.678+0000 7f1233e00640 1 -- 172.21.15.28:0/2668572975 shutdown_connections 2024-04-26T20:57:46.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.678+0000 7f1233e00640 1 -- 172.21.15.28:0/2668572975 wait complete. 2024-04-26T20:57:46.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.680+0000 7f1233e00640 1 Processor -- start 2024-04-26T20:57:46.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.680+0000 7f1233e00640 1 -- start start 2024-04-26T20:57:46.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.681+0000 7f1233e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1218003a00 0x7f123415a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:46.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.681+0000 7f1233e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f123415e650 0x7f123415ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:46.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.681+0000 7f1233e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12340b0690 0x7f12340b0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:46.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.681+0000 7f1233e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f123415cc80 con 0x7f123415e650 2024-04-26T20:57:46.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.681+0000 7f1233e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f12340af8e0 con 0x7f12340b0690 2024-04-26T20:57:46.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.681+0000 7f1233e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f123415ce00 con 0x7f1218003a00 2024-04-26T20:57:46.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.681+0000 7f122be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f123415e650 0x7f123415ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:46.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.681+0000 7f122be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f123415e650 0x7f123415ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51196/0 (socket says 172.21.15.28:51196) 2024-04-26T20:57:46.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.681+0000 7f122be00640 1 -- 172.21.15.28:0/1413021149 learned_addr learned my addr 172.21.15.28:0/1413021149 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:46.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.682+0000 7f122be00640 1 -- 172.21.15.28:0/1413021149 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1218003a00 msgr2=0x7f123415a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:46.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.682+0000 7f122be00640 1 --2- 172.21.15.28:0/1413021149 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1218003a00 0x7f123415a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.682+0000 7f122be00640 1 -- 172.21.15.28:0/1413021149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12340b0690 msgr2=0x7f12340b0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:46.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.682+0000 7f1233400640 1 --2- 172.21.15.28:0/1413021149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12340b0690 0x7f12340b0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:46.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.682+0000 7f122be00640 1 --2- 172.21.15.28:0/1413021149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12340b0690 0x7f12340b0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.682+0000 7f122be00640 1 -- 172.21.15.28:0/1413021149 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123415af40 con 0x7f123415e650 2024-04-26T20:57:46.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.682+0000 7f122be00640 1 --2- 172.21.15.28:0/1413021149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f123415e650 0x7f123415ead0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f121c00e820 tx=0x7f121c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:46.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.682+0000 7f122aa00640 1 -- 172.21.15.28:0/1413021149 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c017070 con 0x7f123415e650 2024-04-26T20:57:46.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.683+0000 7f122aa00640 1 -- 172.21.15.28:0/1413021149 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f121c00c940 con 0x7f123415e650 2024-04-26T20:57:46.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.683+0000 7f122aa00640 1 -- 172.21.15.28:0/1413021149 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c00cd30 con 0x7f123415e650 2024-04-26T20:57:46.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.683+0000 7f1233e00640 1 -- 172.21.15.28:0/1413021149 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f123415f010 con 0x7f123415e650 2024-04-26T20:57:46.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.683+0000 7f1233e00640 1 -- 172.21.15.28:0/1413021149 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f123415b4c0 con 0x7f123415e650 2024-04-26T20:57:46.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.684+0000 7f1233e00640 1 -- 172.21.15.28:0/1413021149 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f123406a2c0 con 0x7f123415e650 2024-04-26T20:57:46.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.685+0000 7f122aa00640 1 -- 172.21.15.28:0/1413021149 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f121c01a070 con 0x7f123415e650 2024-04-26T20:57:46.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.686+0000 7f122aa00640 1 --2- 172.21.15.28:0/1413021149 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f121006c8f0 0x7f121006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:46.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.686+0000 7f122aa00640 1 -- 172.21.15.28:0/1413021149 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f121c08b990 con 0x7f123415e650 2024-04-26T20:57:46.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.687+0000 7f1232a00640 1 --2- 172.21.15.28:0/1413021149 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f121006c8f0 0x7f121006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:46.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.688+0000 7f1232a00640 1 --2- 172.21.15.28:0/1413021149 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f121006c8f0 0x7f121006edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1224032000 tx=0x7f1224007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:46.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.692+0000 7f122aa00640 1 -- 172.21.15.28:0/1413021149 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f121c059dd0 con 0x7f123415e650 2024-04-26T20:57:46.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.895+0000 7f1233e00640 1 -- 172.21.15.28:0/1413021149 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f123406fd80 con 0x7f123415e650 2024-04-26T20:57:46.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.896+0000 7f122aa00640 1 -- 172.21.15.28:0/1413021149 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f121c0596d0 con 0x7f123415e650 2024-04-26T20:57:46.897 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-04-26T20:57:46.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.898+0000 7f1233e00640 1 -- 172.21.15.28:0/1413021149 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f121006c8f0 msgr2=0x7f121006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:46.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.898+0000 7f1233e00640 1 --2- 172.21.15.28:0/1413021149 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f121006c8f0 0x7f121006edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1224032000 tx=0x7f1224007000 comp rx=0 tx=0).stop 2024-04-26T20:57:46.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.899+0000 7f1233e00640 1 -- 172.21.15.28:0/1413021149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f123415e650 msgr2=0x7f123415ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:46.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.899+0000 7f1233e00640 1 --2- 172.21.15.28:0/1413021149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f123415e650 0x7f123415ead0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f121c00e820 tx=0x7f121c00ece0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.899+0000 7f1233e00640 1 -- 172.21.15.28:0/1413021149 shutdown_connections 2024-04-26T20:57:46.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.899+0000 7f1233e00640 1 --2- 172.21.15.28:0/1413021149 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f121006c8f0 0x7f121006edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.899+0000 7f1233e00640 1 --2- 172.21.15.28:0/1413021149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f12340b0690 0x7f12340b0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.899+0000 7f1233e00640 1 --2- 172.21.15.28:0/1413021149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f123415e650 0x7f123415ead0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.899+0000 7f1233e00640 1 --2- 172.21.15.28:0/1413021149 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1218003a00 0x7f123415a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:46.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.899+0000 7f1233e00640 1 -- 172.21.15.28:0/1413021149 >> 172.21.15.28:0/1413021149 conn(0x7f123406c530 msgr2=0x7f123406f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:46.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.899+0000 7f1233e00640 1 -- 172.21.15.28:0/1413021149 shutdown_connections 2024-04-26T20:57:46.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:46.899+0000 7f1233e00640 1 -- 172.21.15.28:0/1413021149 wait complete. 2024-04-26T20:57:46.917 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:57:47.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.124+0000 7fe6e2a00640 1 Processor -- start 2024-04-26T20:57:47.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.124+0000 7fe6e2a00640 1 -- start start 2024-04-26T20:57:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.126+0000 7fe6e2a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6dc146060 0x7fe6dc071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.126+0000 7fe6e2a00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe6dc146670 0x7fe6dc1609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.126+0000 7fe6e2a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6dc146440 con 0x7fe6dc160ef0 2024-04-26T20:57:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.126+0000 7fe6e2a00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fe6dc06f510 con 0x7fe6dc146670 2024-04-26T20:57:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.126+0000 7fe6e2a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe6dc06f010 con 0x7fe6dc146060 2024-04-26T20:57:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.126+0000 7fe6e2000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe6dc160ef0 0x7fe6dc06de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58454/0 (socket says 172.21.15.28:58454) 2024-04-26T20:57:47.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.126+0000 7fe6e2000640 1 -- 172.21.15.28:0/3159641581 learned_addr learned my addr 172.21.15.28:0/3159641581 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:47.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.127+0000 7fe6e0c00640 1 --2- 172.21.15.28:0/3159641581 >> v2:172.21.15.103:3300/0 conn(0x7fe6dc146670 0x7fe6dc1609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:47.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.127+0000 7fe6e1600640 1 --2- 172.21.15.28:0/3159641581 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6dc146060 0x7fe6dc071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:47.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.127+0000 7fe6dbe00640 1 -- 172.21.15.28:0/3159641581 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 245879042 0 0) 0x7fe6dc146440 con 0x7fe6dc160ef0 2024-04-26T20:57:47.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.127+0000 7fe6dbe00640 1 -- 172.21.15.28:0/3159641581 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6c4003520 con 0x7fe6dc160ef0 2024-04-26T20:57:47.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.127+0000 7fe6dbe00640 1 -- 172.21.15.28:0/3159641581 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4119241023 0 0) 0x7fe6c4003520 con 0x7fe6dc160ef0 2024-04-26T20:57:47.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.127+0000 7fe6dbe00640 1 -- 172.21.15.28:0/3159641581 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6dc146060 msgr2=0x7fe6dc071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:47.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.127+0000 7fe6dbe00640 1 --2- 172.21.15.28:0/3159641581 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6dc146060 0x7fe6dc071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:47.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.127+0000 7fe6dbe00640 1 -- 172.21.15.28:0/3159641581 >> v2:172.21.15.103:3300/0 conn(0x7fe6dc146670 msgr2=0x7fe6dc1609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:47.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.127+0000 7fe6dbe00640 1 --2- 172.21.15.28:0/3159641581 >> v2:172.21.15.103:3300/0 conn(0x7fe6dc146670 0x7fe6dc1609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:47.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.127+0000 7fe6dbe00640 1 -- 172.21.15.28:0/3159641581 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6dc1617a0 con 0x7fe6dc160ef0 2024-04-26T20:57:47.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.127+0000 7fe6dbe00640 1 -- 172.21.15.28:0/3159641581 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fe6d4001670 con 0x7fe6dc160ef0 2024-04-26T20:57:47.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.127+0000 7fe6dbe00640 1 -- 172.21.15.28:0/3159641581 >> v1:172.21.15.28:6789/0 conn(0x7fe6dc160ef0 legacy=0x7fe6dc06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:47.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.127+0000 7fe6dbe00640 1 --2- 172.21.15.28:0/3159641581 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6c40039b0 0x7fe6c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:47.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.127+0000 7fe6dbe00640 1 --2- 172.21.15.28:0/3159641581 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6dc146060 0x7fe6c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.127+0000 7fe6dbe00640 1 --2- 172.21.15.28:0/3159641581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6c4004d40 0x7fe6c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.128+0000 7fe6dbe00640 1 -- 172.21.15.28:0/3159641581 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc06f510 con 0x7fe6c40039b0 2024-04-26T20:57:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.128+0000 7fe6dbe00640 1 -- 172.21.15.28:0/3159641581 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc06f010 con 0x7fe6c4004d40 2024-04-26T20:57:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.128+0000 7fe6dbe00640 1 -- 172.21.15.28:0/3159641581 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe6c400b9f0 con 0x7fe6dc146060 2024-04-26T20:57:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.128+0000 7fe6e1600640 1 --2- 172.21.15.28:0/3159641581 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6c40039b0 0x7fe6c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.128+0000 7fe6e2000640 1 --2- 172.21.15.28:0/3159641581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6c4004d40 0x7fe6c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.128+0000 7fe6e1600640 1 -- 172.21.15.28:0/3159641581 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6dc146060 msgr2=0x7fe6c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.128+0000 7fe6e1600640 1 --2- 172.21.15.28:0/3159641581 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6dc146060 0x7fe6c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.128+0000 7fe6e1600640 1 -- 172.21.15.28:0/3159641581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6c4004d40 msgr2=0x7fe6c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.128+0000 7fe6e1600640 1 --2- 172.21.15.28:0/3159641581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6c4004d40 0x7fe6c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.128+0000 7fe6e1600640 1 -- 172.21.15.28:0/3159641581 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c4003520 con 0x7fe6c40039b0 2024-04-26T20:57:47.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.128+0000 7fe6e2000640 1 --2- 172.21.15.28:0/3159641581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6c4004d40 0x7fe6c40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.129+0000 7fe6e1600640 1 --2- 172.21.15.28:0/3159641581 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6c40039b0 0x7fe6c4003dd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009fc0 tx=0x7fe6c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91091c1d160757b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.129+0000 7fe6dbe00640 1 -- 172.21.15.28:0/3159641581 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0009c10 con 0x7fe6c40039b0 2024-04-26T20:57:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.130+0000 7fe6dbe00640 1 -- 172.21.15.28:0/3159641581 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d0036070 con 0x7fe6c40039b0 2024-04-26T20:57:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.130+0000 7fe6dbe00640 1 -- 172.21.15.28:0/3159641581 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d002d070 con 0x7fe6c40039b0 2024-04-26T20:57:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.130+0000 7fe6e2a00640 1 -- 172.21.15.28:0/3159641581 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6c40039b0 msgr2=0x7fe6c4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.130+0000 7fe6e2a00640 1 --2- 172.21.15.28:0/3159641581 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6c40039b0 0x7fe6c4003dd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009fc0 tx=0x7fe6c4005830 comp rx=0 tx=0).stop 2024-04-26T20:57:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.130+0000 7fe6e2000640 1 -- 172.21.15.28:0/3159641581 reap_dead start 2024-04-26T20:57:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.130+0000 7fe6e2a00640 1 -- 172.21.15.28:0/3159641581 shutdown_connections 2024-04-26T20:57:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.130+0000 7fe6e2a00640 1 -- 172.21.15.28:0/3159641581 >> 172.21.15.28:0/3159641581 conn(0x7fe6dc0d0a50 msgr2=0x7fe6dc0d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.130+0000 7fe6e2a00640 1 -- 172.21.15.28:0/3159641581 shutdown_connections 2024-04-26T20:57:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.130+0000 7fe6e2a00640 1 -- 172.21.15.28:0/3159641581 wait complete. 2024-04-26T20:57:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.132+0000 7fe6e2a00640 1 Processor -- start 2024-04-26T20:57:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.132+0000 7fe6e2a00640 1 -- start start 2024-04-26T20:57:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.133+0000 7fe6e2a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6c40039b0 0x7fe6dc0b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:47.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.133+0000 7fe6e2a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6dc15d060 0x7fe6dc141b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:47.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.133+0000 7fe6e2a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6dc1420c0 0x7fe6dc13f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:47.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.133+0000 7fe6e2a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc0cc0b0 con 0x7fe6dc15d060 2024-04-26T20:57:47.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.133+0000 7fe6e2a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc06ff60 con 0x7fe6c40039b0 2024-04-26T20:57:47.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.133+0000 7fe6e2a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe6dc0cc230 con 0x7fe6dc1420c0 2024-04-26T20:57:47.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.133+0000 7fe6e0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6dc15d060 0x7fe6dc141b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:47.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.134+0000 7fe6e0c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6dc15d060 0x7fe6dc141b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51210/0 (socket says 172.21.15.28:51210) 2024-04-26T20:57:47.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.134+0000 7fe6e0c00640 1 -- 172.21.15.28:0/1069982318 learned_addr learned my addr 172.21.15.28:0/1069982318 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:47.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.134+0000 7fe6e1600640 1 --2- 172.21.15.28:0/1069982318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6c40039b0 0x7fe6dc0b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:47.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.134+0000 7fe6e0c00640 1 -- 172.21.15.28:0/1069982318 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6dc1420c0 msgr2=0x7fe6dc13f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:47.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.134+0000 7fe6e2000640 1 --2- 172.21.15.28:0/1069982318 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6dc1420c0 0x7fe6dc13f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:47.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.134+0000 7fe6e0c00640 1 --2- 172.21.15.28:0/1069982318 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6dc1420c0 0x7fe6dc13f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:47.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.134+0000 7fe6e0c00640 1 -- 172.21.15.28:0/1069982318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6c40039b0 msgr2=0x7fe6dc0b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:47.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.134+0000 7fe6e0c00640 1 --2- 172.21.15.28:0/1069982318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6c40039b0 0x7fe6dc0b4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:47.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.134+0000 7fe6e0c00640 1 -- 172.21.15.28:0/1069982318 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6dc0b51b0 con 0x7fe6dc15d060 2024-04-26T20:57:47.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.134+0000 7fe6e1600640 1 --2- 172.21.15.28:0/1069982318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6c40039b0 0x7fe6dc0b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:47.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.134+0000 7fe6e2000640 1 --2- 172.21.15.28:0/1069982318 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6dc1420c0 0x7fe6dc13f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:47.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.134+0000 7fe6e0c00640 1 --2- 172.21.15.28:0/1069982318 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6dc15d060 0x7fe6dc141b80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800e890 tx=0x7fe6c800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:47.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.135+0000 7fe6d9600640 1 -- 172.21.15.28:0/1069982318 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8017070 con 0x7fe6dc15d060 2024-04-26T20:57:47.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.135+0000 7fe6d9600640 1 -- 172.21.15.28:0/1069982318 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6c800c940 con 0x7fe6dc15d060 2024-04-26T20:57:47.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.135+0000 7fe6d9600640 1 -- 172.21.15.28:0/1069982318 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c800cd10 con 0x7fe6dc15d060 2024-04-26T20:57:47.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.135+0000 7fe6e2a00640 1 -- 172.21.15.28:0/1069982318 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6dc0b3bd0 con 0x7fe6dc15d060 2024-04-26T20:57:47.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.135+0000 7fe6e2a00640 1 -- 172.21.15.28:0/1069982318 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6dc0b4190 con 0x7fe6dc15d060 2024-04-26T20:57:47.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.136+0000 7fe6e2a00640 1 -- 172.21.15.28:0/1069982318 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6dc071930 con 0x7fe6dc15d060 2024-04-26T20:57:47.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.137+0000 7fe6d9600640 1 -- 172.21.15.28:0/1069982318 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe6c801a070 con 0x7fe6dc15d060 2024-04-26T20:57:47.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.138+0000 7fe6d9600640 1 --2- 172.21.15.28:0/1069982318 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe6bc06c920 0x7fe6bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:47.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.138+0000 7fe6d9600640 1 -- 172.21.15.28:0/1069982318 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fe6c808ba10 con 0x7fe6dc15d060 2024-04-26T20:57:47.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.138+0000 7fe6e1600640 1 --2- 172.21.15.28:0/1069982318 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe6bc06c920 0x7fe6bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:47.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.139+0000 7fe6e1600640 1 --2- 172.21.15.28:0/1069982318 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe6bc06c920 0x7fe6bc06ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0035000 tx=0x7fe6d002f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:47.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.142+0000 7fe6d9600640 1 -- 172.21.15.28:0/1069982318 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe6dc071930 con 0x7fe6dc15d060 2024-04-26T20:57:47.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.359+0000 7fe6e2a00640 1 -- 172.21.15.28:0/1069982318 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe6dc06a360 con 0x7fe6dc15d060 2024-04-26T20:57:47.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.360+0000 7fe6d9600640 1 -- 172.21.15.28:0/1069982318 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fe6c8059db0 con 0x7fe6dc15d060 2024-04-26T20:57:47.360 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-04-26T20:57:47.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.362+0000 7fe6e2a00640 1 -- 172.21.15.28:0/1069982318 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe6bc06c920 msgr2=0x7fe6bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:47.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.362+0000 7fe6e2a00640 1 --2- 172.21.15.28:0/1069982318 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe6bc06c920 0x7fe6bc06ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0035000 tx=0x7fe6d002f000 comp rx=0 tx=0).stop 2024-04-26T20:57:47.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.362+0000 7fe6e2a00640 1 -- 172.21.15.28:0/1069982318 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6dc15d060 msgr2=0x7fe6dc141b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:47.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.362+0000 7fe6e2a00640 1 --2- 172.21.15.28:0/1069982318 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6dc15d060 0x7fe6dc141b80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800e890 tx=0x7fe6c800ed50 comp rx=0 tx=0).stop 2024-04-26T20:57:47.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.363+0000 7fe6e2a00640 1 -- 172.21.15.28:0/1069982318 shutdown_connections 2024-04-26T20:57:47.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.363+0000 7fe6e2a00640 1 --2- 172.21.15.28:0/1069982318 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe6bc06c920 0x7fe6bc06ede0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:47.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.363+0000 7fe6e2a00640 1 --2- 172.21.15.28:0/1069982318 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6dc1420c0 0x7fe6dc13f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:47.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.363+0000 7fe6e2a00640 1 --2- 172.21.15.28:0/1069982318 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6dc15d060 0x7fe6dc141b80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:47.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.363+0000 7fe6e2a00640 1 --2- 172.21.15.28:0/1069982318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6c40039b0 0x7fe6dc0b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:47.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.363+0000 7fe6e2a00640 1 -- 172.21.15.28:0/1069982318 >> 172.21.15.28:0/1069982318 conn(0x7fe6dc0d0a50 msgr2=0x7fe6dc0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:47.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.363+0000 7fe6e2a00640 1 -- 172.21.15.28:0/1069982318 shutdown_connections 2024-04-26T20:57:47.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.363+0000 7fe6e2a00640 1 -- 172.21.15.28:0/1069982318 wait complete. 2024-04-26T20:57:47.382 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi028.front.sepia.ceph.com... 2024-04-26T20:57:47.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:47.383 DEBUG:teuthology.orchestra.run.smithi028:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-26T20:57:47.466 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:57:47.466 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T20:57:47.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:47.466 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-26T20:57:47.523 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:57:47.523 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:57:47.582 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:57:47.582 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:57:47.637 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:57:47.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.855+0000 7effc7e00640 1 Processor -- start 2024-04-26T20:57:47.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.855+0000 7effc7e00640 1 -- start start 2024-04-26T20:57:47.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.857+0000 7effc7e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effc8146060 0x7effc8071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:47.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.857+0000 7effc7e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7effc8146670 0x7effc81609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:47.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.857+0000 7effc7e00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effc8146440 con 0x7effc8160ef0 2024-04-26T20:57:47.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.857+0000 7effc7e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7effc806f510 con 0x7effc8146670 2024-04-26T20:57:47.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.857+0000 7effc7e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7effc806f010 con 0x7effc8146060 2024-04-26T20:57:47.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.858+0000 7effc6000640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7effc8146670 0x7effc81609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:47.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.858+0000 7effc6000640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7effc8146670 0x7effc81609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.28:33664/0 (socket says 172.21.15.28:33664) 2024-04-26T20:57:47.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.858+0000 7effc6000640 1 -- 172.21.15.28:0/3000149420 learned_addr learned my addr 172.21.15.28:0/3000149420 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:47.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.858+0000 7effc6a00640 1 --2- 172.21.15.28:0/3000149420 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effc8146060 0x7effc8071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:47.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.858+0000 7effc6000640 1 -- 172.21.15.28:0/3000149420 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effc8146060 msgr2=0x7effc8071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:47.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.858+0000 7effc6000640 1 --2- 172.21.15.28:0/3000149420 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effc8146060 0x7effc8071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:47.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.858+0000 7effc6000640 1 -- 172.21.15.28:0/3000149420 >> v1:172.21.15.103:6789/0 conn(0x7effc8160ef0 legacy=0x7effc806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:47.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.858+0000 7effc6000640 1 -- 172.21.15.28:0/3000149420 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc81617a0 con 0x7effc8146670 2024-04-26T20:57:47.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.860+0000 7effc6000640 1 --2- 172.21.15.28:0/3000149420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effc8146670 0x7effc81609b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7effb4009870 tx=0x7effb402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57efef42b2cbd32b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:47.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.861+0000 7effc5600640 1 -- 172.21.15.28:0/3000149420 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb4037070 con 0x7effc8146670 2024-04-26T20:57:47.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.861+0000 7effc5600640 1 -- 172.21.15.28:0/3000149420 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effb4032440 con 0x7effc8146670 2024-04-26T20:57:47.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.861+0000 7effc5600640 1 -- 172.21.15.28:0/3000149420 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb4032810 con 0x7effc8146670 2024-04-26T20:57:47.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.861+0000 7effc7e00640 1 -- 172.21.15.28:0/3000149420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effc8146670 msgr2=0x7effc81609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:47.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.861+0000 7effc7e00640 1 --2- 172.21.15.28:0/3000149420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effc8146670 0x7effc81609b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7effb4009870 tx=0x7effb402a1c0 comp rx=0 tx=0).stop 2024-04-26T20:57:47.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.861+0000 7effc7e00640 1 -- 172.21.15.28:0/3000149420 shutdown_connections 2024-04-26T20:57:47.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.861+0000 7effc7e00640 1 --2- 172.21.15.28:0/3000149420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effc8146670 0x7effc81609b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:47.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.861+0000 7effc7e00640 1 --2- 172.21.15.28:0/3000149420 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effc8146060 0x7effc8071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:47.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.861+0000 7effc7e00640 1 -- 172.21.15.28:0/3000149420 >> 172.21.15.28:0/3000149420 conn(0x7effc80d0a50 msgr2=0x7effc80d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:47.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.861+0000 7effc7e00640 1 -- 172.21.15.28:0/3000149420 shutdown_connections 2024-04-26T20:57:47.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.861+0000 7effc7e00640 1 -- 172.21.15.28:0/3000149420 wait complete. 2024-04-26T20:57:47.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.863+0000 7effc7e00640 1 Processor -- start 2024-04-26T20:57:47.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.863+0000 7effc7e00640 1 -- start start 2024-04-26T20:57:47.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.864+0000 7effc7e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effc8146060 0x7effc815d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:47.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.864+0000 7effc7e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effc8146670 0x7effc8134e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:47.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.864+0000 7effc7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effc8160ef0 0x7effc81e6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:47.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.864+0000 7effc7e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7effc80cc0b0 con 0x7effc8160ef0 2024-04-26T20:57:47.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.864+0000 7effc7e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7effc806ff60 con 0x7effc8146060 2024-04-26T20:57:47.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.864+0000 7effc7e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7effc80cc230 con 0x7effc8146670 2024-04-26T20:57:47.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.865+0000 7effc7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effc8160ef0 0x7effc81e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:47.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.865+0000 7effc7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effc8160ef0 0x7effc81e6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51222/0 (socket says 172.21.15.28:51222) 2024-04-26T20:57:47.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.865+0000 7effc7400640 1 -- 172.21.15.28:0/1762404439 learned_addr learned my addr 172.21.15.28:0/1762404439 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:47.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.865+0000 7effc6a00640 1 --2- 172.21.15.28:0/1762404439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effc8146060 0x7effc815d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:47.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.865+0000 7effc6000640 1 --2- 172.21.15.28:0/1762404439 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effc8146670 0x7effc8134e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:47.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.865+0000 7effc7400640 1 -- 172.21.15.28:0/1762404439 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effc8146670 msgr2=0x7effc8134e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:47.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.865+0000 7effc7400640 1 --2- 172.21.15.28:0/1762404439 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effc8146670 0x7effc8134e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:47.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.865+0000 7effc7400640 1 -- 172.21.15.28:0/1762404439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effc8146060 msgr2=0x7effc815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:47.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.865+0000 7effc7400640 1 --2- 172.21.15.28:0/1762404439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effc8146060 0x7effc815d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:47.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.865+0000 7effc7400640 1 -- 172.21.15.28:0/1762404439 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc81e70d0 con 0x7effc8160ef0 2024-04-26T20:57:47.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.866+0000 7effc7400640 1 --2- 172.21.15.28:0/1762404439 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effc8160ef0 0x7effc81e6680 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7effc000c6e0 tx=0x7effc000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:47.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.866+0000 7effbea00640 1 -- 172.21.15.28:0/1762404439 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0003b70 con 0x7effc8160ef0 2024-04-26T20:57:47.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.866+0000 7effbea00640 1 -- 172.21.15.28:0/1762404439 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effc0003d30 con 0x7effc8160ef0 2024-04-26T20:57:47.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.866+0000 7effbea00640 1 -- 172.21.15.28:0/1762404439 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc0005310 con 0x7effc8160ef0 2024-04-26T20:57:47.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.866+0000 7effc7e00640 1 -- 172.21.15.28:0/1762404439 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc81e73b0 con 0x7effc8160ef0 2024-04-26T20:57:47.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.866+0000 7effc7e00640 1 -- 172.21.15.28:0/1762404439 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc81e7970 con 0x7effc8160ef0 2024-04-26T20:57:47.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.867+0000 7effc7e00640 1 -- 172.21.15.28:0/1762404439 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc806a360 con 0x7effc8160ef0 2024-04-26T20:57:47.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.868+0000 7effbea00640 1 -- 172.21.15.28:0/1762404439 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7effc00054b0 con 0x7effc8160ef0 2024-04-26T20:57:47.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.869+0000 7effbea00640 1 --2- 172.21.15.28:0/1762404439 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7effa806c920 0x7effa806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:47.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.869+0000 7effbea00640 1 -- 172.21.15.28:0/1762404439 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7effc008b8e0 con 0x7effc8160ef0 2024-04-26T20:57:47.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.870+0000 7effc6a00640 1 --2- 172.21.15.28:0/1762404439 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7effa806c920 0x7effa806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:47.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.870+0000 7effc6a00640 1 --2- 172.21.15.28:0/1762404439 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7effa806c920 0x7effa806ede0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7effc81449c0 tx=0x7effb8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:47.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:47.874+0000 7effbea00640 1 -- 172.21.15.28:0/1762404439 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7effc0059d20 con 0x7effc8160ef0 2024-04-26T20:57:48.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.070+0000 7effc7e00640 1 -- 172.21.15.28:0/1762404439 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7effc806e110 con 0x7effc8160ef0 2024-04-26T20:57:48.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.072+0000 7effbea00640 1 -- 172.21.15.28:0/1762404439 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16831 (secure 0 0 0) 0x7effc008b070 con 0x7effc8160ef0 2024-04-26T20:57:48.073 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:48.073 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":23,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:57:37.996297+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:57:48.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.075+0000 7effc7e00640 1 -- 172.21.15.28:0/1762404439 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7effa806c920 msgr2=0x7effa806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:48.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.075+0000 7effc7e00640 1 --2- 172.21.15.28:0/1762404439 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7effa806c920 0x7effa806ede0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7effc81449c0 tx=0x7effb8009190 comp rx=0 tx=0).stop 2024-04-26T20:57:48.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.075+0000 7effc7e00640 1 -- 172.21.15.28:0/1762404439 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effc8160ef0 msgr2=0x7effc81e6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:48.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.075+0000 7effc7e00640 1 --2- 172.21.15.28:0/1762404439 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effc8160ef0 0x7effc81e6680 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7effc000c6e0 tx=0x7effc000cba0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.075+0000 7effc7e00640 1 -- 172.21.15.28:0/1762404439 shutdown_connections 2024-04-26T20:57:48.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.075+0000 7effc7e00640 1 --2- 172.21.15.28:0/1762404439 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7effa806c920 0x7effa806ede0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.075+0000 7effc7e00640 1 --2- 172.21.15.28:0/1762404439 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7effc8160ef0 0x7effc81e6680 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.075+0000 7effc7e00640 1 --2- 172.21.15.28:0/1762404439 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7effc8146670 0x7effc8134e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.075+0000 7effc7e00640 1 --2- 172.21.15.28:0/1762404439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7effc8146060 0x7effc815d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.075+0000 7effc7e00640 1 -- 172.21.15.28:0/1762404439 >> 172.21.15.28:0/1762404439 conn(0x7effc80d0a50 msgr2=0x7effc80cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:48.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.075+0000 7effc7e00640 1 -- 172.21.15.28:0/1762404439 shutdown_connections 2024-04-26T20:57:48.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.075+0000 7effc7e00640 1 -- 172.21.15.28:0/1762404439 wait complete. 2024-04-26T20:57:48.094 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:56:48.688498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:56:50.117541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:57:48.095 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:57:48.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.300+0000 7f7eeb200640 1 Processor -- start 2024-04-26T20:57:48.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.300+0000 7f7eeb200640 1 -- start start 2024-04-26T20:57:48.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.302+0000 7f7eeb200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ee415efd0 0x7f7ee40d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:48.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.302+0000 7f7eeb200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f7ee40b4f40 0x7f7ee40d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:48.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.302+0000 7f7eeb200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7ee40b5c50 0x7f7ee40ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:48.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.302+0000 7f7eeb200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f7ee40b3ea0 con 0x7f7ee40b5c50 2024-04-26T20:57:48.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.302+0000 7f7eeb200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f7ee40b4180 con 0x7f7ee40b4f40 2024-04-26T20:57:48.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.302+0000 7f7eeb200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee40b4460 con 0x7f7ee415efd0 2024-04-26T20:57:48.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.302+0000 7f7eea800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7ee40b5c50 0x7f7ee40ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:48.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.302+0000 7f7eea800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7ee40b5c50 0x7f7ee40ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51236/0 (socket says 172.21.15.28:51236) 2024-04-26T20:57:48.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.302+0000 7f7eea800640 1 -- 172.21.15.28:0/3179321496 learned_addr learned my addr 172.21.15.28:0/3179321496 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:48.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.303+0000 7f7ee9400640 1 --2- 172.21.15.28:0/3179321496 >> v2:172.21.15.103:3300/0 conn(0x7f7ee40b4f40 0x7f7ee40d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:48.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.303+0000 7f7ee9e00640 1 --2- 172.21.15.28:0/3179321496 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ee415efd0 0x7f7ee40d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:48.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.303+0000 7f7eea800640 1 -- 172.21.15.28:0/3179321496 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ee415efd0 msgr2=0x7f7ee40d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:48.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.303+0000 7f7eea800640 1 --2- 172.21.15.28:0/3179321496 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ee415efd0 0x7f7ee40d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.303+0000 7f7eea800640 1 -- 172.21.15.28:0/3179321496 >> v2:172.21.15.103:3300/0 conn(0x7f7ee40b4f40 msgr2=0x7f7ee40d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:48.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.303+0000 7f7eea800640 1 --2- 172.21.15.28:0/3179321496 >> v2:172.21.15.103:3300/0 conn(0x7f7ee40b4f40 0x7f7ee40d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.303+0000 7f7eea800640 1 -- 172.21.15.28:0/3179321496 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee40b4740 con 0x7f7ee40b5c50 2024-04-26T20:57:48.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.304+0000 7f7eea800640 1 --2- 172.21.15.28:0/3179321496 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ee40b5c50 0x7f7ee40ae8c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7ee40d1950 tx=0x7f7ee002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26e3b297c261ab6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:48.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.305+0000 7f7ee8a00640 1 -- 172.21.15.28:0/3179321496 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee0036070 con 0x7f7ee40b5c50 2024-04-26T20:57:48.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.305+0000 7f7ee8a00640 1 -- 172.21.15.28:0/3179321496 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee0031600 con 0x7f7ee40b5c50 2024-04-26T20:57:48.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.305+0000 7f7ee8a00640 1 -- 172.21.15.28:0/3179321496 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee00319d0 con 0x7f7ee40b5c50 2024-04-26T20:57:48.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.306+0000 7f7eeb200640 1 -- 172.21.15.28:0/3179321496 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ee40b5c50 msgr2=0x7f7ee40ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:48.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.306+0000 7f7eeb200640 1 --2- 172.21.15.28:0/3179321496 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ee40b5c50 0x7f7ee40ae8c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7ee40d1950 tx=0x7f7ee002a6a0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.306+0000 7f7eeb200640 1 -- 172.21.15.28:0/3179321496 shutdown_connections 2024-04-26T20:57:48.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.306+0000 7f7eeb200640 1 --2- 172.21.15.28:0/3179321496 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ee40b5c50 0x7f7ee40ae8c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.306+0000 7f7eeb200640 1 --2- 172.21.15.28:0/3179321496 >> v2:172.21.15.103:3300/0 conn(0x7f7ee40b4f40 0x7f7ee40d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.306+0000 7f7eeb200640 1 --2- 172.21.15.28:0/3179321496 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ee415efd0 0x7f7ee40d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.306+0000 7f7eeb200640 1 -- 172.21.15.28:0/3179321496 >> 172.21.15.28:0/3179321496 conn(0x7f7ee406c530 msgr2=0x7f7ee406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:48.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.306+0000 7f7eeb200640 1 -- 172.21.15.28:0/3179321496 shutdown_connections 2024-04-26T20:57:48.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.306+0000 7f7eeb200640 1 -- 172.21.15.28:0/3179321496 wait complete. 2024-04-26T20:57:48.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.308+0000 7f7eeb200640 1 Processor -- start 2024-04-26T20:57:48.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.308+0000 7f7eeb200640 1 -- start start 2024-04-26T20:57:48.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.310+0000 7f7eeb200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ee40b4f40 0x7f7ee41f5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:48.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.310+0000 7f7eeb200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ee40b5c50 0x7f7ee4153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:48.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.310+0000 7f7eeb200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ee415efd0 0x7f7ee4154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:48.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.310+0000 7f7eeb200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee40b56c0 con 0x7f7ee40b4f40 2024-04-26T20:57:48.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.310+0000 7f7eeb200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee415f3b0 con 0x7f7ee415efd0 2024-04-26T20:57:48.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.310+0000 7f7eeb200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee40b5840 con 0x7f7ee40b5c50 2024-04-26T20:57:48.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.310+0000 7f7ee9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ee40b4f40 0x7f7ee41f5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:48.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.310+0000 7f7ee9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ee40b4f40 0x7f7ee41f5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51242/0 (socket says 172.21.15.28:51242) 2024-04-26T20:57:48.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.310+0000 7f7ee9e00640 1 -- 172.21.15.28:0/2699365233 learned_addr learned my addr 172.21.15.28:0/2699365233 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:48.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.310+0000 7f7eea800640 1 --2- 172.21.15.28:0/2699365233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ee415efd0 0x7f7ee4154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:48.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.311+0000 7f7ee9e00640 1 -- 172.21.15.28:0/2699365233 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ee40b5c50 msgr2=0x7f7ee4153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:48.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.311+0000 7f7ee9e00640 1 --2- 172.21.15.28:0/2699365233 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ee40b5c50 0x7f7ee4153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.311+0000 7f7ee9e00640 1 -- 172.21.15.28:0/2699365233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ee415efd0 msgr2=0x7f7ee4154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:48.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.311+0000 7f7ee9e00640 1 --2- 172.21.15.28:0/2699365233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ee415efd0 0x7f7ee4154870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.311+0000 7f7ee9e00640 1 -- 172.21.15.28:0/2699365233 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee4160f70 con 0x7f7ee40b4f40 2024-04-26T20:57:48.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.311+0000 7f7eea800640 1 --2- 172.21.15.28:0/2699365233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ee415efd0 0x7f7ee4154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:48.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.311+0000 7f7ee9e00640 1 --2- 172.21.15.28:0/2699365233 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ee40b4f40 0x7f7ee41f5090 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800d800 tx=0x7f7ed800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:48.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.311+0000 7f7ede000640 1 -- 172.21.15.28:0/2699365233 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed800b6c0 con 0x7f7ee40b4f40 2024-04-26T20:57:48.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.311+0000 7f7ede000640 1 -- 172.21.15.28:0/2699365233 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed8019070 con 0x7f7ee40b4f40 2024-04-26T20:57:48.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.311+0000 7f7ede000640 1 -- 172.21.15.28:0/2699365233 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed800ba80 con 0x7f7ee40b4f40 2024-04-26T20:57:48.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.311+0000 7f7eeb200640 1 -- 172.21.15.28:0/2699365233 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ee41f5680 con 0x7f7ee40b4f40 2024-04-26T20:57:48.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.311+0000 7f7eeb200640 1 -- 172.21.15.28:0/2699365233 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ee41f5c40 con 0x7f7ee40b4f40 2024-04-26T20:57:48.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.313+0000 7f7eeb200640 1 -- 172.21.15.28:0/2699365233 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ee40afbc0 con 0x7f7ee40b4f40 2024-04-26T20:57:48.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.314+0000 7f7ede000640 1 -- 172.21.15.28:0/2699365233 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ed800bc20 con 0x7f7ee40b4f40 2024-04-26T20:57:48.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.314+0000 7f7ede000640 1 --2- 172.21.15.28:0/2699365233 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7ec806c8f0 0x7f7ec806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:48.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.315+0000 7f7ede000640 1 -- 172.21.15.28:0/2699365233 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f7ed808bf90 con 0x7f7ee40b4f40 2024-04-26T20:57:48.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.315+0000 7f7ee9400640 1 --2- 172.21.15.28:0/2699365233 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7ec806c8f0 0x7f7ec806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:48.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.316+0000 7f7ee9400640 1 --2- 172.21.15.28:0/2699365233 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7ec806c8f0 0x7f7ec806edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4006a90 tx=0x7f7ed4006a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:48.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.319+0000 7f7ede000640 1 -- 172.21.15.28:0/2699365233 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7ee40afbc0 con 0x7f7ee40b4f40 2024-04-26T20:57:48.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.528+0000 7f7eeb200640 1 -- 172.21.15.28:0/2699365233 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7ee406a250 con 0x7f7ee40b4f40 2024-04-26T20:57:48.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.529+0000 7f7ede000640 1 -- 172.21.15.28:0/2699365233 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f7ed805a3d0 con 0x7f7ee40b4f40 2024-04-26T20:57:48.529 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:57:48.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.532+0000 7f7eeb200640 1 -- 172.21.15.28:0/2699365233 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7ec806c8f0 msgr2=0x7f7ec806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:48.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.532+0000 7f7eeb200640 1 --2- 172.21.15.28:0/2699365233 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7ec806c8f0 0x7f7ec806edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4006a90 tx=0x7f7ed4006a20 comp rx=0 tx=0).stop 2024-04-26T20:57:48.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.532+0000 7f7eeb200640 1 -- 172.21.15.28:0/2699365233 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ee40b4f40 msgr2=0x7f7ee41f5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:48.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.532+0000 7f7eeb200640 1 --2- 172.21.15.28:0/2699365233 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ee40b4f40 0x7f7ee41f5090 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800d800 tx=0x7f7ed800dcc0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.532+0000 7f7eeb200640 1 -- 172.21.15.28:0/2699365233 shutdown_connections 2024-04-26T20:57:48.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.532+0000 7f7eeb200640 1 --2- 172.21.15.28:0/2699365233 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7ec806c8f0 0x7f7ec806edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.532+0000 7f7eeb200640 1 --2- 172.21.15.28:0/2699365233 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ee415efd0 0x7f7ee4154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.532+0000 7f7eeb200640 1 --2- 172.21.15.28:0/2699365233 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ee40b5c50 0x7f7ee4153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.532+0000 7f7eeb200640 1 --2- 172.21.15.28:0/2699365233 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ee40b4f40 0x7f7ee41f5090 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.532+0000 7f7eeb200640 1 -- 172.21.15.28:0/2699365233 >> 172.21.15.28:0/2699365233 conn(0x7f7ee406c530 msgr2=0x7f7ee415cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:48.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.532+0000 7f7eeb200640 1 -- 172.21.15.28:0/2699365233 shutdown_connections 2024-04-26T20:57:48.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.532+0000 7f7eeb200640 1 -- 172.21.15.28:0/2699365233 wait complete. 2024-04-26T20:57:48.551 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:57:48.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.769+0000 7f54d0a00640 1 Processor -- start 2024-04-26T20:57:48.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.770+0000 7f54d0a00640 1 -- start start 2024-04-26T20:57:48.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.771+0000 7f54d0a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f54cc0b4a90 0x7f54cc072fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:48.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.771+0000 7f54d0a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54cc0b5090 con 0x7f54cc073500 2024-04-26T20:57:48.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.771+0000 7f54d0a00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54cc0b5370 con 0x7f54cc161040 2024-04-26T20:57:48.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.771+0000 7f54d0a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f54cc073990 con 0x7f54cc0b4a90 2024-04-26T20:57:48.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.772+0000 7f54ca800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f54cc073500 0x7f54cc06e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58464/0 (socket says 172.21.15.28:58464) 2024-04-26T20:57:48.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.772+0000 7f54ca800640 1 -- 172.21.15.28:0/3382773054 learned_addr learned my addr 172.21.15.28:0/3382773054 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:48.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.772+0000 7f54c3e00640 1 --2- 172.21.15.28:0/3382773054 >> v2:172.21.15.28:3300/0 conn(0x7f54cc0b4a90 0x7f54cc072fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:48.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.772+0000 7f54c9400640 1 -- 172.21.15.28:0/3382773054 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 579296397 0 0) 0x7f54cc0b5090 con 0x7f54cc073500 2024-04-26T20:57:48.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.772+0000 7f54c9400640 1 -- 172.21.15.28:0/3382773054 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54b0003520 con 0x7f54cc073500 2024-04-26T20:57:48.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.772+0000 7f54c3e00640 1 -- 172.21.15.28:0/3382773054 >> v1:172.21.15.103:6789/0 conn(0x7f54cc161040 legacy=0x7f54cc1644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:48.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.772+0000 7f54c3e00640 1 -- 172.21.15.28:0/3382773054 >> v1:172.21.15.28:6789/0 conn(0x7f54cc073500 legacy=0x7f54cc06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:48.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.772+0000 7f54c3e00640 1 -- 172.21.15.28:0/3382773054 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54cc0b5650 con 0x7f54cc0b4a90 2024-04-26T20:57:48.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.773+0000 7f54c3e00640 1 --2- 172.21.15.28:0/3382773054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54cc0b4a90 0x7f54cc072fc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f54b40097a0 tx=0x7f54b402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a888731e8b494e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:48.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.774+0000 7f54c9400640 1 -- 172.21.15.28:0/3382773054 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4037070 con 0x7f54cc0b4a90 2024-04-26T20:57:48.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.774+0000 7f54c9400640 1 -- 172.21.15.28:0/3382773054 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54b4032440 con 0x7f54cc0b4a90 2024-04-26T20:57:48.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.774+0000 7f54c9400640 1 -- 172.21.15.28:0/3382773054 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4032810 con 0x7f54cc0b4a90 2024-04-26T20:57:48.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.774+0000 7f54d0a00640 1 -- 172.21.15.28:0/3382773054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54cc0b4a90 msgr2=0x7f54cc072fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:48.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.774+0000 7f54d0a00640 1 --2- 172.21.15.28:0/3382773054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54cc0b4a90 0x7f54cc072fc0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f54b40097a0 tx=0x7f54b402a230 comp rx=0 tx=0).stop 2024-04-26T20:57:48.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.774+0000 7f54d0a00640 1 -- 172.21.15.28:0/3382773054 shutdown_connections 2024-04-26T20:57:48.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.774+0000 7f54d0a00640 1 --2- 172.21.15.28:0/3382773054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54cc0b4a90 0x7f54cc072fc0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.774+0000 7f54d0a00640 1 -- 172.21.15.28:0/3382773054 >> 172.21.15.28:0/3382773054 conn(0x7f54cc06c770 msgr2=0x7f54cc06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:48.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.775+0000 7f54d0a00640 1 -- 172.21.15.28:0/3382773054 shutdown_connections 2024-04-26T20:57:48.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.775+0000 7f54d0a00640 1 -- 172.21.15.28:0/3382773054 wait complete. 2024-04-26T20:57:48.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.776+0000 7f54d0a00640 1 Processor -- start 2024-04-26T20:57:48.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.776+0000 7f54d0a00640 1 -- start start 2024-04-26T20:57:48.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.777+0000 7f54d0a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54cc073500 0x7f54cc1f9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:48.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.777+0000 7f54d0a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f54cc0b4a90 0x7f54cc14df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:48.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.777+0000 7f54d0a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54cc161040 0x7f54cc163670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:48.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.777+0000 7f54d0a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f54cc161420 con 0x7f54cc161040 2024-04-26T20:57:48.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.777+0000 7f54d0a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f54cc0b5c10 con 0x7f54cc073500 2024-04-26T20:57:48.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.777+0000 7f54d0a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f54cc0b6130 con 0x7f54cc0b4a90 2024-04-26T20:57:48.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.778+0000 7f54ca800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54cc161040 0x7f54cc163670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:48.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.778+0000 7f54ca800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54cc161040 0x7f54cc163670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51258/0 (socket says 172.21.15.28:51258) 2024-04-26T20:57:48.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.778+0000 7f54ca800640 1 -- 172.21.15.28:0/1818305763 learned_addr learned my addr 172.21.15.28:0/1818305763 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:48.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.778+0000 7f54c3e00640 1 --2- 172.21.15.28:0/1818305763 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f54cc0b4a90 0x7f54cc14df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:48.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.778+0000 7f54c9e00640 1 --2- 172.21.15.28:0/1818305763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54cc073500 0x7f54cc1f9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.778+0000 7f54ca800640 1 -- 172.21.15.28:0/1818305763 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f54cc0b4a90 msgr2=0x7f54cc14df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.778+0000 7f54ca800640 1 --2- 172.21.15.28:0/1818305763 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f54cc0b4a90 0x7f54cc14df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.778+0000 7f54ca800640 1 -- 172.21.15.28:0/1818305763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54cc073500 msgr2=0x7f54cc1f9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.778+0000 7f54ca800640 1 --2- 172.21.15.28:0/1818305763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54cc073500 0x7f54cc1f9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.778+0000 7f54ca800640 1 -- 172.21.15.28:0/1818305763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54cc163bb0 con 0x7f54cc161040 2024-04-26T20:57:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.779+0000 7f54ca800640 1 --2- 172.21.15.28:0/1818305763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54cc161040 0x7f54cc163670 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f54c400cb40 tx=0x7f54c400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.779+0000 7f54c2000640 1 -- 172.21.15.28:0/1818305763 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4017070 con 0x7f54cc161040 2024-04-26T20:57:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.779+0000 7f54c2000640 1 -- 172.21.15.28:0/1818305763 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54c400aca0 con 0x7f54cc161040 2024-04-26T20:57:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.779+0000 7f54c2000640 1 -- 172.21.15.28:0/1818305763 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4012620 con 0x7f54cc161040 2024-04-26T20:57:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.779+0000 7f54d0a00640 1 -- 172.21.15.28:0/1818305763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54cc1617f0 con 0x7f54cc161040 2024-04-26T20:57:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.779+0000 7f54d0a00640 1 -- 172.21.15.28:0/1818305763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54cc161db0 con 0x7f54cc161040 2024-04-26T20:57:48.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.780+0000 7f54d0a00640 1 -- 172.21.15.28:0/1818305763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54cc06a360 con 0x7f54cc161040 2024-04-26T20:57:48.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.781+0000 7f54c2000640 1 -- 172.21.15.28:0/1818305763 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f54c401a070 con 0x7f54cc161040 2024-04-26T20:57:48.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.782+0000 7f54c2000640 1 --2- 172.21.15.28:0/1818305763 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f54a806c8f0 0x7f54a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:48.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.782+0000 7f54c2000640 1 -- 172.21.15.28:0/1818305763 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f54c408ba90 con 0x7f54cc161040 2024-04-26T20:57:48.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.782+0000 7f54c9e00640 1 --2- 172.21.15.28:0/1818305763 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f54a806c8f0 0x7f54a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:48.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.783+0000 7f54c9e00640 1 --2- 172.21.15.28:0/1818305763 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f54a806c8f0 0x7f54a806edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f54bc000f60 tx=0x7f54bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:48.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.786+0000 7f54c2000640 1 -- 172.21.15.28:0/1818305763 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f54c4059e30 con 0x7f54cc161040 2024-04-26T20:57:48.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.991+0000 7f54d0a00640 1 -- 172.21.15.28:0/1818305763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f54cc166330 con 0x7f54cc161040 2024-04-26T20:57:48.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.992+0000 7f54c2000640 1 -- 172.21.15.28:0/1818305763 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f54c4059730 con 0x7f54cc161040 2024-04-26T20:57:48.992 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-04-26T20:57:48.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.996+0000 7f54d0a00640 1 -- 172.21.15.28:0/1818305763 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f54a806c8f0 msgr2=0x7f54a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:48.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.996+0000 7f54d0a00640 1 --2- 172.21.15.28:0/1818305763 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f54a806c8f0 0x7f54a806edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f54bc000f60 tx=0x7f54bc009040 comp rx=0 tx=0).stop 2024-04-26T20:57:48.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.996+0000 7f54d0a00640 1 -- 172.21.15.28:0/1818305763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54cc161040 msgr2=0x7f54cc163670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:48.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.996+0000 7f54d0a00640 1 --2- 172.21.15.28:0/1818305763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54cc161040 0x7f54cc163670 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f54c400cb40 tx=0x7f54c400a060 comp rx=0 tx=0).stop 2024-04-26T20:57:48.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.996+0000 7f54d0a00640 1 -- 172.21.15.28:0/1818305763 shutdown_connections 2024-04-26T20:57:48.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.996+0000 7f54d0a00640 1 --2- 172.21.15.28:0/1818305763 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f54a806c8f0 0x7f54a806edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.996+0000 7f54d0a00640 1 --2- 172.21.15.28:0/1818305763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54cc161040 0x7f54cc163670 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.996+0000 7f54d0a00640 1 --2- 172.21.15.28:0/1818305763 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f54cc0b4a90 0x7f54cc14df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.996+0000 7f54d0a00640 1 --2- 172.21.15.28:0/1818305763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54cc073500 0x7f54cc1f9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:48.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.996+0000 7f54d0a00640 1 -- 172.21.15.28:0/1818305763 >> 172.21.15.28:0/1818305763 conn(0x7f54cc06c770 msgr2=0x7f54cc072550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:48.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.996+0000 7f54d0a00640 1 -- 172.21.15.28:0/1818305763 shutdown_connections 2024-04-26T20:57:48.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:48.996+0000 7f54d0a00640 1 -- 172.21.15.28:0/1818305763 wait complete. 2024-04-26T20:57:49.015 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:57:49.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.216+0000 7f91e0200640 1 Processor -- start 2024-04-26T20:57:49.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.217+0000 7f91e0200640 1 -- start start 2024-04-26T20:57:49.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.218+0000 7f91e0200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f91d815cf00 0x7f91d80d2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:49.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.218+0000 7f91e0200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f91d815d2e0 0x7f91d80d32e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:49.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.218+0000 7f91e0200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91d8071ae0 con 0x7f91d8070d40 2024-04-26T20:57:49.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.218+0000 7f91e0200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f91d80718b0 con 0x7f91d815d2e0 2024-04-26T20:57:49.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.218+0000 7f91e0200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f91d80afd30 con 0x7f91d815cf00 2024-04-26T20:57:49.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.219+0000 7f91dce00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f91d815d2e0 0x7f91d80d32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:49.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.219+0000 7f91dce00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f91d815d2e0 0x7f91d80d32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51264/0 (socket says 172.21.15.28:51264) 2024-04-26T20:57:49.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.219+0000 7f91dce00640 1 -- 172.21.15.28:0/1217891991 learned_addr learned my addr 172.21.15.28:0/1217891991 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:49.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.219+0000 7f91dd800640 1 --2- 172.21.15.28:0/1217891991 >> v2:172.21.15.103:3300/0 conn(0x7f91d815cf00 0x7f91d80d2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:49.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.219+0000 7f91dce00640 1 -- 172.21.15.28:0/1217891991 >> v2:172.21.15.103:3300/0 conn(0x7f91d815cf00 msgr2=0x7f91d80d2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:49.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.219+0000 7f91dce00640 1 --2- 172.21.15.28:0/1217891991 >> v2:172.21.15.103:3300/0 conn(0x7f91d815cf00 0x7f91d80d2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:49.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.219+0000 7f91dce00640 1 -- 172.21.15.28:0/1217891991 >> v1:172.21.15.28:6789/0 conn(0x7f91d8070d40 legacy=0x7f91d80711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:49.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.219+0000 7f91dce00640 1 -- 172.21.15.28:0/1217891991 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d8071dc0 con 0x7f91d815d2e0 2024-04-26T20:57:49.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.219+0000 7f91d7e00640 1 -- 172.21.15.28:0/1217891991 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1719423596 0 0) 0x7f91d8071ae0 con 0x7f91d8070d40 2024-04-26T20:57:49.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.220+0000 7f91dce00640 1 --2- 172.21.15.28:0/1217891991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d815d2e0 0x7f91d80d32e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f91d80d0960 tx=0x7f91c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a476b47a48d449c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:49.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.221+0000 7f91d7e00640 1 -- 172.21.15.28:0/1217891991 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4037070 con 0x7f91d815d2e0 2024-04-26T20:57:49.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.221+0000 7f91d7e00640 1 -- 172.21.15.28:0/1217891991 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c4032440 con 0x7f91d815d2e0 2024-04-26T20:57:49.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.221+0000 7f91d7e00640 1 -- 172.21.15.28:0/1217891991 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4032810 con 0x7f91d815d2e0 2024-04-26T20:57:49.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.221+0000 7f91e0200640 1 -- 172.21.15.28:0/1217891991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d815d2e0 msgr2=0x7f91d80d32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:49.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.221+0000 7f91e0200640 1 --2- 172.21.15.28:0/1217891991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d815d2e0 0x7f91d80d32e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f91d80d0960 tx=0x7f91c402a270 comp rx=0 tx=0).stop 2024-04-26T20:57:49.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.221+0000 7f91e0200640 1 -- 172.21.15.28:0/1217891991 shutdown_connections 2024-04-26T20:57:49.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.221+0000 7f91e0200640 1 --2- 172.21.15.28:0/1217891991 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d815d2e0 0x7f91d80d32e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:49.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.221+0000 7f91e0200640 1 --2- 172.21.15.28:0/1217891991 >> v2:172.21.15.103:3300/0 conn(0x7f91d815cf00 0x7f91d80d2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:49.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.221+0000 7f91e0200640 1 -- 172.21.15.28:0/1217891991 >> 172.21.15.28:0/1217891991 conn(0x7f91d809ba50 msgr2=0x7f91d809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:49.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.222+0000 7f91e0200640 1 -- 172.21.15.28:0/1217891991 shutdown_connections 2024-04-26T20:57:49.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.222+0000 7f91e0200640 1 -- 172.21.15.28:0/1217891991 wait complete. 2024-04-26T20:57:49.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.223+0000 7f91e0200640 1 Processor -- start 2024-04-26T20:57:49.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.223+0000 7f91e0200640 1 -- start start 2024-04-26T20:57:49.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.224+0000 7f91e0200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d8070d40 0x7f91d815fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:49.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.224+0000 7f91e0200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91d815cf00 0x7f91d81500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:49.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.224+0000 7f91e0200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f91d815d2e0 0x7f91d8150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:49.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.224+0000 7f91e0200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f91d80cfa20 con 0x7f91d8070d40 2024-04-26T20:57:49.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.224+0000 7f91e0200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f91d80cf8a0 con 0x7f91d815cf00 2024-04-26T20:57:49.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.224+0000 7f91e0200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f91d80cfba0 con 0x7f91d815d2e0 2024-04-26T20:57:49.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.225+0000 7f91dd800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d8070d40 0x7f91d815fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:49.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.225+0000 7f91dd800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d8070d40 0x7f91d815fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51272/0 (socket says 172.21.15.28:51272) 2024-04-26T20:57:49.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.225+0000 7f91dd800640 1 -- 172.21.15.28:0/3105136770 learned_addr learned my addr 172.21.15.28:0/3105136770 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:49.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.225+0000 7f91dd800640 1 -- 172.21.15.28:0/3105136770 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f91d815d2e0 msgr2=0x7f91d8150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:49.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.225+0000 7f91dd800640 1 --2- 172.21.15.28:0/3105136770 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f91d815d2e0 0x7f91d8150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:49.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.225+0000 7f91dd800640 1 -- 172.21.15.28:0/3105136770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91d815cf00 msgr2=0x7f91d81500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:49.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.225+0000 7f91dd800640 1 --2- 172.21.15.28:0/3105136770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91d815cf00 0x7f91d81500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:49.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.225+0000 7f91dd800640 1 -- 172.21.15.28:0/3105136770 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d806d340 con 0x7f91d8070d40 2024-04-26T20:57:49.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.225+0000 7f91dd800640 1 --2- 172.21.15.28:0/3105136770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d8070d40 0x7f91d815fe90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f91c800d800 tx=0x7f91c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:49.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.226+0000 7f91d5600640 1 -- 172.21.15.28:0/3105136770 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c800b6c0 con 0x7f91d8070d40 2024-04-26T20:57:49.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.226+0000 7f91d5600640 1 -- 172.21.15.28:0/3105136770 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91c8019070 con 0x7f91d8070d40 2024-04-26T20:57:49.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.226+0000 7f91d5600640 1 -- 172.21.15.28:0/3105136770 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c800ba60 con 0x7f91d8070d40 2024-04-26T20:57:49.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.226+0000 7f91e0200640 1 -- 172.21.15.28:0/3105136770 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d806d620 con 0x7f91d8070d40 2024-04-26T20:57:49.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.226+0000 7f91e0200640 1 -- 172.21.15.28:0/3105136770 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d806dbe0 con 0x7f91d8070d40 2024-04-26T20:57:49.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.227+0000 7f91e0200640 1 -- 172.21.15.28:0/3105136770 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91d8076520 con 0x7f91d8070d40 2024-04-26T20:57:49.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.228+0000 7f91d5600640 1 -- 172.21.15.28:0/3105136770 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f91c800bc00 con 0x7f91d8070d40 2024-04-26T20:57:49.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.229+0000 7f91d5600640 1 --2- 172.21.15.28:0/3105136770 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f91b806c8f0 0x7f91b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:49.229 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.229+0000 7f91d5600640 1 -- 172.21.15.28:0/3105136770 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f91c808c140 con 0x7f91d8070d40 2024-04-26T20:57:49.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.230+0000 7f91dce00640 1 --2- 172.21.15.28:0/3105136770 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f91b806c8f0 0x7f91b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:49.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.230+0000 7f91dce00640 1 --2- 172.21.15.28:0/3105136770 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f91b806c8f0 0x7f91b806edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f91c4034000 tx=0x7f91c402c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:49.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.234+0000 7f91d5600640 1 -- 172.21.15.28:0/3105136770 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f91c805a450 con 0x7f91d8070d40 2024-04-26T20:57:49.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.428+0000 7f91e0200640 1 -- 172.21.15.28:0/3105136770 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f91d80702e0 con 0x7f91d8070d40 2024-04-26T20:57:49.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.429+0000 7f91d5600640 1 -- 172.21.15.28:0/3105136770 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f91c8059d50 con 0x7f91d8070d40 2024-04-26T20:57:49.429 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-04-26T20:57:49.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.431+0000 7f91e0200640 1 -- 172.21.15.28:0/3105136770 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f91b806c8f0 msgr2=0x7f91b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:49.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.431+0000 7f91e0200640 1 --2- 172.21.15.28:0/3105136770 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f91b806c8f0 0x7f91b806edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f91c4034000 tx=0x7f91c402c000 comp rx=0 tx=0).stop 2024-04-26T20:57:49.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.431+0000 7f91e0200640 1 -- 172.21.15.28:0/3105136770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d8070d40 msgr2=0x7f91d815fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:49.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.431+0000 7f91e0200640 1 --2- 172.21.15.28:0/3105136770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d8070d40 0x7f91d815fe90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f91c800d800 tx=0x7f91c800dcc0 comp rx=0 tx=0).stop 2024-04-26T20:57:49.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.431+0000 7f91e0200640 1 -- 172.21.15.28:0/3105136770 shutdown_connections 2024-04-26T20:57:49.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.431+0000 7f91e0200640 1 --2- 172.21.15.28:0/3105136770 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f91b806c8f0 0x7f91b806edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:49.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.431+0000 7f91e0200640 1 --2- 172.21.15.28:0/3105136770 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f91d815d2e0 0x7f91d8150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:49.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.431+0000 7f91e0200640 1 --2- 172.21.15.28:0/3105136770 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91d815cf00 0x7f91d81500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:49.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.431+0000 7f91e0200640 1 --2- 172.21.15.28:0/3105136770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d8070d40 0x7f91d815fe90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:49.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.431+0000 7f91e0200640 1 -- 172.21.15.28:0/3105136770 >> 172.21.15.28:0/3105136770 conn(0x7f91d809ba50 msgr2=0x7f91d80ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:49.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.432+0000 7f91e0200640 1 -- 172.21.15.28:0/3105136770 shutdown_connections 2024-04-26T20:57:49.432 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.432+0000 7f91e0200640 1 -- 172.21.15.28:0/3105136770 wait complete. 2024-04-26T20:57:49.449 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:57:49.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.640+0000 7fc357200640 1 Processor -- start 2024-04-26T20:57:49.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.640+0000 7fc357200640 1 -- start start 2024-04-26T20:57:49.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.642+0000 7fc357200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc358146060 0x7fc358071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:49.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.642+0000 7fc357200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc358161530 con 0x7fc3581610c0 2024-04-26T20:57:49.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.642+0000 7fc357200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc358146440 con 0x7fc358146670 2024-04-26T20:57:49.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.642+0000 7fc357200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc35806f510 con 0x7fc358146060 2024-04-26T20:57:49.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.642+0000 7fc356800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc3581610c0 0x7fc3580cb0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:58470/0 (socket says 172.21.15.28:58470) 2024-04-26T20:57:49.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.642+0000 7fc356800640 1 -- 172.21.15.28:0/2729200742 learned_addr learned my addr 172.21.15.28:0/2729200742 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:49.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.642+0000 7fc355e00640 1 --2- 172.21.15.28:0/2729200742 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc358146060 0x7fc358071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:49.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.643+0000 7fc354a00640 1 -- 172.21.15.28:0/2729200742 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3472780897 0 0) 0x7fc358161530 con 0x7fc3581610c0 2024-04-26T20:57:49.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.643+0000 7fc354a00640 1 -- 172.21.15.28:0/2729200742 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc33c003520 con 0x7fc3581610c0 2024-04-26T20:57:49.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.643+0000 7fc354a00640 1 -- 172.21.15.28:0/2729200742 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1093233999 0 0) 0x7fc33c003520 con 0x7fc3581610c0 2024-04-26T20:57:49.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.643+0000 7fc354a00640 1 -- 172.21.15.28:0/2729200742 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc358146060 msgr2=0x7fc358071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:49.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.643+0000 7fc354a00640 1 --2- 172.21.15.28:0/2729200742 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc358146060 0x7fc358071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:49.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.643+0000 7fc354a00640 1 -- 172.21.15.28:0/2729200742 >> v1:172.21.15.103:6789/0 conn(0x7fc358146670 legacy=0x7fc3581609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:49.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.643+0000 7fc354a00640 1 -- 172.21.15.28:0/2729200742 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc35806f010 con 0x7fc3581610c0 2024-04-26T20:57:49.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.643+0000 7fc354a00640 1 -- 172.21.15.28:0/2729200742 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fc3500019f0 con 0x7fc3581610c0 2024-04-26T20:57:49.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.643+0000 7fc354a00640 1 -- 172.21.15.28:0/2729200742 >> v1:172.21.15.28:6789/0 conn(0x7fc3581610c0 legacy=0x7fc3580cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:49.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.643+0000 7fc354a00640 1 --2- 172.21.15.28:0/2729200742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc33c0039e0 0x7fc33c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:49.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.643+0000 7fc354a00640 1 --2- 172.21.15.28:0/2729200742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc33c004800 0x7fc33c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:49.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.643+0000 7fc354a00640 1 --2- 172.21.15.28:0/2729200742 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc358146060 0x7fc33c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:49.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.643+0000 7fc354a00640 1 -- 172.21.15.28:0/2729200742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc35806f510 con 0x7fc33c004800 2024-04-26T20:57:49.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.643+0000 7fc354a00640 1 -- 172.21.15.28:0/2729200742 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc33c00baf0 con 0x7fc33c0039e0 2024-04-26T20:57:49.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.643+0000 7fc354a00640 1 -- 172.21.15.28:0/2729200742 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc33c00bcf0 con 0x7fc358146060 2024-04-26T20:57:49.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.644+0000 7fc355400640 1 --2- 172.21.15.28:0/2729200742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc33c004800 0x7fc33c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:49.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.644+0000 7fc355e00640 1 --2- 172.21.15.28:0/2729200742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc33c0039e0 0x7fc33c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:49.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.644+0000 7fc355400640 1 -- 172.21.15.28:0/2729200742 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc358146060 msgr2=0x7fc33c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:49.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.644+0000 7fc355400640 1 --2- 172.21.15.28:0/2729200742 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc358146060 0x7fc33c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:49.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.644+0000 7fc355400640 1 -- 172.21.15.28:0/2729200742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc33c0039e0 msgr2=0x7fc33c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:49.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.644+0000 7fc355400640 1 --2- 172.21.15.28:0/2729200742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc33c0039e0 0x7fc33c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:49.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.644+0000 7fc355400640 1 -- 172.21.15.28:0/2729200742 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33c003520 con 0x7fc33c004800 2024-04-26T20:57:49.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.645+0000 7fc355400640 1 --2- 172.21.15.28:0/2729200742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc33c004800 0x7fc33c004c80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc344001750 tx=0x7fc33c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f14e544ad85395 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:49.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.645+0000 7fc354a00640 1 -- 172.21.15.28:0/2729200742 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344001370 con 0x7fc33c004800 2024-04-26T20:57:49.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.645+0000 7fc354a00640 1 -- 172.21.15.28:0/2729200742 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3440052d0 con 0x7fc33c004800 2024-04-26T20:57:49.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.645+0000 7fc354a00640 1 -- 172.21.15.28:0/2729200742 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34402abe0 con 0x7fc33c004800 2024-04-26T20:57:49.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.646+0000 7fc357200640 1 -- 172.21.15.28:0/2729200742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc33c004800 msgr2=0x7fc33c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:49.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.646+0000 7fc357200640 1 --2- 172.21.15.28:0/2729200742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc33c004800 0x7fc33c004c80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc344001750 tx=0x7fc33c005830 comp rx=0 tx=0).stop 2024-04-26T20:57:49.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.646+0000 7fc356800640 1 -- 172.21.15.28:0/2729200742 reap_dead start 2024-04-26T20:57:49.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.646+0000 7fc357200640 1 -- 172.21.15.28:0/2729200742 shutdown_connections 2024-04-26T20:57:49.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.646+0000 7fc357200640 1 -- 172.21.15.28:0/2729200742 >> 172.21.15.28:0/2729200742 conn(0x7fc3580d0a50 msgr2=0x7fc3580d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:49.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.646+0000 7fc357200640 1 -- 172.21.15.28:0/2729200742 shutdown_connections 2024-04-26T20:57:49.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.646+0000 7fc357200640 1 -- 172.21.15.28:0/2729200742 wait complete. 2024-04-26T20:57:49.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.648+0000 7fc357200640 1 Processor -- start 2024-04-26T20:57:49.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.648+0000 7fc357200640 1 -- start start 2024-04-26T20:57:49.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.649+0000 7fc357200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc33c004800 0x7fc358142490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:49.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.649+0000 7fc357200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc358139600 0x7fc3580d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:49.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.649+0000 7fc357200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc3580b1bf0 0x7fc3580b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:49.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.649+0000 7fc357200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc35806ff60 con 0x7fc358139600 2024-04-26T20:57:49.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.649+0000 7fc357200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc35806fc80 con 0x7fc33c004800 2024-04-26T20:57:49.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.649+0000 7fc357200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc358071e20 con 0x7fc3580b1bf0 2024-04-26T20:57:49.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.649+0000 7fc355400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc358139600 0x7fc3580d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:49.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.649+0000 7fc355400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc358139600 0x7fc3580d00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51286/0 (socket says 172.21.15.28:51286) 2024-04-26T20:57:49.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.649+0000 7fc355400640 1 -- 172.21.15.28:0/710623875 learned_addr learned my addr 172.21.15.28:0/710623875 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:49.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.649+0000 7fc356800640 1 --2- 172.21.15.28:0/710623875 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc3580b1bf0 0x7fc3580b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:49.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.650+0000 7fc355e00640 1 --2- 172.21.15.28:0/710623875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc33c004800 0x7fc358142490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:49.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.650+0000 7fc355400640 1 -- 172.21.15.28:0/710623875 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc3580b1bf0 msgr2=0x7fc3580b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:49.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.650+0000 7fc355400640 1 --2- 172.21.15.28:0/710623875 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc3580b1bf0 0x7fc3580b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:49.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.650+0000 7fc355400640 1 -- 172.21.15.28:0/710623875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc33c004800 msgr2=0x7fc358142490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:49.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.650+0000 7fc355400640 1 --2- 172.21.15.28:0/710623875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc33c004800 0x7fc358142490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:49.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.650+0000 7fc355400640 1 -- 172.21.15.28:0/710623875 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3580b3810 con 0x7fc358139600 2024-04-26T20:57:49.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.650+0000 7fc355e00640 1 --2- 172.21.15.28:0/710623875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc33c004800 0x7fc358142490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:49.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.650+0000 7fc355400640 1 --2- 172.21.15.28:0/710623875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc358139600 0x7fc3580d00f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc344001870 tx=0x7fc344001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:49.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.650+0000 7fc34e000640 1 -- 172.21.15.28:0/710623875 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344042070 con 0x7fc358139600 2024-04-26T20:57:49.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.650+0000 7fc34e000640 1 -- 172.21.15.28:0/710623875 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc34402be50 con 0x7fc358139600 2024-04-26T20:57:49.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.650+0000 7fc34e000640 1 -- 172.21.15.28:0/710623875 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344037040 con 0x7fc358139600 2024-04-26T20:57:49.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.650+0000 7fc357200640 1 -- 172.21.15.28:0/710623875 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3580b3ab0 con 0x7fc358139600 2024-04-26T20:57:49.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.651+0000 7fc357200640 1 -- 172.21.15.28:0/710623875 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3580b4070 con 0x7fc358139600 2024-04-26T20:57:49.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.652+0000 7fc357200640 1 -- 172.21.15.28:0/710623875 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc35806a360 con 0x7fc358139600 2024-04-26T20:57:49.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.653+0000 7fc34e000640 1 -- 172.21.15.28:0/710623875 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc344045070 con 0x7fc358139600 2024-04-26T20:57:49.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.653+0000 7fc34e000640 1 --2- 172.21.15.28:0/710623875 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc33806c920 0x7fc33806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:49.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.654+0000 7fc34e000640 1 -- 172.21.15.28:0/710623875 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fc3440ab450 con 0x7fc358139600 2024-04-26T20:57:49.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.654+0000 7fc355e00640 1 --2- 172.21.15.28:0/710623875 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc33806c920 0x7fc33806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:49.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.655+0000 7fc355e00640 1 --2- 172.21.15.28:0/710623875 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc33806c920 0x7fc33806ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc348009a00 tx=0x7fc348008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:49.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.658+0000 7fc34e000640 1 -- 172.21.15.28:0/710623875 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc344079760 con 0x7fc358139600 2024-04-26T20:57:49.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.868+0000 7fc357200640 1 -- 172.21.15.28:0/710623875 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc3580b5780 con 0x7fc358139600 2024-04-26T20:57:49.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.869+0000 7fc34e000640 1 -- 172.21.15.28:0/710623875 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fc344079060 con 0x7fc358139600 2024-04-26T20:57:49.905 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-04-26T20:57:49.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.872+0000 7fc357200640 1 -- 172.21.15.28:0/710623875 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc33806c920 msgr2=0x7fc33806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:49.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.872+0000 7fc357200640 1 --2- 172.21.15.28:0/710623875 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc33806c920 0x7fc33806ede0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc348009a00 tx=0x7fc348008040 comp rx=0 tx=0).stop 2024-04-26T20:57:49.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.872+0000 7fc357200640 1 -- 172.21.15.28:0/710623875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc358139600 msgr2=0x7fc3580d00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:49.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.872+0000 7fc357200640 1 --2- 172.21.15.28:0/710623875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc358139600 0x7fc3580d00f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc344001870 tx=0x7fc344001120 comp rx=0 tx=0).stop 2024-04-26T20:57:49.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.872+0000 7fc357200640 1 -- 172.21.15.28:0/710623875 shutdown_connections 2024-04-26T20:57:49.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.872+0000 7fc357200640 1 --2- 172.21.15.28:0/710623875 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc33806c920 0x7fc33806ede0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:49.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.872+0000 7fc357200640 1 --2- 172.21.15.28:0/710623875 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc3580b1bf0 0x7fc3580b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:49.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.872+0000 7fc357200640 1 --2- 172.21.15.28:0/710623875 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc358139600 0x7fc3580d00f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:49.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.872+0000 7fc357200640 1 --2- 172.21.15.28:0/710623875 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc33c004800 0x7fc358142490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:49.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.872+0000 7fc357200640 1 -- 172.21.15.28:0/710623875 >> 172.21.15.28:0/710623875 conn(0x7fc3580d0a50 msgr2=0x7fc3580b0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:49.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.872+0000 7fc357200640 1 -- 172.21.15.28:0/710623875 shutdown_connections 2024-04-26T20:57:49.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:49.872+0000 7fc357200640 1 -- 172.21.15.28:0/710623875 wait complete. 2024-04-26T20:57:49.906 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:57:49.906 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:57:49.965 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:57:50.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.178+0000 7f92a8a00640 1 Processor -- start 2024-04-26T20:57:50.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.179+0000 7f92a8a00640 1 -- start start 2024-04-26T20:57:50.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.180+0000 7f92a8a00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f92a415f1b0 0x7f92a40d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:50.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.180+0000 7f92a8a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f92a40cf700 0x7f92a40d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:50.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.180+0000 7f92a8a00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92a40d0710 con 0x7f92a40d5370 2024-04-26T20:57:50.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.180+0000 7f92a8a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f92a40d57e0 con 0x7f92a40cf700 2024-04-26T20:57:50.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.180+0000 7f92a8a00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f92a40cff00 con 0x7f92a415f1b0 2024-04-26T20:57:50.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.180+0000 7f92a2a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f92a40cf700 0x7f92a40d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:50.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.181+0000 7f92a2a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f92a40cf700 0x7f92a40d4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37988/0 (socket says 172.21.15.28:37988) 2024-04-26T20:57:50.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.181+0000 7f92a2a00640 1 -- 172.21.15.28:0/3770366714 learned_addr learned my addr 172.21.15.28:0/3770366714 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:50.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.181+0000 7f92a3400640 1 --2- 172.21.15.28:0/3770366714 >> v2:172.21.15.103:3300/0 conn(0x7f92a415f1b0 0x7f92a40d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:50.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.181+0000 7f92a2a00640 1 -- 172.21.15.28:0/3770366714 >> v2:172.21.15.103:3300/0 conn(0x7f92a415f1b0 msgr2=0x7f92a40d4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:50.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.181+0000 7f92a2a00640 1 --2- 172.21.15.28:0/3770366714 >> v2:172.21.15.103:3300/0 conn(0x7f92a415f1b0 0x7f92a40d4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.181+0000 7f92a2a00640 1 -- 172.21.15.28:0/3770366714 >> v1:172.21.15.103:6789/0 conn(0x7f92a40d5370 legacy=0x7f92a40b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:50.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.181+0000 7f92a2a00640 1 -- 172.21.15.28:0/3770366714 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a40d0150 con 0x7f92a40cf700 2024-04-26T20:57:50.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.182+0000 7f92a2a00640 1 --2- 172.21.15.28:0/3770366714 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f92a40cf700 0x7f92a40d4e30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f92a40d0080 tx=0x7f928c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed2613d90943dc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:50.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.182+0000 7f92a2000640 1 -- 172.21.15.28:0/3770366714 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c037070 con 0x7f92a40cf700 2024-04-26T20:57:50.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.182+0000 7f92a2000640 1 -- 172.21.15.28:0/3770366714 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f928c032440 con 0x7f92a40cf700 2024-04-26T20:57:50.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.182+0000 7f92a2000640 1 -- 172.21.15.28:0/3770366714 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928c032810 con 0x7f92a40cf700 2024-04-26T20:57:50.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.183+0000 7f92a8a00640 1 -- 172.21.15.28:0/3770366714 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f92a40cf700 msgr2=0x7f92a40d4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:50.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.183+0000 7f92a8a00640 1 --2- 172.21.15.28:0/3770366714 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f92a40cf700 0x7f92a40d4e30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f92a40d0080 tx=0x7f928c02a2c0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.183+0000 7f92a8a00640 1 -- 172.21.15.28:0/3770366714 shutdown_connections 2024-04-26T20:57:50.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.183+0000 7f92a8a00640 1 --2- 172.21.15.28:0/3770366714 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f92a40cf700 0x7f92a40d4e30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.183+0000 7f92a8a00640 1 --2- 172.21.15.28:0/3770366714 >> v2:172.21.15.103:3300/0 conn(0x7f92a415f1b0 0x7f92a40d4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.183+0000 7f92a8a00640 1 -- 172.21.15.28:0/3770366714 >> 172.21.15.28:0/3770366714 conn(0x7f92a406c770 msgr2=0x7f92a406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:50.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.183+0000 7f92a8a00640 1 -- 172.21.15.28:0/3770366714 shutdown_connections 2024-04-26T20:57:50.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.183+0000 7f92a8a00640 1 -- 172.21.15.28:0/3770366714 wait complete. 2024-04-26T20:57:50.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.185+0000 7f92a8a00640 1 Processor -- start 2024-04-26T20:57:50.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.185+0000 7f92a8a00640 1 -- start start 2024-04-26T20:57:50.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.186+0000 7f92a8a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f92a40cf700 0x7f92a414f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:50.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.186+0000 7f92a8a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f92a40d5370 0x7f92a40ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:50.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.186+0000 7f92a8a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92a415f1b0 0x7f92a4160d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:50.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.186+0000 7f92a8a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f92a40d3f00 con 0x7f92a40cf700 2024-04-26T20:57:50.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.186+0000 7f92a8a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f92a40d4270 con 0x7f92a415f1b0 2024-04-26T20:57:50.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.186+0000 7f92a8a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f92a415f590 con 0x7f92a40d5370 2024-04-26T20:57:50.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.186+0000 7f92a3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f92a40cf700 0x7f92a414f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:50.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.186+0000 7f92a3400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f92a40cf700 0x7f92a414f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37994/0 (socket says 172.21.15.28:37994) 2024-04-26T20:57:50.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.186+0000 7f92a3400640 1 -- 172.21.15.28:0/3132282570 learned_addr learned my addr 172.21.15.28:0/3132282570 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:50.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.187+0000 7f92a2a00640 1 --2- 172.21.15.28:0/3132282570 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f92a40d5370 0x7f92a40ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:50.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.187+0000 7f92a3e00640 1 --2- 172.21.15.28:0/3132282570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92a415f1b0 0x7f92a4160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:50.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.187+0000 7f92a3400640 1 -- 172.21.15.28:0/3132282570 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f92a40d5370 msgr2=0x7f92a40ca770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:50.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.187+0000 7f92a3400640 1 --2- 172.21.15.28:0/3132282570 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f92a40d5370 0x7f92a40ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.187+0000 7f92a3400640 1 -- 172.21.15.28:0/3132282570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92a415f1b0 msgr2=0x7f92a4160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:50.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.187+0000 7f92a3400640 1 --2- 172.21.15.28:0/3132282570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92a415f1b0 0x7f92a4160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.187+0000 7f92a3400640 1 -- 172.21.15.28:0/3132282570 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a40c9480 con 0x7f92a40cf700 2024-04-26T20:57:50.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.187+0000 7f92a3e00640 1 --2- 172.21.15.28:0/3132282570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92a415f1b0 0x7f92a4160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:50.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.187+0000 7f92a3400640 1 --2- 172.21.15.28:0/3132282570 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f92a40cf700 0x7f92a414f1b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f929400c7f0 tx=0x7f929400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:50.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.187+0000 7f929b400640 1 -- 172.21.15.28:0/3132282570 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294015070 con 0x7f92a40cf700 2024-04-26T20:57:50.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.187+0000 7f929b400640 1 -- 172.21.15.28:0/3132282570 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9294004240 con 0x7f92a40cf700 2024-04-26T20:57:50.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.187+0000 7f929b400640 1 -- 172.21.15.28:0/3132282570 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294007620 con 0x7f92a40cf700 2024-04-26T20:57:50.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.187+0000 7f92a8a00640 1 -- 172.21.15.28:0/3132282570 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a40c9760 con 0x7f92a40cf700 2024-04-26T20:57:50.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.187+0000 7f92a8a00640 1 -- 172.21.15.28:0/3132282570 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a40c9d20 con 0x7f92a40cf700 2024-04-26T20:57:50.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.189+0000 7f92a8a00640 1 -- 172.21.15.28:0/3132282570 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a406a360 con 0x7f92a40cf700 2024-04-26T20:57:50.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.190+0000 7f929b400640 1 -- 172.21.15.28:0/3132282570 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f92940079d0 con 0x7f92a40cf700 2024-04-26T20:57:50.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.190+0000 7f929b400640 1 --2- 172.21.15.28:0/3132282570 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f928006c8f0 0x7f928006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:50.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.191+0000 7f929b400640 1 -- 172.21.15.28:0/3132282570 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f929408c100 con 0x7f92a40cf700 2024-04-26T20:57:50.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.191+0000 7f92a2a00640 1 --2- 172.21.15.28:0/3132282570 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f928006c8f0 0x7f928006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:50.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.192+0000 7f92a2a00640 1 --2- 172.21.15.28:0/3132282570 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f928006c8f0 0x7f928006edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f928c034000 tx=0x7f928c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:50.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.195+0000 7f929b400640 1 -- 172.21.15.28:0/3132282570 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f929405a410 con 0x7f92a40cf700 2024-04-26T20:57:50.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.391+0000 7f92a8a00640 1 -- 172.21.15.28:0/3132282570 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92a4068bd0 con 0x7f92a40cf700 2024-04-26T20:57:50.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.393+0000 7f929b400640 1 -- 172.21.15.28:0/3132282570 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16831 (secure 0 0 0) 0x7f9294059d10 con 0x7f92a40cf700 2024-04-26T20:57:50.394 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:50.394 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":23,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:57:37.996297+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:57:50.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.396+0000 7f92a8a00640 1 -- 172.21.15.28:0/3132282570 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f928006c8f0 msgr2=0x7f928006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:50.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.396+0000 7f92a8a00640 1 --2- 172.21.15.28:0/3132282570 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f928006c8f0 0x7f928006edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f928c034000 tx=0x7f928c02e000 comp rx=0 tx=0).stop 2024-04-26T20:57:50.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.396+0000 7f92a8a00640 1 -- 172.21.15.28:0/3132282570 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f92a40cf700 msgr2=0x7f92a414f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:50.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.396+0000 7f92a8a00640 1 --2- 172.21.15.28:0/3132282570 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f92a40cf700 0x7f92a414f1b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f929400c7f0 tx=0x7f929400ccb0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.396+0000 7f92a8a00640 1 -- 172.21.15.28:0/3132282570 shutdown_connections 2024-04-26T20:57:50.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.396+0000 7f92a8a00640 1 --2- 172.21.15.28:0/3132282570 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f928006c8f0 0x7f928006edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.396+0000 7f92a8a00640 1 --2- 172.21.15.28:0/3132282570 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f92a415f1b0 0x7f92a4160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.396+0000 7f92a8a00640 1 --2- 172.21.15.28:0/3132282570 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f92a40d5370 0x7f92a40ca770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.396+0000 7f92a8a00640 1 --2- 172.21.15.28:0/3132282570 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f92a40cf700 0x7f92a414f1b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.396+0000 7f92a8a00640 1 -- 172.21.15.28:0/3132282570 >> 172.21.15.28:0/3132282570 conn(0x7f92a406c770 msgr2=0x7f92a406cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:50.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.397+0000 7f92a8a00640 1 -- 172.21.15.28:0/3132282570 shutdown_connections 2024-04-26T20:57:50.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.397+0000 7f92a8a00640 1 -- 172.21.15.28:0/3132282570 wait complete. 2024-04-26T20:57:50.416 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:56:48.688498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:56:50.117541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:57:50.417 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:57:50.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.627+0000 7f4c1ac00640 1 Processor -- start 2024-04-26T20:57:50.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.627+0000 7f4c1ac00640 1 -- start start 2024-04-26T20:57:50.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.629+0000 7f4c1ac00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c1414ce40 0x7f4c1406f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:50.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.629+0000 7f4c1ac00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4c14166f50 0x7f4c140d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:50.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.629+0000 7f4c1ac00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c140721e0 con 0x7f4c140d1250 2024-04-26T20:57:50.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.629+0000 7f4c1ac00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f4c140cd4e0 con 0x7f4c14166f50 2024-04-26T20:57:50.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.629+0000 7f4c1ac00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f4c1406fdd0 con 0x7f4c1414ce40 2024-04-26T20:57:50.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.630+0000 7f4c18e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4c14166f50 0x7f4c140d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:50.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.630+0000 7f4c18e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4c14166f50 0x7f4c140d0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38004/0 (socket says 172.21.15.28:38004) 2024-04-26T20:57:50.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.630+0000 7f4c18e00640 1 -- 172.21.15.28:0/3279537848 learned_addr learned my addr 172.21.15.28:0/3279537848 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:50.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.630+0000 7f4c19800640 1 --2- 172.21.15.28:0/3279537848 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c1414ce40 0x7f4c1406f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:50.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.630+0000 7f4c18e00640 1 -- 172.21.15.28:0/3279537848 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c1414ce40 msgr2=0x7f4c1406f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:50.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.630+0000 7f4c18e00640 1 --2- 172.21.15.28:0/3279537848 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c1414ce40 0x7f4c1406f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.630+0000 7f4c18e00640 1 -- 172.21.15.28:0/3279537848 >> v1:172.21.15.103:6789/0 conn(0x7f4c140d1250 legacy=0x7f4c1406df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:50.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.630+0000 7f4c18e00640 1 -- 172.21.15.28:0/3279537848 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c1414d220 con 0x7f4c14166f50 2024-04-26T20:57:50.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.631+0000 7f4c18e00640 1 --2- 172.21.15.28:0/3279537848 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c14166f50 0x7f4c140d0d10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4c04009870 tx=0x7f4c0402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e98a4105415357d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:50.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.632+0000 7f4c13e00640 1 -- 172.21.15.28:0/3279537848 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c04037070 con 0x7f4c14166f50 2024-04-26T20:57:50.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.632+0000 7f4c13e00640 1 -- 172.21.15.28:0/3279537848 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c04032440 con 0x7f4c14166f50 2024-04-26T20:57:50.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.632+0000 7f4c13e00640 1 -- 172.21.15.28:0/3279537848 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c04032810 con 0x7f4c14166f50 2024-04-26T20:57:50.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.632+0000 7f4c1ac00640 1 -- 172.21.15.28:0/3279537848 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c14166f50 msgr2=0x7f4c140d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:50.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.632+0000 7f4c1ac00640 1 --2- 172.21.15.28:0/3279537848 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c14166f50 0x7f4c140d0d10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4c04009870 tx=0x7f4c0402a210 comp rx=0 tx=0).stop 2024-04-26T20:57:50.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.632+0000 7f4c1ac00640 1 -- 172.21.15.28:0/3279537848 shutdown_connections 2024-04-26T20:57:50.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.632+0000 7f4c1ac00640 1 --2- 172.21.15.28:0/3279537848 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c14166f50 0x7f4c140d0d10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.632+0000 7f4c1ac00640 1 --2- 172.21.15.28:0/3279537848 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c1414ce40 0x7f4c1406f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.632+0000 7f4c1ac00640 1 -- 172.21.15.28:0/3279537848 >> 172.21.15.28:0/3279537848 conn(0x7f4c1406c530 msgr2=0x7f4c1406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:50.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.633+0000 7f4c1ac00640 1 -- 172.21.15.28:0/3279537848 shutdown_connections 2024-04-26T20:57:50.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.633+0000 7f4c1ac00640 1 -- 172.21.15.28:0/3279537848 wait complete. 2024-04-26T20:57:50.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.634+0000 7f4c1ac00640 1 Processor -- start 2024-04-26T20:57:50.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.634+0000 7f4c1ac00640 1 -- start start 2024-04-26T20:57:50.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.635+0000 7f4c1ac00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c140d1250 0x7f4c1413b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:50.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.635+0000 7f4c1ac00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c1414ce40 0x7f4c140b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:50.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.635+0000 7f4c1ac00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c14166f50 0x7f4c140b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:50.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.635+0000 7f4c1ac00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4c14167a80 con 0x7f4c1414ce40 2024-04-26T20:57:50.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.635+0000 7f4c1ac00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c141677a0 con 0x7f4c140d1250 2024-04-26T20:57:50.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.635+0000 7f4c1ac00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f4c14167d60 con 0x7f4c14166f50 2024-04-26T20:57:50.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.636+0000 7f4c18e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c1414ce40 0x7f4c140b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:50.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.636+0000 7f4c18e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c1414ce40 0x7f4c140b16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38010/0 (socket says 172.21.15.28:38010) 2024-04-26T20:57:50.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.636+0000 7f4c18e00640 1 -- 172.21.15.28:0/1621809663 learned_addr learned my addr 172.21.15.28:0/1621809663 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:50.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.636+0000 7f4c19800640 1 --2- 172.21.15.28:0/1621809663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c140d1250 0x7f4c1413b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:50.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.636+0000 7f4c18e00640 1 -- 172.21.15.28:0/1621809663 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c14166f50 msgr2=0x7f4c140b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:50.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.636+0000 7f4c18e00640 1 --2- 172.21.15.28:0/1621809663 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c14166f50 0x7f4c140b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.636+0000 7f4c18e00640 1 -- 172.21.15.28:0/1621809663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c140d1250 msgr2=0x7f4c1413b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:50.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.636+0000 7f4c18e00640 1 --2- 172.21.15.28:0/1621809663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c140d1250 0x7f4c1413b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.636+0000 7f4c18e00640 1 -- 172.21.15.28:0/1621809663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c140b24a0 con 0x7f4c1414ce40 2024-04-26T20:57:50.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.637+0000 7f4c18e00640 1 --2- 172.21.15.28:0/1621809663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c1414ce40 0x7f4c140b16f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4c040090b0 tx=0x7f4c040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:50.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.637+0000 7f4c11600640 1 -- 172.21.15.28:0/1621809663 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c04041020 con 0x7f4c1414ce40 2024-04-26T20:57:50.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.637+0000 7f4c11600640 1 -- 172.21.15.28:0/1621809663 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c040306c0 con 0x7f4c1414ce40 2024-04-26T20:57:50.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.637+0000 7f4c11600640 1 -- 172.21.15.28:0/1621809663 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c04037040 con 0x7f4c1414ce40 2024-04-26T20:57:50.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.637+0000 7f4c1ac00640 1 -- 172.21.15.28:0/1621809663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c141386d0 con 0x7f4c1414ce40 2024-04-26T20:57:50.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.637+0000 7f4c1ac00640 1 -- 172.21.15.28:0/1621809663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c14138c90 con 0x7f4c1414ce40 2024-04-26T20:57:50.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.639+0000 7f4c1ac00640 1 -- 172.21.15.28:0/1621809663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c1406a250 con 0x7f4c1414ce40 2024-04-26T20:57:50.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.640+0000 7f4c11600640 1 -- 172.21.15.28:0/1621809663 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c04044020 con 0x7f4c1414ce40 2024-04-26T20:57:50.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.641+0000 7f4c11600640 1 --2- 172.21.15.28:0/1621809663 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4bf806c920 0x7f4bf806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:50.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.641+0000 7f4c11600640 1 -- 172.21.15.28:0/1621809663 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f4c040ac3f0 con 0x7f4c1414ce40 2024-04-26T20:57:50.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.641+0000 7f4c19800640 1 --2- 172.21.15.28:0/1621809663 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4bf806c920 0x7f4bf806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:50.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.642+0000 7f4c19800640 1 --2- 172.21.15.28:0/1621809663 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4bf806c920 0x7f4bf806ede0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4c08006fd0 tx=0x7f4c08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:50.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.645+0000 7f4c11600640 1 -- 172.21.15.28:0/1621809663 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4c0404b030 con 0x7f4c1414ce40 2024-04-26T20:57:50.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.849+0000 7f4c1ac00640 1 -- 172.21.15.28:0/1621809663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4c14166cc0 con 0x7f4c1414ce40 2024-04-26T20:57:50.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.850+0000 7f4c11600640 1 -- 172.21.15.28:0/1621809663 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f4c0407a830 con 0x7f4c1414ce40 2024-04-26T20:57:50.850 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:57:50.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.853+0000 7f4c1ac00640 1 -- 172.21.15.28:0/1621809663 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4bf806c920 msgr2=0x7f4bf806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:50.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.853+0000 7f4c1ac00640 1 --2- 172.21.15.28:0/1621809663 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4bf806c920 0x7f4bf806ede0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4c08006fd0 tx=0x7f4c08008040 comp rx=0 tx=0).stop 2024-04-26T20:57:50.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.853+0000 7f4c1ac00640 1 -- 172.21.15.28:0/1621809663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c1414ce40 msgr2=0x7f4c140b16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:50.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.853+0000 7f4c1ac00640 1 --2- 172.21.15.28:0/1621809663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c1414ce40 0x7f4c140b16f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4c040090b0 tx=0x7f4c040046c0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.853+0000 7f4c1ac00640 1 -- 172.21.15.28:0/1621809663 shutdown_connections 2024-04-26T20:57:50.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.853+0000 7f4c1ac00640 1 --2- 172.21.15.28:0/1621809663 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4bf806c920 0x7f4bf806ede0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.853+0000 7f4c1ac00640 1 --2- 172.21.15.28:0/1621809663 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c14166f50 0x7f4c140b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.853+0000 7f4c1ac00640 1 --2- 172.21.15.28:0/1621809663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c1414ce40 0x7f4c140b16f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.853+0000 7f4c1ac00640 1 --2- 172.21.15.28:0/1621809663 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c140d1250 0x7f4c1413b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:50.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.853+0000 7f4c1ac00640 1 -- 172.21.15.28:0/1621809663 >> 172.21.15.28:0/1621809663 conn(0x7f4c1406c530 msgr2=0x7f4c140cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:50.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.853+0000 7f4c1ac00640 1 -- 172.21.15.28:0/1621809663 shutdown_connections 2024-04-26T20:57:50.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:50.853+0000 7f4c1ac00640 1 -- 172.21.15.28:0/1621809663 wait complete. 2024-04-26T20:57:50.873 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:57:51.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.086+0000 7fa803e00640 1 Processor -- start 2024-04-26T20:57:51.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.086+0000 7fa803e00640 1 -- start start 2024-04-26T20:57:51.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.088+0000 7fa803e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa7fc06fc10 0x7fa7fc073f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:51.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.089+0000 7fa803e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7fc15e310 con 0x7fa7fc073580 2024-04-26T20:57:51.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.089+0000 7fa803e00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7fc072aa0 con 0x7fa7fc0ceaa0 2024-04-26T20:57:51.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.089+0000 7fa803e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fa7fc074490 con 0x7fa7fc06fc10 2024-04-26T20:57:51.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.089+0000 7fa800a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa7fc073580 0x7fa7fc0ce390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38836/0 (socket says 172.21.15.28:38836) 2024-04-26T20:57:51.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.089+0000 7fa800a00640 1 -- 172.21.15.28:0/3868446649 learned_addr learned my addr 172.21.15.28:0/3868446649 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:51.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.089+0000 7fa801400640 1 --2- 172.21.15.28:0/3868446649 >> v2:172.21.15.103:3300/0 conn(0x7fa7fc06fc10 0x7fa7fc073f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:51.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.090+0000 7fa7fbe00640 1 -- 172.21.15.28:0/3868446649 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 113404501 0 0) 0x7fa7fc15e310 con 0x7fa7fc073580 2024-04-26T20:57:51.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.090+0000 7fa7fbe00640 1 -- 172.21.15.28:0/3868446649 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7e0003520 con 0x7fa7fc073580 2024-04-26T20:57:51.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.090+0000 7fa801400640 1 -- 172.21.15.28:0/3868446649 >> v1:172.21.15.103:6789/0 conn(0x7fa7fc0ceaa0 legacy=0x7fa7fc06e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:51.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.090+0000 7fa801400640 1 -- 172.21.15.28:0/3868446649 >> v1:172.21.15.28:6789/0 conn(0x7fa7fc073580 legacy=0x7fa7fc0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:51.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.090+0000 7fa801400640 1 -- 172.21.15.28:0/3868446649 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc072d80 con 0x7fa7fc06fc10 2024-04-26T20:57:51.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.091+0000 7fa801400640 1 --2- 172.21.15.28:0/3868446649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7fc06fc10 0x7fa7fc073f50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009d90 tx=0x7fa7ec02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11b1d59d1db76a2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:51.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.097+0000 7fa7fbe00640 1 -- 172.21.15.28:0/3868446649 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec007740 con 0x7fa7fc06fc10 2024-04-26T20:57:51.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.097+0000 7fa7fbe00640 1 -- 172.21.15.28:0/3868446649 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7ec0078e0 con 0x7fa7fc06fc10 2024-04-26T20:57:51.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.097+0000 7fa7fbe00640 1 -- 172.21.15.28:0/3868446649 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec007cb0 con 0x7fa7fc06fc10 2024-04-26T20:57:51.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.098+0000 7fa803e00640 1 -- 172.21.15.28:0/3868446649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7fc06fc10 msgr2=0x7fa7fc073f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:51.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.098+0000 7fa803e00640 1 --2- 172.21.15.28:0/3868446649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7fc06fc10 0x7fa7fc073f50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec009d90 tx=0x7fa7ec02a660 comp rx=0 tx=0).stop 2024-04-26T20:57:51.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.098+0000 7fa803e00640 1 -- 172.21.15.28:0/3868446649 shutdown_connections 2024-04-26T20:57:51.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.098+0000 7fa803e00640 1 --2- 172.21.15.28:0/3868446649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7fc06fc10 0x7fa7fc073f50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.098+0000 7fa803e00640 1 -- 172.21.15.28:0/3868446649 >> 172.21.15.28:0/3868446649 conn(0x7fa7fc06c770 msgr2=0x7fa7fc06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:51.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.098+0000 7fa803e00640 1 -- 172.21.15.28:0/3868446649 shutdown_connections 2024-04-26T20:57:51.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.098+0000 7fa803e00640 1 -- 172.21.15.28:0/3868446649 wait complete. 2024-04-26T20:57:51.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.100+0000 7fa803e00640 1 Processor -- start 2024-04-26T20:57:51.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.100+0000 7fa803e00640 1 -- start start 2024-04-26T20:57:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.101+0000 7fa803e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa7fc06fc10 0x7fa7fc14b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.101+0000 7fa803e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7fc073580 0x7fa7fc1509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.101+0000 7fa803e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7fc0ceaa0 0x7fa7fc143570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.101+0000 7fa803e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc073a30 con 0x7fa7fc0ceaa0 2024-04-26T20:57:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.101+0000 7fa803e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc15dfa0 con 0x7fa7fc073580 2024-04-26T20:57:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.101+0000 7fa803e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa7fc154840 con 0x7fa7fc06fc10 2024-04-26T20:57:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.101+0000 7fa801e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7fc0ceaa0 0x7fa7fc143570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.101+0000 7fa801e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7fc0ceaa0 0x7fa7fc143570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38020/0 (socket says 172.21.15.28:38020) 2024-04-26T20:57:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.101+0000 7fa801e00640 1 -- 172.21.15.28:0/644852987 learned_addr learned my addr 172.21.15.28:0/644852987 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:51.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.102+0000 7fa800a00640 1 --2- 172.21.15.28:0/644852987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7fc073580 0x7fa7fc1509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:51.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.102+0000 7fa801400640 1 --2- 172.21.15.28:0/644852987 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa7fc06fc10 0x7fa7fc14b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:51.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.102+0000 7fa801e00640 1 -- 172.21.15.28:0/644852987 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa7fc06fc10 msgr2=0x7fa7fc14b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:51.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.102+0000 7fa801e00640 1 --2- 172.21.15.28:0/644852987 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa7fc06fc10 0x7fa7fc14b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.102+0000 7fa801e00640 1 -- 172.21.15.28:0/644852987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7fc073580 msgr2=0x7fa7fc1509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:51.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.102+0000 7fa801e00640 1 --2- 172.21.15.28:0/644852987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7fc073580 0x7fa7fc1509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.102+0000 7fa801e00640 1 -- 172.21.15.28:0/644852987 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc14bbf0 con 0x7fa7fc0ceaa0 2024-04-26T20:57:51.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.102+0000 7fa801e00640 1 --2- 172.21.15.28:0/644852987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7fc0ceaa0 0x7fa7fc143570 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4001eb0 tx=0x7fa7f400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.102+0000 7fa7f9600640 1 -- 172.21.15.28:0/644852987 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4017070 con 0x7fa7fc0ceaa0 2024-04-26T20:57:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.102+0000 7fa7f9600640 1 -- 172.21.15.28:0/644852987 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7f40052d0 con 0x7fa7fc0ceaa0 2024-04-26T20:57:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.102+0000 7fa7f9600640 1 -- 172.21.15.28:0/644852987 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4003810 con 0x7fa7fc0ceaa0 2024-04-26T20:57:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.102+0000 7fa803e00640 1 -- 172.21.15.28:0/644852987 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7fc14ca50 con 0x7fa7fc0ceaa0 2024-04-26T20:57:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.103+0000 7fa803e00640 1 -- 172.21.15.28:0/644852987 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7fc14d010 con 0x7fa7fc0ceaa0 2024-04-26T20:57:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.104+0000 7fa803e00640 1 -- 172.21.15.28:0/644852987 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7fc14d210 con 0x7fa7fc0ceaa0 2024-04-26T20:57:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.105+0000 7fa7f9600640 1 -- 172.21.15.28:0/644852987 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7f401a070 con 0x7fa7fc0ceaa0 2024-04-26T20:57:51.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.106+0000 7fa7f9600640 1 --2- 172.21.15.28:0/644852987 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa7dc06c8f0 0x7fa7dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:51.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.106+0000 7fa7f9600640 1 -- 172.21.15.28:0/644852987 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fa7f408af40 con 0x7fa7fc0ceaa0 2024-04-26T20:57:51.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.106+0000 7fa801400640 1 --2- 172.21.15.28:0/644852987 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa7dc06c8f0 0x7fa7dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:51.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.107+0000 7fa801400640 1 --2- 172.21.15.28:0/644852987 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa7dc06c8f0 0x7fa7dc06edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec034000 tx=0x7fa7ec033040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:51.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.110+0000 7fa7f9600640 1 -- 172.21.15.28:0/644852987 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa7f40592e0 con 0x7fa7fc0ceaa0 2024-04-26T20:57:51.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.313+0000 7fa803e00640 1 -- 172.21.15.28:0/644852987 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa7fc0d0160 con 0x7fa7fc0ceaa0 2024-04-26T20:57:51.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.313+0000 7fa7f9600640 1 -- 172.21.15.28:0/644852987 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fa7fc0d0160 con 0x7fa7fc0ceaa0 2024-04-26T20:57:51.315 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-04-26T20:57:51.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.316+0000 7fa803e00640 1 -- 172.21.15.28:0/644852987 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa7dc06c8f0 msgr2=0x7fa7dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:51.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.316+0000 7fa803e00640 1 --2- 172.21.15.28:0/644852987 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa7dc06c8f0 0x7fa7dc06edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec034000 tx=0x7fa7ec033040 comp rx=0 tx=0).stop 2024-04-26T20:57:51.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.316+0000 7fa803e00640 1 -- 172.21.15.28:0/644852987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7fc0ceaa0 msgr2=0x7fa7fc143570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:51.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.316+0000 7fa803e00640 1 --2- 172.21.15.28:0/644852987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7fc0ceaa0 0x7fa7fc143570 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4001eb0 tx=0x7fa7f400c890 comp rx=0 tx=0).stop 2024-04-26T20:57:51.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.316+0000 7fa803e00640 1 -- 172.21.15.28:0/644852987 shutdown_connections 2024-04-26T20:57:51.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.316+0000 7fa803e00640 1 --2- 172.21.15.28:0/644852987 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa7dc06c8f0 0x7fa7dc06edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.316+0000 7fa803e00640 1 --2- 172.21.15.28:0/644852987 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7fc0ceaa0 0x7fa7fc143570 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.316+0000 7fa803e00640 1 --2- 172.21.15.28:0/644852987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7fc073580 0x7fa7fc1509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.316+0000 7fa803e00640 1 --2- 172.21.15.28:0/644852987 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa7fc06fc10 0x7fa7fc14b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.316+0000 7fa803e00640 1 -- 172.21.15.28:0/644852987 >> 172.21.15.28:0/644852987 conn(0x7fa7fc06c770 msgr2=0x7fa7fc15cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:51.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.316+0000 7fa803e00640 1 -- 172.21.15.28:0/644852987 shutdown_connections 2024-04-26T20:57:51.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.316+0000 7fa803e00640 1 -- 172.21.15.28:0/644852987 wait complete. 2024-04-26T20:57:51.336 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:57:51.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.528+0000 7fccb4a00640 1 Processor -- start 2024-04-26T20:57:51.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.528+0000 7fccb4a00640 1 -- start start 2024-04-26T20:57:51.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.530+0000 7fccb4a00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fccb00d0c70 0x7fccb0161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.530+0000 7fccb4a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fccb00d1050 0x7fccb006df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.530+0000 7fccb4a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fccb006e460 0x7fccb006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.530+0000 7fccb4a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fccb0161860 con 0x7fccb00d1050 2024-04-26T20:57:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.530+0000 7fccb4a00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fccb00cf2f0 con 0x7fccb00d0c70 2024-04-26T20:57:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.530+0000 7fccb4a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fccb00cf5d0 con 0x7fccb006e460 2024-04-26T20:57:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.530+0000 7fccaea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fccb00d1050 0x7fccb006df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.530+0000 7fccaea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fccb00d1050 0x7fccb006df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38034/0 (socket says 172.21.15.28:38034) 2024-04-26T20:57:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.530+0000 7fccaea00640 1 -- 172.21.15.28:0/1548431632 learned_addr learned my addr 172.21.15.28:0/1548431632 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.530+0000 7fccaea00640 1 -- 172.21.15.28:0/1548431632 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fccb006e460 msgr2=0x7fccb006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.530+0000 7fccaea00640 1 --2- 172.21.15.28:0/1548431632 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fccb006e460 0x7fccb006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.530+0000 7fccaea00640 1 -- 172.21.15.28:0/1548431632 >> v2:172.21.15.103:3300/0 conn(0x7fccb00d0c70 msgr2=0x7fccb0161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.530+0000 7fccaea00640 1 --2- 172.21.15.28:0/1548431632 >> v2:172.21.15.103:3300/0 conn(0x7fccb00d0c70 0x7fccb0161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.530+0000 7fccaea00640 1 -- 172.21.15.28:0/1548431632 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb00cf8b0 con 0x7fccb00d1050 2024-04-26T20:57:51.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.531+0000 7fccaea00640 1 --2- 172.21.15.28:0/1548431632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccb00d1050 0x7fccb006df20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fccb00743a0 tx=0x7fcc9c02a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e0075dca8a04f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:51.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.532+0000 7fccae000640 1 -- 172.21.15.28:0/1548431632 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c037070 con 0x7fccb00d1050 2024-04-26T20:57:51.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.532+0000 7fccae000640 1 -- 172.21.15.28:0/1548431632 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc9c032440 con 0x7fccb00d1050 2024-04-26T20:57:51.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.532+0000 7fccae000640 1 -- 172.21.15.28:0/1548431632 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c032810 con 0x7fccb00d1050 2024-04-26T20:57:51.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.533+0000 7fccb4a00640 1 -- 172.21.15.28:0/1548431632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccb00d1050 msgr2=0x7fccb006df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:51.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.533+0000 7fccb4a00640 1 --2- 172.21.15.28:0/1548431632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccb00d1050 0x7fccb006df20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fccb00743a0 tx=0x7fcc9c02a1d0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.533+0000 7fccb4a00640 1 -- 172.21.15.28:0/1548431632 shutdown_connections 2024-04-26T20:57:51.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.533+0000 7fccb4a00640 1 --2- 172.21.15.28:0/1548431632 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fccb006e460 0x7fccb006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.533+0000 7fccb4a00640 1 --2- 172.21.15.28:0/1548431632 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccb00d1050 0x7fccb006df20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.533+0000 7fccb4a00640 1 --2- 172.21.15.28:0/1548431632 >> v2:172.21.15.103:3300/0 conn(0x7fccb00d0c70 0x7fccb0161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.533+0000 7fccb4a00640 1 -- 172.21.15.28:0/1548431632 >> 172.21.15.28:0/1548431632 conn(0x7fccb006c530 msgr2=0x7fccb006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:51.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.533+0000 7fccb4a00640 1 -- 172.21.15.28:0/1548431632 shutdown_connections 2024-04-26T20:57:51.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.533+0000 7fccb4a00640 1 -- 172.21.15.28:0/1548431632 wait complete. 2024-04-26T20:57:51.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.534+0000 7fccb4a00640 1 Processor -- start 2024-04-26T20:57:51.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.535+0000 7fccb4a00640 1 -- start start 2024-04-26T20:57:51.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.535+0000 7fccb4a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fccb006e460 0x7fccb015c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:51.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.535+0000 7fccb4a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccb00d0c70 0x7fccb015c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:51.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.535+0000 7fccb4a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccb00d1050 0x7fccb013ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:51.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.535+0000 7fccb4a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fccb0146840 con 0x7fccb00d1050 2024-04-26T20:57:51.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.535+0000 7fccb4a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fccb00cfe70 con 0x7fccb00d0c70 2024-04-26T20:57:51.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.535+0000 7fccb4a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fccb00d14e0 con 0x7fccb006e460 2024-04-26T20:57:51.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.536+0000 7fccafe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccb00d1050 0x7fccb013ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:51.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.536+0000 7fccafe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccb00d1050 0x7fccb013ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38038/0 (socket says 172.21.15.28:38038) 2024-04-26T20:57:51.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.536+0000 7fccafe00640 1 -- 172.21.15.28:0/3994880701 learned_addr learned my addr 172.21.15.28:0/3994880701 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:51.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.536+0000 7fccaea00640 1 --2- 172.21.15.28:0/3994880701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccb00d0c70 0x7fccb015c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:51.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.536+0000 7fccafe00640 1 -- 172.21.15.28:0/3994880701 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fccb006e460 msgr2=0x7fccb015c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:51.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.536+0000 7fccaf400640 1 --2- 172.21.15.28:0/3994880701 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fccb006e460 0x7fccb015c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:51.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.536+0000 7fccafe00640 1 --2- 172.21.15.28:0/3994880701 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fccb006e460 0x7fccb015c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.536+0000 7fccafe00640 1 -- 172.21.15.28:0/3994880701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccb00d0c70 msgr2=0x7fccb015c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:51.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.536+0000 7fccafe00640 1 --2- 172.21.15.28:0/3994880701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccb00d0c70 0x7fccb015c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.536+0000 7fccafe00640 1 -- 172.21.15.28:0/3994880701 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb01412c0 con 0x7fccb00d1050 2024-04-26T20:57:51.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.536+0000 7fccaea00640 1 --2- 172.21.15.28:0/3994880701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccb00d0c70 0x7fccb015c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:51.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.537+0000 7fccafe00640 1 --2- 172.21.15.28:0/3994880701 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccb00d1050 0x7fccb013ee40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcca800b970 tx=0x7fcca800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:51.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.537+0000 7fcca7400640 1 -- 172.21.15.28:0/3994880701 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8019070 con 0x7fccb00d1050 2024-04-26T20:57:51.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.537+0000 7fcca7400640 1 -- 172.21.15.28:0/3994880701 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcca800cb60 con 0x7fccb00d1050 2024-04-26T20:57:51.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.537+0000 7fccb4a00640 1 -- 172.21.15.28:0/3994880701 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb01415a0 con 0x7fccb00d1050 2024-04-26T20:57:51.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.537+0000 7fcca7400640 1 -- 172.21.15.28:0/3994880701 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcca8014440 con 0x7fccb00d1050 2024-04-26T20:57:51.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.537+0000 7fccb4a00640 1 -- 172.21.15.28:0/3994880701 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb0141b60 con 0x7fccb00d1050 2024-04-26T20:57:51.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.538+0000 7fccb4a00640 1 -- 172.21.15.28:0/3994880701 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccb006a2c0 con 0x7fccb00d1050 2024-04-26T20:57:51.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.539+0000 7fcca7400640 1 -- 172.21.15.28:0/3994880701 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcca801c070 con 0x7fccb00d1050 2024-04-26T20:57:51.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.540+0000 7fcca7400640 1 --2- 172.21.15.28:0/3994880701 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcc9006c8f0 0x7fcc9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:51.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.540+0000 7fcca7400640 1 -- 172.21.15.28:0/3994880701 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fcca808bab0 con 0x7fccb00d1050 2024-04-26T20:57:51.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.540+0000 7fccaf400640 1 --2- 172.21.15.28:0/3994880701 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcc9006c8f0 0x7fcc9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:51.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.541+0000 7fccaf400640 1 --2- 172.21.15.28:0/3994880701 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcc9006c8f0 0x7fcc9006edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcca000a810 tx=0x7fcca0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:51.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.545+0000 7fcca7400640 1 -- 172.21.15.28:0/3994880701 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcca8059d30 con 0x7fccb00d1050 2024-04-26T20:57:51.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.749+0000 7fccb4a00640 1 -- 172.21.15.28:0/3994880701 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fccb00d07a0 con 0x7fccb00d1050 2024-04-26T20:57:51.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.750+0000 7fcca7400640 1 -- 172.21.15.28:0/3994880701 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7fcca8059630 con 0x7fccb00d1050 2024-04-26T20:57:51.751 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-04-26T20:57:51.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.752+0000 7fccb4a00640 1 -- 172.21.15.28:0/3994880701 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcc9006c8f0 msgr2=0x7fcc9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:51.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.752+0000 7fccb4a00640 1 --2- 172.21.15.28:0/3994880701 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcc9006c8f0 0x7fcc9006edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcca000a810 tx=0x7fcca0008040 comp rx=0 tx=0).stop 2024-04-26T20:57:51.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.752+0000 7fccb4a00640 1 -- 172.21.15.28:0/3994880701 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccb00d1050 msgr2=0x7fccb013ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:51.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.752+0000 7fccb4a00640 1 --2- 172.21.15.28:0/3994880701 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccb00d1050 0x7fccb013ee40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcca800b970 tx=0x7fcca800be30 comp rx=0 tx=0).stop 2024-04-26T20:57:51.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.753+0000 7fccb4a00640 1 -- 172.21.15.28:0/3994880701 shutdown_connections 2024-04-26T20:57:51.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.753+0000 7fccb4a00640 1 --2- 172.21.15.28:0/3994880701 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcc9006c8f0 0x7fcc9006edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.753+0000 7fccb4a00640 1 --2- 172.21.15.28:0/3994880701 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fccb00d1050 0x7fccb013ee40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.753+0000 7fccb4a00640 1 --2- 172.21.15.28:0/3994880701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fccb00d0c70 0x7fccb015c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.753+0000 7fccb4a00640 1 --2- 172.21.15.28:0/3994880701 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fccb006e460 0x7fccb015c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.753+0000 7fccb4a00640 1 -- 172.21.15.28:0/3994880701 >> 172.21.15.28:0/3994880701 conn(0x7fccb006c530 msgr2=0x7fccb00b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:51.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.753+0000 7fccb4a00640 1 -- 172.21.15.28:0/3994880701 shutdown_connections 2024-04-26T20:57:51.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.753+0000 7fccb4a00640 1 -- 172.21.15.28:0/3994880701 wait complete. 2024-04-26T20:57:51.773 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:57:51.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.983+0000 7f86df200640 1 Processor -- start 2024-04-26T20:57:51.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.983+0000 7f86df200640 1 -- start start 2024-04-26T20:57:51.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.984+0000 7f86df200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f86e0146670 0x7f86e0071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:51.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.984+0000 7f86df200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86e01610c0 0x7f86e00cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:51.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.984+0000 7f86df200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86e0161510 con 0x7f86e0146060 2024-04-26T20:57:51.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.984+0000 7f86df200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f86e006f510 con 0x7f86e0146670 2024-04-26T20:57:51.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.984+0000 7f86df200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f86e0146440 con 0x7f86e01610c0 2024-04-26T20:57:51.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.985+0000 7f86de800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86e01610c0 0x7f86e00cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:51.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.985+0000 7f86de800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86e01610c0 0x7f86e00cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.28:51306/0 (socket says 172.21.15.28:51306) 2024-04-26T20:57:51.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.985+0000 7f86de800640 1 -- 172.21.15.28:0/1783415859 learned_addr learned my addr 172.21.15.28:0/1783415859 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:51.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.985+0000 7f86dd400640 1 --2- 172.21.15.28:0/1783415859 >> v2:172.21.15.103:3300/0 conn(0x7f86e0146670 0x7f86e0071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:51.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.986+0000 7f86de800640 1 -- 172.21.15.28:0/1783415859 >> v2:172.21.15.103:3300/0 conn(0x7f86e0146670 msgr2=0x7f86e0071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:51.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.986+0000 7f86de800640 1 --2- 172.21.15.28:0/1783415859 >> v2:172.21.15.103:3300/0 conn(0x7f86e0146670 0x7f86e0071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.986+0000 7f86de800640 1 -- 172.21.15.28:0/1783415859 >> v1:172.21.15.103:6789/0 conn(0x7f86e0146060 legacy=0x7f86e01609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:51.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.986+0000 7f86de800640 1 -- 172.21.15.28:0/1783415859 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e006f010 con 0x7f86e01610c0 2024-04-26T20:57:51.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.988+0000 7f86de800640 1 --2- 172.21.15.28:0/1783415859 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86e01610c0 0x7f86e00cb0a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009870 tx=0x7f86d802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f95d0c47b23236bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:51.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.988+0000 7f86dca00640 1 -- 172.21.15.28:0/1783415859 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8037070 con 0x7f86e01610c0 2024-04-26T20:57:51.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.988+0000 7f86dca00640 1 -- 172.21.15.28:0/1783415859 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d8032440 con 0x7f86e01610c0 2024-04-26T20:57:51.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.988+0000 7f86dca00640 1 -- 172.21.15.28:0/1783415859 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8032810 con 0x7f86e01610c0 2024-04-26T20:57:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.989+0000 7f86df200640 1 -- 172.21.15.28:0/1783415859 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86e01610c0 msgr2=0x7f86e00cb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.989+0000 7f86df200640 1 --2- 172.21.15.28:0/1783415859 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86e01610c0 0x7f86e00cb0a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009870 tx=0x7f86d802a1c0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.989+0000 7f86df200640 1 -- 172.21.15.28:0/1783415859 shutdown_connections 2024-04-26T20:57:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.989+0000 7f86df200640 1 --2- 172.21.15.28:0/1783415859 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86e01610c0 0x7f86e00cb0a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.989+0000 7f86df200640 1 --2- 172.21.15.28:0/1783415859 >> v2:172.21.15.103:3300/0 conn(0x7f86e0146670 0x7f86e0071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.989+0000 7f86df200640 1 -- 172.21.15.28:0/1783415859 >> 172.21.15.28:0/1783415859 conn(0x7f86e00d0a50 msgr2=0x7f86e00d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:51.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.989+0000 7f86df200640 1 -- 172.21.15.28:0/1783415859 shutdown_connections 2024-04-26T20:57:51.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.989+0000 7f86df200640 1 -- 172.21.15.28:0/1783415859 wait complete. 2024-04-26T20:57:51.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.991+0000 7f86df200640 1 Processor -- start 2024-04-26T20:57:51.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.991+0000 7f86df200640 1 -- start start 2024-04-26T20:57:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.993+0000 7f86df200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86e0146060 0x7f86e00ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.993+0000 7f86df200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86e0146670 0x7f86e0134d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.993+0000 7f86df200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e01610c0 0x7f86e0138750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.993+0000 7f86df200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f86e006ff60 con 0x7f86e01610c0 2024-04-26T20:57:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.993+0000 7f86df200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f86e006fc80 con 0x7f86e0146060 2024-04-26T20:57:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.993+0000 7f86df200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f86e00cb5e0 con 0x7f86e0146670 2024-04-26T20:57:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.993+0000 7f86de800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e01610c0 0x7f86e0138750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.993+0000 7f86de800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e01610c0 0x7f86e0138750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38050/0 (socket says 172.21.15.28:38050) 2024-04-26T20:57:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.993+0000 7f86de800640 1 -- 172.21.15.28:0/2606604949 learned_addr learned my addr 172.21.15.28:0/2606604949 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.994+0000 7f86dd400640 1 --2- 172.21.15.28:0/2606604949 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86e0146670 0x7f86e0134d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.994+0000 7f86dde00640 1 --2- 172.21.15.28:0/2606604949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86e0146060 0x7f86e00ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.994+0000 7f86de800640 1 -- 172.21.15.28:0/2606604949 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86e0146670 msgr2=0x7f86e0134d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.994+0000 7f86de800640 1 --2- 172.21.15.28:0/2606604949 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86e0146670 0x7f86e0134d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.994+0000 7f86de800640 1 -- 172.21.15.28:0/2606604949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86e0146060 msgr2=0x7f86e00ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:51.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.994+0000 7f86de800640 1 --2- 172.21.15.28:0/2606604949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86e0146060 0x7f86e00ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:51.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.994+0000 7f86de800640 1 -- 172.21.15.28:0/2606604949 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86e0135810 con 0x7f86e01610c0 2024-04-26T20:57:51.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.994+0000 7f86de800640 1 --2- 172.21.15.28:0/2606604949 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e01610c0 0x7f86e0138750 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009870 tx=0x7f86d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:51.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.994+0000 7f86d6000640 1 -- 172.21.15.28:0/2606604949 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8041020 con 0x7f86e01610c0 2024-04-26T20:57:51.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.994+0000 7f86d6000640 1 -- 172.21.15.28:0/2606604949 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d803d640 con 0x7f86e01610c0 2024-04-26T20:57:51.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.994+0000 7f86d6000640 1 -- 172.21.15.28:0/2606604949 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d8037040 con 0x7f86e01610c0 2024-04-26T20:57:51.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.994+0000 7f86df200640 1 -- 172.21.15.28:0/2606604949 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86e0135af0 con 0x7f86e01610c0 2024-04-26T20:57:51.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.995+0000 7f86df200640 1 -- 172.21.15.28:0/2606604949 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86e01438a0 con 0x7f86e01610c0 2024-04-26T20:57:51.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.996+0000 7f86df200640 1 -- 172.21.15.28:0/2606604949 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86e0135cc0 con 0x7f86e01610c0 2024-04-26T20:57:51.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.997+0000 7f86d6000640 1 -- 172.21.15.28:0/2606604949 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86d8044020 con 0x7f86e01610c0 2024-04-26T20:57:51.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.998+0000 7f86d6000640 1 --2- 172.21.15.28:0/2606604949 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f86c006c8f0 0x7f86c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:51.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.998+0000 7f86d6000640 1 -- 172.21.15.28:0/2606604949 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f86d80ac420 con 0x7f86e01610c0 2024-04-26T20:57:52.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.999+0000 7f86dde00640 1 --2- 172.21.15.28:0/2606604949 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f86c006c8f0 0x7f86c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:52.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:51.999+0000 7f86dde00640 1 --2- 172.21.15.28:0/2606604949 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f86c006c8f0 0x7f86c006edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f86e00ce7b0 tx=0x7f86d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:52.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.003+0000 7f86d6000640 1 -- 172.21.15.28:0/2606604949 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f86e0135cc0 con 0x7f86e01610c0 2024-04-26T20:57:52.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.212+0000 7f86df200640 1 -- 172.21.15.28:0/2606604949 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f86e00cec40 con 0x7f86e01610c0 2024-04-26T20:57:52.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.213+0000 7f86d6000640 1 -- 172.21.15.28:0/2606604949 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f86d804b030 con 0x7f86e01610c0 2024-04-26T20:57:52.214 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-04-26T20:57:52.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.215+0000 7f86df200640 1 -- 172.21.15.28:0/2606604949 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f86c006c8f0 msgr2=0x7f86c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:52.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.215+0000 7f86df200640 1 --2- 172.21.15.28:0/2606604949 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f86c006c8f0 0x7f86c006edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f86e00ce7b0 tx=0x7f86d0009040 comp rx=0 tx=0).stop 2024-04-26T20:57:52.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.215+0000 7f86df200640 1 -- 172.21.15.28:0/2606604949 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e01610c0 msgr2=0x7f86e0138750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:52.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.215+0000 7f86df200640 1 --2- 172.21.15.28:0/2606604949 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e01610c0 0x7f86e0138750 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f86d8009870 tx=0x7f86d80046c0 comp rx=0 tx=0).stop 2024-04-26T20:57:52.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.215+0000 7f86df200640 1 -- 172.21.15.28:0/2606604949 shutdown_connections 2024-04-26T20:57:52.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.215+0000 7f86df200640 1 --2- 172.21.15.28:0/2606604949 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f86c006c8f0 0x7f86c006edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:52.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.215+0000 7f86df200640 1 --2- 172.21.15.28:0/2606604949 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e01610c0 0x7f86e0138750 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:52.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.215+0000 7f86df200640 1 --2- 172.21.15.28:0/2606604949 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86e0146670 0x7f86e0134d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:52.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.215+0000 7f86df200640 1 --2- 172.21.15.28:0/2606604949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86e0146060 0x7f86e00ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:52.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.215+0000 7f86df200640 1 -- 172.21.15.28:0/2606604949 >> 172.21.15.28:0/2606604949 conn(0x7f86e00d0a50 msgr2=0x7f86e015f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:52.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.215+0000 7f86df200640 1 -- 172.21.15.28:0/2606604949 shutdown_connections 2024-04-26T20:57:52.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.216+0000 7f86df200640 1 -- 172.21.15.28:0/2606604949 wait complete. 2024-04-26T20:57:52.236 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi103.front.sepia.ceph.com... 2024-04-26T20:57:52.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:52.236 DEBUG:teuthology.orchestra.run.smithi103:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-26T20:57:52.320 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:57:52.320 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi103.front.sepia.ceph.com 2024-04-26T20:57:52.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:57:52.321 DEBUG:teuthology.orchestra.run.smithi103:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-26T20:57:52.373 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T20:57:52.373 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:57:52.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.586+0000 7f0050a00640 1 Processor -- start 2024-04-26T20:57:52.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.586+0000 7f0050a00640 1 -- start start 2024-04-26T20:57:52.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.588+0000 7f0050a00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f004c145080 0x7f004c0cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:52.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.588+0000 7f0050a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f004c072760 con 0x7f004c144b90 2024-04-26T20:57:52.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.588+0000 7f0050a00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f004c072a40 con 0x7f004c073800 2024-04-26T20:57:52.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.588+0000 7f0050a00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f004c072d20 con 0x7f004c145080 2024-04-26T20:57:52.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.588+0000 7f004b400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f004c144b90 0x7f004c0ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38848/0 (socket says 172.21.15.28:38848) 2024-04-26T20:57:52.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.588+0000 7f004b400640 1 -- 172.21.15.28:0/1025742649 learned_addr learned my addr 172.21.15.28:0/1025742649 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:52.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.589+0000 7f004aa00640 1 --2- 172.21.15.28:0/1025742649 >> v2:172.21.15.103:3300/0 conn(0x7f004c145080 0x7f004c0cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:52.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.589+0000 7f004a000640 1 -- 172.21.15.28:0/1025742649 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3336330903 0 0) 0x7f004c072760 con 0x7f004c144b90 2024-04-26T20:57:52.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.589+0000 7f004a000640 1 -- 172.21.15.28:0/1025742649 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0030003520 con 0x7f004c144b90 2024-04-26T20:57:52.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.589+0000 7f004a000640 1 -- 172.21.15.28:0/1025742649 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1979877037 0 0) 0x7f0030003520 con 0x7f004c144b90 2024-04-26T20:57:52.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.589+0000 7f004a000640 1 -- 172.21.15.28:0/1025742649 >> v2:172.21.15.103:3300/0 conn(0x7f004c145080 msgr2=0x7f004c0cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:52.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.589+0000 7f004a000640 1 --2- 172.21.15.28:0/1025742649 >> v2:172.21.15.103:3300/0 conn(0x7f004c145080 0x7f004c0cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:52.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.589+0000 7f004a000640 1 -- 172.21.15.28:0/1025742649 >> v1:172.21.15.103:6789/0 conn(0x7f004c073800 legacy=0x7f004c06f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:52.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.589+0000 7f004a000640 1 -- 172.21.15.28:0/1025742649 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004c073000 con 0x7f004c144b90 2024-04-26T20:57:52.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.589+0000 7f004a000640 1 -- 172.21.15.28:0/1025742649 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f003c001670 con 0x7f004c144b90 2024-04-26T20:57:52.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.589+0000 7f004a000640 1 -- 172.21.15.28:0/1025742649 >> v1:172.21.15.28:6789/0 conn(0x7f004c144b90 legacy=0x7f004c0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:52.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.589+0000 7f004a000640 1 --2- 172.21.15.28:0/1025742649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00300039b0 0x7f0030003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:52.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.589+0000 7f004a000640 1 --2- 172.21.15.28:0/1025742649 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0030004800 0x7f0030004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:52.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.589+0000 7f004a000640 1 --2- 172.21.15.28:0/1025742649 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f00300052e0 0x7f003000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:52.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.589+0000 7f004a000640 1 -- 172.21.15.28:0/1025742649 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f004c072d20 con 0x7f0030004800 2024-04-26T20:57:52.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.589+0000 7f004a000640 1 -- 172.21.15.28:0/1025742649 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f003000c1b0 con 0x7f00300039b0 2024-04-26T20:57:52.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.589+0000 7f004a000640 1 -- 172.21.15.28:0/1025742649 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f003000c3b0 con 0x7f00300052e0 2024-04-26T20:57:52.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.590+0000 7f004aa00640 1 --2- 172.21.15.28:0/1025742649 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0030004800 0x7f0030004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:52.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.590+0000 7f004aa00640 1 -- 172.21.15.28:0/1025742649 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f00300052e0 msgr2=0x7f003000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:52.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.590+0000 7f004aa00640 1 --2- 172.21.15.28:0/1025742649 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f00300052e0 0x7f003000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:52.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.590+0000 7f004aa00640 1 -- 172.21.15.28:0/1025742649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00300039b0 msgr2=0x7f0030003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:52.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.590+0000 7f004aa00640 1 --2- 172.21.15.28:0/1025742649 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f00300039b0 0x7f0030003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:52.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.590+0000 7f004aa00640 1 -- 172.21.15.28:0/1025742649 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0030003520 con 0x7f0030004800 2024-04-26T20:57:52.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.590+0000 7f004be00640 1 -- 172.21.15.28:0/1025742649 reap_dead start 2024-04-26T20:57:52.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.591+0000 7f004aa00640 1 --2- 172.21.15.28:0/1025742649 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0030004800 0x7f0030004c80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f004c145c20 tx=0x7f0030005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21b87f5728e381 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:52.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.591+0000 7f004a000640 1 -- 172.21.15.28:0/1025742649 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034030070 con 0x7f0030004800 2024-04-26T20:57:52.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.591+0000 7f004a000640 1 -- 172.21.15.28:0/1025742649 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f003402ada0 con 0x7f0030004800 2024-04-26T20:57:52.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.591+0000 7f004a000640 1 -- 172.21.15.28:0/1025742649 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003400b610 con 0x7f0030004800 2024-04-26T20:57:52.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.592+0000 7f0050a00640 1 -- 172.21.15.28:0/1025742649 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0030004800 msgr2=0x7f0030004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:52.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.592+0000 7f0050a00640 1 --2- 172.21.15.28:0/1025742649 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0030004800 0x7f0030004c80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f004c145c20 tx=0x7f0030005830 comp rx=0 tx=0).stop 2024-04-26T20:57:52.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.592+0000 7f0050a00640 1 -- 172.21.15.28:0/1025742649 shutdown_connections 2024-04-26T20:57:52.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.592+0000 7f0050a00640 1 --2- 172.21.15.28:0/1025742649 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0030004800 0x7f0030004c80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:52.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.592+0000 7f0050a00640 1 -- 172.21.15.28:0/1025742649 >> 172.21.15.28:0/1025742649 conn(0x7f004c0d0a50 msgr2=0x7f004c0d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:52.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.592+0000 7f0050a00640 1 -- 172.21.15.28:0/1025742649 shutdown_connections 2024-04-26T20:57:52.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.592+0000 7f0050a00640 1 -- 172.21.15.28:0/1025742649 wait complete. 2024-04-26T20:57:52.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.594+0000 7f0050a00640 1 Processor -- start 2024-04-26T20:57:52.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.594+0000 7f0050a00640 1 -- start start 2024-04-26T20:57:52.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.595+0000 7f0050a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0030004800 0x7f004c15c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:52.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.595+0000 7f0050a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f004c140d40 0x7f004c140520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:52.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.595+0000 7f0050a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f004c15b5d0 0x7f004c0b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:52.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.595+0000 7f0050a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f004c0cb220 con 0x7f004c140d40 2024-04-26T20:57:52.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.595+0000 7f0050a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f004c0cb0a0 con 0x7f004c15b5d0 2024-04-26T20:57:52.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.595+0000 7f0050a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f004c0cb3a0 con 0x7f0030004800 2024-04-26T20:57:52.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.595+0000 7f004aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f004c140d40 0x7f004c140520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:52.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.595+0000 7f004aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f004c140d40 0x7f004c140520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38058/0 (socket says 172.21.15.28:38058) 2024-04-26T20:57:52.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.595+0000 7f004aa00640 1 -- 172.21.15.28:0/2310473347 learned_addr learned my addr 172.21.15.28:0/2310473347 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:52.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.596+0000 7f004aa00640 1 -- 172.21.15.28:0/2310473347 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0030004800 msgr2=0x7f004c15c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:52.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.596+0000 7f004aa00640 1 --2- 172.21.15.28:0/2310473347 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0030004800 0x7f004c15c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:52.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.596+0000 7f004aa00640 1 -- 172.21.15.28:0/2310473347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f004c15b5d0 msgr2=0x7f004c0b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:52.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.596+0000 7f004aa00640 1 --2- 172.21.15.28:0/2310473347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f004c15b5d0 0x7f004c0b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:52.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.596+0000 7f004aa00640 1 -- 172.21.15.28:0/2310473347 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f004c1418a0 con 0x7f004c140d40 2024-04-26T20:57:52.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.596+0000 7f004aa00640 1 --2- 172.21.15.28:0/2310473347 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f004c140d40 0x7f004c140520 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f003400bb00 tx=0x7f0034004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:52.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.596+0000 7f0043400640 1 -- 172.21.15.28:0/2310473347 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0034043020 con 0x7f004c140d40 2024-04-26T20:57:52.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.597+0000 7f0043400640 1 -- 172.21.15.28:0/2310473347 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f003403f680 con 0x7f004c140d40 2024-04-26T20:57:52.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.597+0000 7f0043400640 1 -- 172.21.15.28:0/2310473347 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f003400b610 con 0x7f004c140d40 2024-04-26T20:57:52.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.597+0000 7f0050a00640 1 -- 172.21.15.28:0/2310473347 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f004c1412e0 con 0x7f004c140d40 2024-04-26T20:57:52.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.597+0000 7f0050a00640 1 -- 172.21.15.28:0/2310473347 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f004c13ebb0 con 0x7f004c140d40 2024-04-26T20:57:52.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.598+0000 7f0050a00640 1 -- 172.21.15.28:0/2310473347 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f004c06a360 con 0x7f004c140d40 2024-04-26T20:57:52.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.599+0000 7f0043400640 1 -- 172.21.15.28:0/2310473347 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0034046020 con 0x7f004c140d40 2024-04-26T20:57:52.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.600+0000 7f0043400640 1 --2- 172.21.15.28:0/2310473347 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f002806c8f0 0x7f002806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:52.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.600+0000 7f0043400640 1 -- 172.21.15.28:0/2310473347 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f00340ac7d0 con 0x7f004c140d40 2024-04-26T20:57:52.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.600+0000 7f004b400640 1 --2- 172.21.15.28:0/2310473347 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f002806c8f0 0x7f002806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:52.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.601+0000 7f004b400640 1 --2- 172.21.15.28:0/2310473347 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f002806c8f0 0x7f002806edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f004c06b240 tx=0x7f003c001560 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:52.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.604+0000 7f0043400640 1 -- 172.21.15.28:0/2310473347 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f003404d030 con 0x7f004c140d40 2024-04-26T20:57:52.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.841+0000 7f0050a00640 1 -- 172.21.15.28:0/2310473347 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f004c140a60 con 0x7f004c140d40 2024-04-26T20:57:52.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.843+0000 7f0043400640 1 -- 172.21.15.28:0/2310473347 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7f003407aa50 con 0x7f004c140d40 2024-04-26T20:57:52.844 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:52.844 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/2818824721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2818824721},{"type":"v1","addr":"172.21.15.103:6834","nonce":2818824721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/259712913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":259712913},{"type":"v1","addr":"172.21.15.28:6836","nonce":259712913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/989575415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":989575415},{"type":"v1","addr":"172.21.15.28:6837","nonce":989575415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:56:51.143688+0000","modified":"2024-04-26T20:56:54.228571+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6835/2361950169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":2361950169},{"type":"v1","addr":"172.21.15.103:6835","nonce":2361950169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-26T20:57:52.844 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 8 2024-04-26T20:57:52.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.845+0000 7f0050a00640 1 -- 172.21.15.28:0/2310473347 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f002806c8f0 msgr2=0x7f002806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:52.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.846+0000 7f0050a00640 1 --2- 172.21.15.28:0/2310473347 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f002806c8f0 0x7f002806edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f004c06b240 tx=0x7f003c001560 comp rx=0 tx=0).stop 2024-04-26T20:57:52.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.846+0000 7f0050a00640 1 -- 172.21.15.28:0/2310473347 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f004c140d40 msgr2=0x7f004c140520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:52.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.846+0000 7f0050a00640 1 --2- 172.21.15.28:0/2310473347 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f004c140d40 0x7f004c140520 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f003400bb00 tx=0x7f0034004730 comp rx=0 tx=0).stop 2024-04-26T20:57:52.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.846+0000 7f0050a00640 1 -- 172.21.15.28:0/2310473347 shutdown_connections 2024-04-26T20:57:52.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.846+0000 7f0050a00640 1 --2- 172.21.15.28:0/2310473347 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f002806c8f0 0x7f002806edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.846+0000 7f0050a00640 1 --2- 172.21.15.28:0/2310473347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f004c15b5d0 0x7f004c0b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.846+0000 7f0050a00640 1 --2- 172.21.15.28:0/2310473347 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f004c140d40 0x7f004c140520 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.846+0000 7f0050a00640 1 --2- 172.21.15.28:0/2310473347 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0030004800 0x7f004c15c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.846+0000 7f0050a00640 1 -- 172.21.15.28:0/2310473347 >> 172.21.15.28:0/2310473347 conn(0x7f004c0d0a50 msgr2=0x7f004c0c9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.846+0000 7f0050a00640 1 -- 172.21.15.28:0/2310473347 shutdown_connections 2024-04-26T20:57:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:52.846+0000 7f0050a00640 1 -- 172.21.15.28:0/2310473347 wait complete. 2024-04-26T20:57:52.864 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:57:53.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.062+0000 7fc40b200640 1 Processor -- start 2024-04-26T20:57:53.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.062+0000 7fc40b200640 1 -- start start 2024-04-26T20:57:53.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.064+0000 7fc40b200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc40c146060 0x7fc40c071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:53.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.064+0000 7fc40b200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fc40c146670 0x7fc40c1609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:53.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.064+0000 7fc40b200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc40c160ef0 0x7fc40c06de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:53.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.064+0000 7fc40b200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fc40c06f510 con 0x7fc40c146060 2024-04-26T20:57:53.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.064+0000 7fc40b200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fc40c146440 con 0x7fc40c146670 2024-04-26T20:57:53.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.064+0000 7fc40b200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc40c06f010 con 0x7fc40c160ef0 2024-04-26T20:57:53.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.064+0000 7fc409e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc40c146060 0x7fc40c071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:53.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.064+0000 7fc409e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc40c146060 0x7fc40c071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38070/0 (socket says 172.21.15.28:38070) 2024-04-26T20:57:53.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.064+0000 7fc409e00640 1 -- 172.21.15.28:0/3590815974 learned_addr learned my addr 172.21.15.28:0/3590815974 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:53.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.065+0000 7fc409400640 1 --2- 172.21.15.28:0/3590815974 >> v2:172.21.15.103:3300/0 conn(0x7fc40c146670 0x7fc40c1609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:53.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.065+0000 7fc409e00640 1 -- 172.21.15.28:0/3590815974 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc40c160ef0 msgr2=0x7fc40c06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:53.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.065+0000 7fc40a800640 1 --2- 172.21.15.28:0/3590815974 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc40c160ef0 0x7fc40c06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:53.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.065+0000 7fc409e00640 1 --2- 172.21.15.28:0/3590815974 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc40c160ef0 0x7fc40c06de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:53.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.065+0000 7fc409e00640 1 -- 172.21.15.28:0/3590815974 >> v2:172.21.15.103:3300/0 conn(0x7fc40c146670 msgr2=0x7fc40c1609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:53.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.065+0000 7fc409e00640 1 --2- 172.21.15.28:0/3590815974 >> v2:172.21.15.103:3300/0 conn(0x7fc40c146670 0x7fc40c1609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:53.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.065+0000 7fc409e00640 1 -- 172.21.15.28:0/3590815974 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40c1617a0 con 0x7fc40c146060 2024-04-26T20:57:53.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.065+0000 7fc409400640 1 --2- 172.21.15.28:0/3590815974 >> v2:172.21.15.103:3300/0 conn(0x7fc40c146670 0x7fc40c1609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:53.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.066+0000 7fc409e00640 1 --2- 172.21.15.28:0/3590815974 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc40c146060 0x7fc40c071e20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc40c06f8c0 tx=0x7fc3fc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37fd46f8cabe02a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:53.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.066+0000 7fc408a00640 1 -- 172.21.15.28:0/3590815974 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc036070 con 0x7fc40c146060 2024-04-26T20:57:53.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.066+0000 7fc408a00640 1 -- 172.21.15.28:0/3590815974 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3fc031600 con 0x7fc40c146060 2024-04-26T20:57:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.066+0000 7fc408a00640 1 -- 172.21.15.28:0/3590815974 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3fc0319d0 con 0x7fc40c146060 2024-04-26T20:57:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.067+0000 7fc40b200640 1 -- 172.21.15.28:0/3590815974 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc40c146060 msgr2=0x7fc40c071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.067+0000 7fc40b200640 1 --2- 172.21.15.28:0/3590815974 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc40c146060 0x7fc40c071e20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc40c06f8c0 tx=0x7fc3fc02a610 comp rx=0 tx=0).stop 2024-04-26T20:57:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.067+0000 7fc40b200640 1 -- 172.21.15.28:0/3590815974 shutdown_connections 2024-04-26T20:57:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.067+0000 7fc40b200640 1 --2- 172.21.15.28:0/3590815974 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc40c160ef0 0x7fc40c06de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.067+0000 7fc40b200640 1 --2- 172.21.15.28:0/3590815974 >> v2:172.21.15.103:3300/0 conn(0x7fc40c146670 0x7fc40c1609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.067+0000 7fc40b200640 1 --2- 172.21.15.28:0/3590815974 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc40c146060 0x7fc40c071e20 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.067+0000 7fc40b200640 1 -- 172.21.15.28:0/3590815974 >> 172.21.15.28:0/3590815974 conn(0x7fc40c0d0a50 msgr2=0x7fc40c0d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.067+0000 7fc40b200640 1 -- 172.21.15.28:0/3590815974 shutdown_connections 2024-04-26T20:57:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.067+0000 7fc40b200640 1 -- 172.21.15.28:0/3590815974 wait complete. 2024-04-26T20:57:53.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.069+0000 7fc40b200640 1 Processor -- start 2024-04-26T20:57:53.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.069+0000 7fc40b200640 1 -- start start 2024-04-26T20:57:53.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.070+0000 7fc40b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc40c146060 0x7fc40c0cf670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:53.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.070+0000 7fc40b200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc40c146670 0x7fc40c0cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:53.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.070+0000 7fc40b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc40c160ef0 0x7fc40c0b5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:53.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.070+0000 7fc40b200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c161340 con 0x7fc40c160ef0 2024-04-26T20:57:53.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.070+0000 7fc40b200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc40c06ff60 con 0x7fc40c146060 2024-04-26T20:57:53.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.070+0000 7fc40b200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc40c1614c0 con 0x7fc40c146670 2024-04-26T20:57:53.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.070+0000 7fc40a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc40c160ef0 0x7fc40c0b5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:53.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.070+0000 7fc40a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc40c160ef0 0x7fc40c0b5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38086/0 (socket says 172.21.15.28:38086) 2024-04-26T20:57:53.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.070+0000 7fc40a800640 1 -- 172.21.15.28:0/1906228689 learned_addr learned my addr 172.21.15.28:0/1906228689 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:53.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.071+0000 7fc409400640 1 --2- 172.21.15.28:0/1906228689 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc40c146670 0x7fc40c0cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:53.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.071+0000 7fc409e00640 1 --2- 172.21.15.28:0/1906228689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc40c146060 0x7fc40c0cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:53.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.071+0000 7fc40a800640 1 -- 172.21.15.28:0/1906228689 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc40c146670 msgr2=0x7fc40c0cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:53.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.071+0000 7fc40a800640 1 --2- 172.21.15.28:0/1906228689 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc40c146670 0x7fc40c0cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:53.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.071+0000 7fc40a800640 1 -- 172.21.15.28:0/1906228689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc40c146060 msgr2=0x7fc40c0cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:53.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.071+0000 7fc40a800640 1 --2- 172.21.15.28:0/1906228689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc40c146060 0x7fc40c0cf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:53.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.071+0000 7fc40a800640 1 -- 172.21.15.28:0/1906228689 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40c142f00 con 0x7fc40c160ef0 2024-04-26T20:57:53.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.071+0000 7fc409400640 1 --2- 172.21.15.28:0/1906228689 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc40c146670 0x7fc40c0cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:53.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.071+0000 7fc409e00640 1 --2- 172.21.15.28:0/1906228689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc40c146060 0x7fc40c0cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:53.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.071+0000 7fc40a800640 1 --2- 172.21.15.28:0/1906228689 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc40c160ef0 0x7fc40c0b5f20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc40400c550 tx=0x7fc40400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:53.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.071+0000 7fc402000640 1 -- 172.21.15.28:0/1906228689 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404015070 con 0x7fc40c160ef0 2024-04-26T20:57:53.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.071+0000 7fc402000640 1 -- 172.21.15.28:0/1906228689 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc404004220 con 0x7fc40c160ef0 2024-04-26T20:57:53.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.071+0000 7fc402000640 1 -- 172.21.15.28:0/1906228689 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404003610 con 0x7fc40c160ef0 2024-04-26T20:57:53.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.071+0000 7fc40b200640 1 -- 172.21.15.28:0/1906228689 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc40c1431a0 con 0x7fc40c160ef0 2024-04-26T20:57:53.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.072+0000 7fc40b200640 1 -- 172.21.15.28:0/1906228689 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc40c143760 con 0x7fc40c160ef0 2024-04-26T20:57:53.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.073+0000 7fc40b200640 1 -- 172.21.15.28:0/1906228689 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc40c0cad50 con 0x7fc40c160ef0 2024-04-26T20:57:53.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.074+0000 7fc402000640 1 -- 172.21.15.28:0/1906228689 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4040037b0 con 0x7fc40c160ef0 2024-04-26T20:57:53.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.075+0000 7fc402000640 1 --2- 172.21.15.28:0/1906228689 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc3ec06c8f0 0x7fc3ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:53.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.075+0000 7fc402000640 1 -- 172.21.15.28:0/1906228689 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7fc40408b950 con 0x7fc40c160ef0 2024-04-26T20:57:53.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.075+0000 7fc409e00640 1 --2- 172.21.15.28:0/1906228689 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc3ec06c8f0 0x7fc3ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:53.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.076+0000 7fc409e00640 1 --2- 172.21.15.28:0/1906228689 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc3ec06c8f0 0x7fc3ec06edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc033000 tx=0x7fc3fc009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:53.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.080+0000 7fc402000640 1 -- 172.21.15.28:0/1906228689 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc40c0cad50 con 0x7fc40c160ef0 2024-04-26T20:57:53.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.291+0000 7fc40b200640 1 -- 172.21.15.28:0/1906228689 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc40c06a360 con 0x7fc40c160ef0 2024-04-26T20:57:53.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.294+0000 7fc402000640 1 -- 172.21.15.28:0/1906228689 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16831 (secure 0 0 0) 0x7fc404059cf0 con 0x7fc40c160ef0 2024-04-26T20:57:53.295 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:53.295 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":23,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:57:37.996297+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:57:53.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.296+0000 7fc40b200640 1 -- 172.21.15.28:0/1906228689 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc3ec06c8f0 msgr2=0x7fc3ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:53.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.296+0000 7fc40b200640 1 --2- 172.21.15.28:0/1906228689 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc3ec06c8f0 0x7fc3ec06edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc3fc033000 tx=0x7fc3fc009000 comp rx=0 tx=0).stop 2024-04-26T20:57:53.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.296+0000 7fc40b200640 1 -- 172.21.15.28:0/1906228689 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc40c160ef0 msgr2=0x7fc40c0b5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:53.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.296+0000 7fc40b200640 1 --2- 172.21.15.28:0/1906228689 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc40c160ef0 0x7fc40c0b5f20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc40400c550 tx=0x7fc40400ca10 comp rx=0 tx=0).stop 2024-04-26T20:57:53.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.297+0000 7fc40b200640 1 -- 172.21.15.28:0/1906228689 shutdown_connections 2024-04-26T20:57:53.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.297+0000 7fc40b200640 1 --2- 172.21.15.28:0/1906228689 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc3ec06c8f0 0x7fc3ec06edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:53.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.297+0000 7fc40b200640 1 --2- 172.21.15.28:0/1906228689 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc40c160ef0 0x7fc40c0b5f20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:53.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.297+0000 7fc40b200640 1 --2- 172.21.15.28:0/1906228689 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc40c146670 0x7fc40c0cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:53.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.297+0000 7fc40b200640 1 --2- 172.21.15.28:0/1906228689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc40c146060 0x7fc40c0cf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:53.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.297+0000 7fc40b200640 1 -- 172.21.15.28:0/1906228689 >> 172.21.15.28:0/1906228689 conn(0x7fc40c0d0a50 msgr2=0x7fc40c0b0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:53.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.297+0000 7fc40b200640 1 -- 172.21.15.28:0/1906228689 shutdown_connections 2024-04-26T20:57:53.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.297+0000 7fc40b200640 1 -- 172.21.15.28:0/1906228689 wait complete. 2024-04-26T20:57:53.318 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:56:48.688498+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:56:50.117541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:57:53.318 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:57:53.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.532+0000 7f45e2e00640 1 Processor -- start 2024-04-26T20:57:53.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.532+0000 7f45e2e00640 1 -- start start 2024-04-26T20:57:53.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.534+0000 7f45e2e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45dc0731f0 0x7f45dc06e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:53.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.534+0000 7f45e2e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f45dc06eb50 0x7f45dc0735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:53.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.534+0000 7f45e2e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45dc06f2d0 con 0x7f45dc073b10 2024-04-26T20:57:53.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.534+0000 7f45e2e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f45dc06f5b0 con 0x7f45dc06eb50 2024-04-26T20:57:53.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.534+0000 7f45e2e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f45dc06f890 con 0x7f45dc0731f0 2024-04-26T20:57:53.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.535+0000 7f45e0e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f45dc073b10 0x7f45dc0b3f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38850/0 (socket says 172.21.15.28:38850) 2024-04-26T20:57:53.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.535+0000 7f45e0e00640 1 -- 172.21.15.28:0/1627841862 learned_addr learned my addr 172.21.15.28:0/1627841862 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:53.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.535+0000 7f45d3e00640 1 --2- 172.21.15.28:0/1627841862 >> v2:172.21.15.103:3300/0 conn(0x7f45dc06eb50 0x7f45dc0735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:53.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.535+0000 7f45dbe00640 1 --2- 172.21.15.28:0/1627841862 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45dc0731f0 0x7f45dc06e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:53.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.535+0000 7f45db400640 1 -- 172.21.15.28:0/1627841862 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1971686960 0 0) 0x7f45dc06f2d0 con 0x7f45dc073b10 2024-04-26T20:57:53.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.535+0000 7f45db400640 1 -- 172.21.15.28:0/1627841862 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45c0003520 con 0x7f45dc073b10 2024-04-26T20:57:53.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.536+0000 7f45db400640 1 -- 172.21.15.28:0/1627841862 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 271348218 0 0) 0x7f45c0003520 con 0x7f45dc073b10 2024-04-26T20:57:53.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.536+0000 7f45dbe00640 1 -- 172.21.15.28:0/1627841862 >> v2:172.21.15.103:3300/0 conn(0x7f45dc06eb50 msgr2=0x7f45dc0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:53.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.536+0000 7f45dbe00640 1 --2- 172.21.15.28:0/1627841862 >> v2:172.21.15.103:3300/0 conn(0x7f45dc06eb50 0x7f45dc0735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:53.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.536+0000 7f45dbe00640 1 -- 172.21.15.28:0/1627841862 >> v1:172.21.15.28:6789/0 conn(0x7f45dc073b10 legacy=0x7f45dc0b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:53.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.536+0000 7f45dbe00640 1 -- 172.21.15.28:0/1627841862 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45dc06fb70 con 0x7f45dc0731f0 2024-04-26T20:57:53.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.537+0000 7f45dbe00640 1 --2- 172.21.15.28:0/1627841862 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45dc0731f0 0x7f45dc06e610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0097a0 tx=0x7f45cc02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20e5b7700f45fb6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:53.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.537+0000 7f45db400640 1 -- 172.21.15.28:0/1627841862 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc037070 con 0x7f45dc0731f0 2024-04-26T20:57:53.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.537+0000 7f45db400640 1 -- 172.21.15.28:0/1627841862 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45cc02aba0 con 0x7f45dc0731f0 2024-04-26T20:57:53.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.537+0000 7f45db400640 1 -- 172.21.15.28:0/1627841862 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d40019f0 con 0x7f45dc0731f0 2024-04-26T20:57:53.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.538+0000 7f45e2e00640 1 -- 172.21.15.28:0/1627841862 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45dc0731f0 msgr2=0x7f45dc06e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:53.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.538+0000 7f45e2e00640 1 --2- 172.21.15.28:0/1627841862 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45dc0731f0 0x7f45dc06e610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0097a0 tx=0x7f45cc02a110 comp rx=0 tx=0).stop 2024-04-26T20:57:53.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.538+0000 7f45e2e00640 1 -- 172.21.15.28:0/1627841862 shutdown_connections 2024-04-26T20:57:53.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.538+0000 7f45e2e00640 1 --2- 172.21.15.28:0/1627841862 >> v2:172.21.15.103:3300/0 conn(0x7f45dc06eb50 0x7f45dc0735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:53.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.538+0000 7f45e2e00640 1 --2- 172.21.15.28:0/1627841862 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45dc0731f0 0x7f45dc06e610 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:53.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.538+0000 7f45e2e00640 1 -- 172.21.15.28:0/1627841862 >> 172.21.15.28:0/1627841862 conn(0x7f45dc06c770 msgr2=0x7f45dc06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:53.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.538+0000 7f45e2e00640 1 -- 172.21.15.28:0/1627841862 shutdown_connections 2024-04-26T20:57:53.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.539+0000 7f45e2e00640 1 -- 172.21.15.28:0/1627841862 wait complete. 2024-04-26T20:57:53.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.541+0000 7f45e2e00640 1 Processor -- start 2024-04-26T20:57:53.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.541+0000 7f45e2e00640 1 -- start start 2024-04-26T20:57:53.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.542+0000 7f45e2e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45dc06eb50 0x7f45dc0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:53.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.542+0000 7f45e2e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45dc0731f0 0x7f45dc0c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:53.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.542+0000 7f45e2e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45dc073b10 0x7f45dc15eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:53.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.543+0000 7f45e2e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc073f70 con 0x7f45dc0731f0 2024-04-26T20:57:53.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.543+0000 7f45e2e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45dc06efb0 con 0x7f45dc073b10 2024-04-26T20:57:53.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.543+0000 7f45e2e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f45dc0b4690 con 0x7f45dc06eb50 2024-04-26T20:57:53.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.543+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45dc0731f0 0x7f45dc0c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:53.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.543+0000 7f45d3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45dc0731f0 0x7f45dc0c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38088/0 (socket says 172.21.15.28:38088) 2024-04-26T20:57:53.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.543+0000 7f45d3e00640 1 -- 172.21.15.28:0/2759501424 learned_addr learned my addr 172.21.15.28:0/2759501424 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:53.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.543+0000 7f45dbe00640 1 --2- 172.21.15.28:0/2759501424 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45dc06eb50 0x7f45dc0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:53.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.543+0000 7f45e0e00640 1 --2- 172.21.15.28:0/2759501424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45dc073b10 0x7f45dc15eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:53.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.544+0000 7f45d3e00640 1 -- 172.21.15.28:0/2759501424 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45dc06eb50 msgr2=0x7f45dc0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:53.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.544+0000 7f45d3e00640 1 --2- 172.21.15.28:0/2759501424 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45dc06eb50 0x7f45dc0cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:53.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.544+0000 7f45d3e00640 1 -- 172.21.15.28:0/2759501424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45dc073b10 msgr2=0x7f45dc15eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:53.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.544+0000 7f45d3e00640 1 --2- 172.21.15.28:0/2759501424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45dc073b10 0x7f45dc15eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:53.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.544+0000 7f45d3e00640 1 -- 172.21.15.28:0/2759501424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45dc0c8e40 con 0x7f45dc0731f0 2024-04-26T20:57:53.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.544+0000 7f45dbe00640 1 --2- 172.21.15.28:0/2759501424 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45dc06eb50 0x7f45dc0cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:57:53.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.544+0000 7f45d3e00640 1 --2- 172.21.15.28:0/2759501424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45dc0731f0 0x7f45dc0c8460 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f45c400e9b0 tx=0x7f45c400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:53.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.544+0000 7f45d8c00640 1 -- 172.21.15.28:0/2759501424 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c4017070 con 0x7f45dc0731f0 2024-04-26T20:57:53.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.544+0000 7f45d8c00640 1 -- 172.21.15.28:0/2759501424 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c400c9a0 con 0x7f45dc0731f0 2024-04-26T20:57:53.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.544+0000 7f45d8c00640 1 -- 172.21.15.28:0/2759501424 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c400cd90 con 0x7f45dc0731f0 2024-04-26T20:57:53.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.544+0000 7f45e2e00640 1 -- 172.21.15.28:0/2759501424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45dc0c9120 con 0x7f45dc0731f0 2024-04-26T20:57:53.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.545+0000 7f45e2e00640 1 -- 172.21.15.28:0/2759501424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45dc0c96e0 con 0x7f45dc0731f0 2024-04-26T20:57:53.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.546+0000 7f45e2e00640 1 -- 172.21.15.28:0/2759501424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45dc06a360 con 0x7f45dc0731f0 2024-04-26T20:57:53.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.547+0000 7f45d8c00640 1 -- 172.21.15.28:0/2759501424 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45c401a070 con 0x7f45dc0731f0 2024-04-26T20:57:53.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.548+0000 7f45d8c00640 1 --2- 172.21.15.28:0/2759501424 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f45b806c920 0x7f45b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:53.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.548+0000 7f45d8c00640 1 -- 172.21.15.28:0/2759501424 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f45c408bc70 con 0x7f45dc0731f0 2024-04-26T20:57:53.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.548+0000 7f45dbe00640 1 --2- 172.21.15.28:0/2759501424 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f45b806c920 0x7f45b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:53.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.549+0000 7f45dbe00640 1 --2- 172.21.15.28:0/2759501424 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f45b806c920 0x7f45b806ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f45cc034000 tx=0x7f45cc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:53.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.553+0000 7f45d8c00640 1 -- 172.21.15.28:0/2759501424 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f45c4059f80 con 0x7f45dc0731f0 2024-04-26T20:57:53.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.760+0000 7f45e2e00640 1 -- 172.21.15.28:0/2759501424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f45dc06d360 con 0x7f45dc0731f0 2024-04-26T20:57:53.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.761+0000 7f45d8c00640 1 -- 172.21.15.28:0/2759501424 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f45c4059880 con 0x7f45dc0731f0 2024-04-26T20:57:53.762 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:57:53.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.764+0000 7f45e2e00640 1 -- 172.21.15.28:0/2759501424 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f45b806c920 msgr2=0x7f45b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:53.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.764+0000 7f45e2e00640 1 --2- 172.21.15.28:0/2759501424 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f45b806c920 0x7f45b806ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f45cc034000 tx=0x7f45cc02e000 comp rx=0 tx=0).stop 2024-04-26T20:57:53.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.764+0000 7f45e2e00640 1 -- 172.21.15.28:0/2759501424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45dc0731f0 msgr2=0x7f45dc0c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:53.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.764+0000 7f45e2e00640 1 --2- 172.21.15.28:0/2759501424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45dc0731f0 0x7f45dc0c8460 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f45c400e9b0 tx=0x7f45c400ee70 comp rx=0 tx=0).stop 2024-04-26T20:57:53.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.764+0000 7f45e2e00640 1 -- 172.21.15.28:0/2759501424 shutdown_connections 2024-04-26T20:57:53.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.764+0000 7f45e2e00640 1 --2- 172.21.15.28:0/2759501424 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f45b806c920 0x7f45b806ede0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:53.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.764+0000 7f45e2e00640 1 --2- 172.21.15.28:0/2759501424 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45dc073b10 0x7f45dc15eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:53.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.764+0000 7f45e2e00640 1 --2- 172.21.15.28:0/2759501424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45dc0731f0 0x7f45dc0c8460 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:53.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.764+0000 7f45e2e00640 1 --2- 172.21.15.28:0/2759501424 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45dc06eb50 0x7f45dc0cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:53.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.764+0000 7f45e2e00640 1 -- 172.21.15.28:0/2759501424 >> 172.21.15.28:0/2759501424 conn(0x7f45dc06c770 msgr2=0x7f45dc147840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:53.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.764+0000 7f45e2e00640 1 -- 172.21.15.28:0/2759501424 shutdown_connections 2024-04-26T20:57:53.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.764+0000 7f45e2e00640 1 -- 172.21.15.28:0/2759501424 wait complete. 2024-04-26T20:57:53.786 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:57:53.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.995+0000 7f7eb9000640 1 Processor -- start 2024-04-26T20:57:53.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.995+0000 7f7eb9000640 1 -- start start 2024-04-26T20:57:53.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.997+0000 7f7eb9000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7eb415d0f0 0x7f7eb4072740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:53.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.997+0000 7f7eb9000640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f7eb40d2e70 0x7f7eb40d32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:53.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.997+0000 7f7eb9000640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eb4071ca0 con 0x7f7eb4071500 2024-04-26T20:57:53.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.997+0000 7f7eb9000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f7eb40d3800 con 0x7f7eb415d0f0 2024-04-26T20:57:53.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.997+0000 7f7eb9000640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f7eb40722b0 con 0x7f7eb40d2e70 2024-04-26T20:57:53.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.997+0000 7f7eb1e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7eb415d0f0 0x7f7eb4072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:53.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.997+0000 7f7eb1e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7eb415d0f0 0x7f7eb4072740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38098/0 (socket says 172.21.15.28:38098) 2024-04-26T20:57:53.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.997+0000 7f7eb1e00640 1 -- 172.21.15.28:0/1285707909 learned_addr learned my addr 172.21.15.28:0/1285707909 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:53.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.998+0000 7f7eb2800640 1 --2- 172.21.15.28:0/1285707909 >> v2:172.21.15.103:3300/0 conn(0x7f7eb40d2e70 0x7f7eb40d32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:53.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.998+0000 7f7eb1e00640 1 -- 172.21.15.28:0/1285707909 >> v2:172.21.15.103:3300/0 conn(0x7f7eb40d2e70 msgr2=0x7f7eb40d32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:53.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.998+0000 7f7eb1e00640 1 --2- 172.21.15.28:0/1285707909 >> v2:172.21.15.103:3300/0 conn(0x7f7eb40d2e70 0x7f7eb40d32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.998+0000 7f7eb1e00640 1 -- 172.21.15.28:0/1285707909 >> v1:172.21.15.103:6789/0 conn(0x7f7eb4071500 legacy=0x7f7eb40d1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:54.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.998+0000 7f7eb1e00640 1 -- 172.21.15.28:0/1285707909 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb40d2840 con 0x7f7eb415d0f0 2024-04-26T20:57:54.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.998+0000 7f7eb2800640 1 --2- 172.21.15.28:0/1285707909 >> v2:172.21.15.103:3300/0 conn(0x7f7eb40d2e70 0x7f7eb40d32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:54.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.999+0000 7f7eb1e00640 1 --2- 172.21.15.28:0/1285707909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb415d0f0 0x7f7eb4072740 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4072220 tx=0x7f7ea402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77982bc9709ff8f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:54.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.999+0000 7f7eb0a00640 1 -- 172.21.15.28:0/1285707909 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4036070 con 0x7f7eb415d0f0 2024-04-26T20:57:54.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.999+0000 7f7eb0a00640 1 -- 172.21.15.28:0/1285707909 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ea4031600 con 0x7f7eb415d0f0 2024-04-26T20:57:54.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:53.999+0000 7f7eb0a00640 1 -- 172.21.15.28:0/1285707909 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea40319d0 con 0x7f7eb415d0f0 2024-04-26T20:57:54.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.000+0000 7f7eb9000640 1 -- 172.21.15.28:0/1285707909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb415d0f0 msgr2=0x7f7eb4072740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:54.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.000+0000 7f7eb9000640 1 --2- 172.21.15.28:0/1285707909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb415d0f0 0x7f7eb4072740 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4072220 tx=0x7f7ea402a660 comp rx=0 tx=0).stop 2024-04-26T20:57:54.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.000+0000 7f7eb9000640 1 -- 172.21.15.28:0/1285707909 shutdown_connections 2024-04-26T20:57:54.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.000+0000 7f7eb9000640 1 --2- 172.21.15.28:0/1285707909 >> v2:172.21.15.103:3300/0 conn(0x7f7eb40d2e70 0x7f7eb40d32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.000+0000 7f7eb9000640 1 --2- 172.21.15.28:0/1285707909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb415d0f0 0x7f7eb4072740 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.000+0000 7f7eb9000640 1 -- 172.21.15.28:0/1285707909 >> 172.21.15.28:0/1285707909 conn(0x7f7eb40b4c30 msgr2=0x7f7eb40b5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:54.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.000+0000 7f7eb9000640 1 -- 172.21.15.28:0/1285707909 shutdown_connections 2024-04-26T20:57:54.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.000+0000 7f7eb9000640 1 -- 172.21.15.28:0/1285707909 wait complete. 2024-04-26T20:57:54.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.002+0000 7f7eb9000640 1 Processor -- start 2024-04-26T20:57:54.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.002+0000 7f7eb9000640 1 -- start start 2024-04-26T20:57:54.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.003+0000 7f7eb9000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eb40d2e70 0x7f7eb4154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:54.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.003+0000 7f7eb9000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb415d0f0 0x7f7eb4150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:54.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.003+0000 7f7eb9000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7eb4150c90 0x7f7eb4151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:54.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.003+0000 7f7eb9000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb4071f40 con 0x7f7eb415d0f0 2024-04-26T20:57:54.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.003+0000 7f7eb9000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb40d2560 con 0x7f7eb40d2e70 2024-04-26T20:57:54.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.003+0000 7f7eb9000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb415d4d0 con 0x7f7eb4150c90 2024-04-26T20:57:54.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.003+0000 7f7eb1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb415d0f0 0x7f7eb4150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:54.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.003+0000 7f7eb1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb415d0f0 0x7f7eb4150750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38102/0 (socket says 172.21.15.28:38102) 2024-04-26T20:57:54.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.003+0000 7f7eb1400640 1 -- 172.21.15.28:0/3436625034 learned_addr learned my addr 172.21.15.28:0/3436625034 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:54.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.004+0000 7f7eb1e00640 1 --2- 172.21.15.28:0/3436625034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eb40d2e70 0x7f7eb4154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:54.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.004+0000 7f7eb2800640 1 --2- 172.21.15.28:0/3436625034 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7eb4150c90 0x7f7eb4151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:54.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.004+0000 7f7eb1400640 1 -- 172.21.15.28:0/3436625034 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7eb4150c90 msgr2=0x7f7eb4151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:54.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.004+0000 7f7eb1400640 1 --2- 172.21.15.28:0/3436625034 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7eb4150c90 0x7f7eb4151140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.004+0000 7f7eb1400640 1 -- 172.21.15.28:0/3436625034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eb40d2e70 msgr2=0x7f7eb4154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:54.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.004+0000 7f7eb1400640 1 --2- 172.21.15.28:0/3436625034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eb40d2e70 0x7f7eb4154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.004+0000 7f7eb1400640 1 -- 172.21.15.28:0/3436625034 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb413e700 con 0x7f7eb415d0f0 2024-04-26T20:57:54.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.004+0000 7f7eb2800640 1 --2- 172.21.15.28:0/3436625034 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7eb4150c90 0x7f7eb4151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:54.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.004+0000 7f7eb1400640 1 --2- 172.21.15.28:0/3436625034 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb415d0f0 0x7f7eb4150750 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00c440 tx=0x7f7e9c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:54.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.004+0000 7f7eaa000640 1 -- 172.21.15.28:0/3436625034 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c017070 con 0x7f7eb415d0f0 2024-04-26T20:57:54.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.004+0000 7f7eaa000640 1 -- 172.21.15.28:0/3436625034 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e9c0035f0 con 0x7f7eb415d0f0 2024-04-26T20:57:54.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.004+0000 7f7eaa000640 1 -- 172.21.15.28:0/3436625034 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c0039e0 con 0x7f7eb415d0f0 2024-04-26T20:57:54.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.005+0000 7f7eb9000640 1 -- 172.21.15.28:0/3436625034 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb413e9c0 con 0x7f7eb415d0f0 2024-04-26T20:57:54.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.005+0000 7f7eb9000640 1 -- 172.21.15.28:0/3436625034 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb414fc70 con 0x7f7eb415d0f0 2024-04-26T20:57:54.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.006+0000 7f7eb9000640 1 -- 172.21.15.28:0/3436625034 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eb406a360 con 0x7f7eb415d0f0 2024-04-26T20:57:54.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.007+0000 7f7eaa000640 1 -- 172.21.15.28:0/3436625034 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e9c003b80 con 0x7f7eb415d0f0 2024-04-26T20:57:54.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.008+0000 7f7eaa000640 1 --2- 172.21.15.28:0/3436625034 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7e9006c920 0x7f7e9006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:54.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.008+0000 7f7eaa000640 1 -- 172.21.15.28:0/3436625034 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f7e9c08adb0 con 0x7f7eb415d0f0 2024-04-26T20:57:54.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.008+0000 7f7eb1e00640 1 --2- 172.21.15.28:0/3436625034 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7e9006c920 0x7f7e9006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:54.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.009+0000 7f7eb1e00640 1 --2- 172.21.15.28:0/3436625034 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7e9006c920 0x7f7e9006ede0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4033000 tx=0x7f7ea4009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:54.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.013+0000 7f7eaa000640 1 -- 172.21.15.28:0/3436625034 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7e9c059160 con 0x7f7eb415d0f0 2024-04-26T20:57:54.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.220+0000 7f7eb9000640 1 -- 172.21.15.28:0/3436625034 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7eb414da80 con 0x7f7eb415d0f0 2024-04-26T20:57:54.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.221+0000 7f7eaa000640 1 -- 172.21.15.28:0/3436625034 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f7e9c0010c0 con 0x7f7eb415d0f0 2024-04-26T20:57:54.222 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-04-26T20:57:54.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.224+0000 7f7eb9000640 1 -- 172.21.15.28:0/3436625034 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7e9006c920 msgr2=0x7f7e9006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:54.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.224+0000 7f7eb9000640 1 --2- 172.21.15.28:0/3436625034 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7e9006c920 0x7f7e9006ede0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7ea4033000 tx=0x7f7ea4009000 comp rx=0 tx=0).stop 2024-04-26T20:57:54.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.224+0000 7f7eb9000640 1 -- 172.21.15.28:0/3436625034 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb415d0f0 msgr2=0x7f7eb4150750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:54.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.224+0000 7f7eb9000640 1 --2- 172.21.15.28:0/3436625034 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb415d0f0 0x7f7eb4150750 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00c440 tx=0x7f7e9c00c900 comp rx=0 tx=0).stop 2024-04-26T20:57:54.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.224+0000 7f7eb9000640 1 -- 172.21.15.28:0/3436625034 shutdown_connections 2024-04-26T20:57:54.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.224+0000 7f7eb9000640 1 --2- 172.21.15.28:0/3436625034 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7e9006c920 0x7f7e9006ede0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.224+0000 7f7eb9000640 1 --2- 172.21.15.28:0/3436625034 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7eb4150c90 0x7f7eb4151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.224+0000 7f7eb9000640 1 --2- 172.21.15.28:0/3436625034 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eb415d0f0 0x7f7eb4150750 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.224+0000 7f7eb9000640 1 --2- 172.21.15.28:0/3436625034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eb40d2e70 0x7f7eb4154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.224+0000 7f7eb9000640 1 -- 172.21.15.28:0/3436625034 >> 172.21.15.28:0/3436625034 conn(0x7f7eb40b4c30 msgr2=0x7f7eb415c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:54.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.225+0000 7f7eb9000640 1 -- 172.21.15.28:0/3436625034 shutdown_connections 2024-04-26T20:57:54.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.225+0000 7f7eb9000640 1 -- 172.21.15.28:0/3436625034 wait complete. 2024-04-26T20:57:54.244 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:57:54.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.450+0000 7f1913e00640 1 Processor -- start 2024-04-26T20:57:54.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.451+0000 7f1913e00640 1 -- start start 2024-04-26T20:57:54.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.452+0000 7f1913e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f19140b3390 0x7f191406efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:54.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.452+0000 7f1913e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1914070690 con 0x7f19140b2f00 2024-04-26T20:57:54.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.452+0000 7f1913e00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1914162d10 con 0x7f191415f1b0 2024-04-26T20:57:54.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.452+0000 7f1913e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f1914163920 con 0x7f19140b3390 2024-04-26T20:57:54.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.453+0000 7f1912000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f19140b2f00 0x7f191406e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38866/0 (socket says 172.21.15.28:38866) 2024-04-26T20:57:54.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.453+0000 7f1912000640 1 -- 172.21.15.28:0/677747033 learned_addr learned my addr 172.21.15.28:0/677747033 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:54.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.453+0000 7f1913400640 1 --2- 172.21.15.28:0/677747033 >> v2:172.21.15.103:3300/0 conn(0x7f19140b3390 0x7f191406efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:54.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.453+0000 7f1911600640 1 -- 172.21.15.28:0/677747033 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1307424689 0 0) 0x7f1914070690 con 0x7f19140b2f00 2024-04-26T20:57:54.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.453+0000 7f1911600640 1 -- 172.21.15.28:0/677747033 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18f8003520 con 0x7f19140b2f00 2024-04-26T20:57:54.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.454+0000 7f1911600640 1 -- 172.21.15.28:0/677747033 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 130445835 0 0) 0x7f1914162d10 con 0x7f191415f1b0 2024-04-26T20:57:54.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.454+0000 7f1911600640 1 -- 172.21.15.28:0/677747033 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1914070690 con 0x7f191415f1b0 2024-04-26T20:57:54.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.454+0000 7f1911600640 1 -- 172.21.15.28:0/677747033 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2109832937 0 0) 0x7f18f8003520 con 0x7f19140b2f00 2024-04-26T20:57:54.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.454+0000 7f1911600640 1 -- 172.21.15.28:0/677747033 >> v2:172.21.15.103:3300/0 conn(0x7f19140b3390 msgr2=0x7f191406efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:54.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.454+0000 7f1911600640 1 --2- 172.21.15.28:0/677747033 >> v2:172.21.15.103:3300/0 conn(0x7f19140b3390 0x7f191406efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.454+0000 7f1911600640 1 -- 172.21.15.28:0/677747033 >> v1:172.21.15.103:6789/0 conn(0x7f191415f1b0 legacy=0x7f191406ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:54.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.454+0000 7f1911600640 1 -- 172.21.15.28:0/677747033 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1914162ff0 con 0x7f19140b2f00 2024-04-26T20:57:54.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.454+0000 7f1911600640 1 -- 172.21.15.28:0/677747033 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f1900001670 con 0x7f19140b2f00 2024-04-26T20:57:54.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.454+0000 7f1911600640 1 -- 172.21.15.28:0/677747033 >> v1:172.21.15.28:6789/0 conn(0x7f19140b2f00 legacy=0x7f191406e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:54.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.454+0000 7f1911600640 1 --2- 172.21.15.28:0/677747033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18f8003b90 0x7f18f8003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:54.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.454+0000 7f1911600640 1 --2- 172.21.15.28:0/677747033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18f8004800 0x7f18f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:54.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.454+0000 7f1911600640 1 --2- 172.21.15.28:0/677747033 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f18f80051c0 0x7f18f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:54.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.454+0000 7f1911600640 1 -- 172.21.15.28:0/677747033 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1914163920 con 0x7f18f8003b90 2024-04-26T20:57:54.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.454+0000 7f1911600640 1 -- 172.21.15.28:0/677747033 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f18f800c090 con 0x7f18f8004800 2024-04-26T20:57:54.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.454+0000 7f1911600640 1 -- 172.21.15.28:0/677747033 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f18f800c290 con 0x7f18f80051c0 2024-04-26T20:57:54.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.454+0000 7f1912a00640 1 --2- 172.21.15.28:0/677747033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18f8003b90 0x7f18f8003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:54.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.455+0000 7f1912a00640 1 -- 172.21.15.28:0/677747033 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f18f80051c0 msgr2=0x7f18f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:54.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.455+0000 7f1912a00640 1 --2- 172.21.15.28:0/677747033 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f18f80051c0 0x7f18f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.455+0000 7f1912a00640 1 -- 172.21.15.28:0/677747033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18f8004800 msgr2=0x7f18f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:54.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.455+0000 7f1912a00640 1 --2- 172.21.15.28:0/677747033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18f8004800 0x7f18f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.455+0000 7f1912a00640 1 -- 172.21.15.28:0/677747033 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f8003520 con 0x7f18f8003b90 2024-04-26T20:57:54.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.455+0000 7f1913400640 1 -- 172.21.15.28:0/677747033 reap_dead start 2024-04-26T20:57:54.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.456+0000 7f1912a00640 1 --2- 172.21.15.28:0/677747033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18f8003b90 0x7f18f8003f90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f19040018d0 tx=0x7f18f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e060bc9c5c826d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:54.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.456+0000 7f1911600640 1 -- 172.21.15.28:0/677747033 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19040014f0 con 0x7f18f8003b90 2024-04-26T20:57:54.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.456+0000 7f1911600640 1 -- 172.21.15.28:0/677747033 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19040052d0 con 0x7f18f8003b90 2024-04-26T20:57:54.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.456+0000 7f1911600640 1 -- 172.21.15.28:0/677747033 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190402ac40 con 0x7f18f8003b90 2024-04-26T20:57:54.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.457+0000 7f1913e00640 1 -- 172.21.15.28:0/677747033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18f8003b90 msgr2=0x7f18f8003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:54.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.457+0000 7f1913e00640 1 --2- 172.21.15.28:0/677747033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18f8003b90 0x7f18f8003f90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f19040018d0 tx=0x7f18f8005830 comp rx=0 tx=0).stop 2024-04-26T20:57:54.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.457+0000 7f1913e00640 1 -- 172.21.15.28:0/677747033 shutdown_connections 2024-04-26T20:57:54.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.457+0000 7f1913e00640 1 --2- 172.21.15.28:0/677747033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f18f8003b90 0x7f18f8003f90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.457+0000 7f1913e00640 1 -- 172.21.15.28:0/677747033 >> 172.21.15.28:0/677747033 conn(0x7f19140b6790 msgr2=0x7f19140b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:54.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.457+0000 7f1913e00640 1 -- 172.21.15.28:0/677747033 shutdown_connections 2024-04-26T20:57:54.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.457+0000 7f1913e00640 1 -- 172.21.15.28:0/677747033 wait complete. 2024-04-26T20:57:54.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.459+0000 7f1913e00640 1 Processor -- start 2024-04-26T20:57:54.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.459+0000 7f1913e00640 1 -- start start 2024-04-26T20:57:54.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.460+0000 7f1913e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18f8003b90 0x7f1914145ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:54.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.460+0000 7f1913e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1914154090 0x7f19141532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:54.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.460+0000 7f1913e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19140b2060 0x7f19140b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:54.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.460+0000 7f1913e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f19140b37e0 con 0x7f1914154090 2024-04-26T20:57:54.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.460+0000 7f1913e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19141635b0 con 0x7f18f8003b90 2024-04-26T20:57:54.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.460+0000 7f1913e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f19140b44c0 con 0x7f19140b2060 2024-04-26T20:57:54.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.460+0000 7f1912000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1914154090 0x7f19141532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:54.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.460+0000 7f1912000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1914154090 0x7f19141532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38132/0 (socket says 172.21.15.28:38132) 2024-04-26T20:57:54.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.460+0000 7f1912000640 1 -- 172.21.15.28:0/1657574129 learned_addr learned my addr 172.21.15.28:0/1657574129 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:54.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.461+0000 7f1912a00640 1 --2- 172.21.15.28:0/1657574129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18f8003b90 0x7f1914145ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:54.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.461+0000 7f1913400640 1 --2- 172.21.15.28:0/1657574129 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19140b2060 0x7f19140b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:54.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.461+0000 7f1912000640 1 -- 172.21.15.28:0/1657574129 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19140b2060 msgr2=0x7f19140b24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:54.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.461+0000 7f1912000640 1 --2- 172.21.15.28:0/1657574129 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19140b2060 0x7f19140b24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.461+0000 7f1912000640 1 -- 172.21.15.28:0/1657574129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18f8003b90 msgr2=0x7f1914145ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:54.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.461+0000 7f1912000640 1 --2- 172.21.15.28:0/1657574129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18f8003b90 0x7f1914145ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.461+0000 7f1912000640 1 -- 172.21.15.28:0/1657574129 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19141463e0 con 0x7f1914154090 2024-04-26T20:57:54.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.461+0000 7f1912a00640 1 --2- 172.21.15.28:0/1657574129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18f8003b90 0x7f1914145ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:54.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.461+0000 7f1912000640 1 --2- 172.21.15.28:0/1657574129 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1914154090 0x7f19141532c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f190000c700 tx=0x7f190000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:54.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.461+0000 7f190aa00640 1 -- 172.21.15.28:0/1657574129 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1900018070 con 0x7f1914154090 2024-04-26T20:57:54.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.461+0000 7f190aa00640 1 -- 172.21.15.28:0/1657574129 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19000072d0 con 0x7f1914154090 2024-04-26T20:57:54.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.461+0000 7f190aa00640 1 -- 172.21.15.28:0/1657574129 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19000047e0 con 0x7f1914154090 2024-04-26T20:57:54.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.461+0000 7f1913e00640 1 -- 172.21.15.28:0/1657574129 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19141560e0 con 0x7f1914154090 2024-04-26T20:57:54.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.462+0000 7f1913e00640 1 -- 172.21.15.28:0/1657574129 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19141566a0 con 0x7f1914154090 2024-04-26T20:57:54.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.463+0000 7f1913e00640 1 -- 172.21.15.28:0/1657574129 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f191406a2c0 con 0x7f1914154090 2024-04-26T20:57:54.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.464+0000 7f190aa00640 1 -- 172.21.15.28:0/1657574129 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f190001b070 con 0x7f1914154090 2024-04-26T20:57:54.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.465+0000 7f190aa00640 1 --2- 172.21.15.28:0/1657574129 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f18f406c8f0 0x7f18f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:54.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.465+0000 7f190aa00640 1 -- 172.21.15.28:0/1657574129 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f190008afe0 con 0x7f1914154090 2024-04-26T20:57:54.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.465+0000 7f1912a00640 1 --2- 172.21.15.28:0/1657574129 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f18f406c8f0 0x7f18f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:54.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.466+0000 7f1912a00640 1 --2- 172.21.15.28:0/1657574129 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f18f406c8f0 0x7f18f406edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1904032000 tx=0x7f1904009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:54.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.469+0000 7f190aa00640 1 -- 172.21.15.28:0/1657574129 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1900059390 con 0x7f1914154090 2024-04-26T20:57:54.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.680+0000 7f1913e00640 1 -- 172.21.15.28:0/1657574129 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1914153800 con 0x7f1914154090 2024-04-26T20:57:54.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.681+0000 7f190aa00640 1 -- 172.21.15.28:0/1657574129 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f1900058c90 con 0x7f1914154090 2024-04-26T20:57:54.682 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-04-26T20:57:54.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.684+0000 7f1913e00640 1 -- 172.21.15.28:0/1657574129 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f18f406c8f0 msgr2=0x7f18f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:54.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.684+0000 7f1913e00640 1 --2- 172.21.15.28:0/1657574129 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f18f406c8f0 0x7f18f406edb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1904032000 tx=0x7f1904009030 comp rx=0 tx=0).stop 2024-04-26T20:57:54.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.684+0000 7f1913e00640 1 -- 172.21.15.28:0/1657574129 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1914154090 msgr2=0x7f19141532c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:54.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.684+0000 7f1913e00640 1 --2- 172.21.15.28:0/1657574129 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1914154090 0x7f19141532c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f190000c700 tx=0x7f190000cbc0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.684+0000 7f1913e00640 1 -- 172.21.15.28:0/1657574129 shutdown_connections 2024-04-26T20:57:54.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.684+0000 7f1913e00640 1 --2- 172.21.15.28:0/1657574129 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f18f406c8f0 0x7f18f406edb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.684+0000 7f1913e00640 1 --2- 172.21.15.28:0/1657574129 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19140b2060 0x7f19140b24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.684+0000 7f1913e00640 1 --2- 172.21.15.28:0/1657574129 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1914154090 0x7f19141532c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.684+0000 7f1913e00640 1 --2- 172.21.15.28:0/1657574129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18f8003b90 0x7f1914145ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.684+0000 7f1913e00640 1 -- 172.21.15.28:0/1657574129 >> 172.21.15.28:0/1657574129 conn(0x7f19140b6790 msgr2=0x7f191406f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:54.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.684+0000 7f1913e00640 1 -- 172.21.15.28:0/1657574129 shutdown_connections 2024-04-26T20:57:54.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.684+0000 7f1913e00640 1 -- 172.21.15.28:0/1657574129 wait complete. 2024-04-26T20:57:54.707 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:57:54.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.908+0000 7f71e2e00640 1 Processor -- start 2024-04-26T20:57:54.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.908+0000 7f71e2e00640 1 -- start start 2024-04-26T20:57:54.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.910+0000 7f71e2e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71dc146060 0x7f71dc071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:54.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.910+0000 7f71e2e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f71dc146670 0x7f71dc1609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:54.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.910+0000 7f71e2e00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71dc146440 con 0x7f71dc160ef0 2024-04-26T20:57:54.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.910+0000 7f71e2e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f71dc06f510 con 0x7f71dc146670 2024-04-26T20:57:54.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.910+0000 7f71e2e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f71dc06f010 con 0x7f71dc146060 2024-04-26T20:57:54.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.911+0000 7f71e1000640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f71dc146670 0x7f71dc1609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:54.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.911+0000 7f71e1a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71dc146060 0x7f71dc071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:54.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.911+0000 7f71e1000640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f71dc146670 0x7f71dc1609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.28:60946/0 (socket says 172.21.15.28:60946) 2024-04-26T20:57:54.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.911+0000 7f71e1a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71dc146060 0x7f71dc071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.28:51420/0 (socket says 172.21.15.28:51420) 2024-04-26T20:57:54.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.911+0000 7f71e2400640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f71dc160ef0 0x7f71dc06de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.28:46278/0 (socket says 172.21.15.28:46278) 2024-04-26T20:57:54.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.911+0000 7f71e1000640 1 -- 172.21.15.28:0/3386494490 learned_addr learned my addr 172.21.15.28:0/3386494490 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:54.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.911+0000 7f71e1a00640 1 -- 172.21.15.28:0/3386494490 >> v2:172.21.15.103:3300/0 conn(0x7f71dc146670 msgr2=0x7f71dc1609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:54.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.912+0000 7f71e1a00640 1 --2- 172.21.15.28:0/3386494490 >> v2:172.21.15.103:3300/0 conn(0x7f71dc146670 0x7f71dc1609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.912+0000 7f71e1a00640 1 -- 172.21.15.28:0/3386494490 >> v1:172.21.15.103:6789/0 conn(0x7f71dc160ef0 legacy=0x7f71dc06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.912+0000 7f71e1a00640 1 -- 172.21.15.28:0/3386494490 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71dc1617a0 con 0x7f71dc146060 2024-04-26T20:57:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.913+0000 7f71e1a00640 1 --2- 172.21.15.28:0/3386494490 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71dc146060 0x7f71dc071e20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f71d00097a0 tx=0x7f71d002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc614a2d7d182c32 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.913+0000 7f71dbe00640 1 -- 172.21.15.28:0/3386494490 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0037070 con 0x7f71dc146060 2024-04-26T20:57:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.913+0000 7f71dbe00640 1 -- 172.21.15.28:0/3386494490 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d002ae50 con 0x7f71dc146060 2024-04-26T20:57:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.913+0000 7f71dbe00640 1 -- 172.21.15.28:0/3386494490 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0032660 con 0x7f71dc146060 2024-04-26T20:57:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.914+0000 7f71e2e00640 1 -- 172.21.15.28:0/3386494490 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71dc146060 msgr2=0x7f71dc071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.914+0000 7f71e2e00640 1 --2- 172.21.15.28:0/3386494490 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71dc146060 0x7f71dc071e20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f71d00097a0 tx=0x7f71d002a100 comp rx=0 tx=0).stop 2024-04-26T20:57:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.914+0000 7f71e2e00640 1 -- 172.21.15.28:0/3386494490 shutdown_connections 2024-04-26T20:57:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.914+0000 7f71e2e00640 1 --2- 172.21.15.28:0/3386494490 >> v2:172.21.15.103:3300/0 conn(0x7f71dc146670 0x7f71dc1609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.914+0000 7f71e2e00640 1 --2- 172.21.15.28:0/3386494490 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71dc146060 0x7f71dc071e20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.914+0000 7f71e2e00640 1 -- 172.21.15.28:0/3386494490 >> 172.21.15.28:0/3386494490 conn(0x7f71dc0d0a50 msgr2=0x7f71dc0d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.914+0000 7f71e2e00640 1 -- 172.21.15.28:0/3386494490 shutdown_connections 2024-04-26T20:57:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.914+0000 7f71e2e00640 1 -- 172.21.15.28:0/3386494490 wait complete. 2024-04-26T20:57:54.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.917+0000 7f71e2e00640 1 Processor -- start 2024-04-26T20:57:54.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.917+0000 7f71e2e00640 1 -- start start 2024-04-26T20:57:54.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.918+0000 7f71e2e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71dc146060 0x7f71dc0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:54.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.918+0000 7f71e2e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71dc146670 0x7f71dc140d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:54.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.918+0000 7f71e2e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71dc160ef0 0x7f71dc135130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:54.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.918+0000 7f71e2e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc0cc0b0 con 0x7f71dc146670 2024-04-26T20:57:54.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.918+0000 7f71e2e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc06ff60 con 0x7f71dc146060 2024-04-26T20:57:54.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.918+0000 7f71e2e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f71dc0cc230 con 0x7f71dc160ef0 2024-04-26T20:57:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.918+0000 7f71e1000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71dc146670 0x7f71dc140d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.918+0000 7f71e1000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71dc146670 0x7f71dc140d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38140/0 (socket says 172.21.15.28:38140) 2024-04-26T20:57:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.918+0000 7f71e1000640 1 -- 172.21.15.28:0/2589292854 learned_addr learned my addr 172.21.15.28:0/2589292854 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.919+0000 7f71e1a00640 1 --2- 172.21.15.28:0/2589292854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71dc146060 0x7f71dc0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.919+0000 7f71e2400640 1 --2- 172.21.15.28:0/2589292854 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71dc160ef0 0x7f71dc135130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.919+0000 7f71e1000640 1 -- 172.21.15.28:0/2589292854 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71dc160ef0 msgr2=0x7f71dc135130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.919+0000 7f71e1000640 1 --2- 172.21.15.28:0/2589292854 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71dc160ef0 0x7f71dc135130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.919+0000 7f71e1000640 1 -- 172.21.15.28:0/2589292854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71dc146060 msgr2=0x7f71dc0b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:54.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.919+0000 7f71e1000640 1 --2- 172.21.15.28:0/2589292854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71dc146060 0x7f71dc0b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.919+0000 7f71e1000640 1 -- 172.21.15.28:0/2589292854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71dc0b6130 con 0x7f71dc146670 2024-04-26T20:57:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.919+0000 7f71e1a00640 1 --2- 172.21.15.28:0/2589292854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71dc146060 0x7f71dc0b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.919+0000 7f71e1000640 1 --2- 172.21.15.28:0/2589292854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71dc146670 0x7f71dc140d50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f71c800e840 tx=0x7f71c800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.920+0000 7f71d9600640 1 -- 172.21.15.28:0/2589292854 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8017070 con 0x7f71dc146670 2024-04-26T20:57:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.920+0000 7f71d9600640 1 -- 172.21.15.28:0/2589292854 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c800c8e0 con 0x7f71dc146670 2024-04-26T20:57:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.920+0000 7f71e2e00640 1 -- 172.21.15.28:0/2589292854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71dc0b5090 con 0x7f71dc146670 2024-04-26T20:57:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.920+0000 7f71d9600640 1 -- 172.21.15.28:0/2589292854 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c800ccd0 con 0x7f71dc146670 2024-04-26T20:57:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.920+0000 7f71e2e00640 1 -- 172.21.15.28:0/2589292854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71dc0b5650 con 0x7f71dc146670 2024-04-26T20:57:54.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.921+0000 7f71e2e00640 1 -- 172.21.15.28:0/2589292854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71dc06a360 con 0x7f71dc146670 2024-04-26T20:57:54.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.922+0000 7f71d9600640 1 -- 172.21.15.28:0/2589292854 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71c801a070 con 0x7f71dc146670 2024-04-26T20:57:54.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.923+0000 7f71d9600640 1 --2- 172.21.15.28:0/2589292854 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f71bc06c920 0x7f71bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:54.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.924+0000 7f71d9600640 1 -- 172.21.15.28:0/2589292854 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f71c808bb80 con 0x7f71dc146670 2024-04-26T20:57:54.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.924+0000 7f71e1a00640 1 --2- 172.21.15.28:0/2589292854 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f71bc06c920 0x7f71bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:54.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.924+0000 7f71e1a00640 1 --2- 172.21.15.28:0/2589292854 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f71bc06c920 0x7f71bc06ede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f71d0034000 tx=0x7f71d002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:54.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:54.928+0000 7f71d9600640 1 -- 172.21.15.28:0/2589292854 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f71c8059e00 con 0x7f71dc146670 2024-04-26T20:57:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.181+0000 7f71e2e00640 1 -- 172.21.15.28:0/2589292854 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71dc073f70 con 0x7f71dc146670 2024-04-26T20:57:55.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.182+0000 7f71d9600640 1 -- 172.21.15.28:0/2589292854 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7f71c8059700 con 0x7f71dc146670 2024-04-26T20:57:55.183 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:55.184 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/2818824721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2818824721},{"type":"v1","addr":"172.21.15.103:6834","nonce":2818824721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/259712913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":259712913},{"type":"v1","addr":"172.21.15.28:6836","nonce":259712913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/989575415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":989575415},{"type":"v1","addr":"172.21.15.28:6837","nonce":989575415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:56:51.143688+0000","modified":"2024-04-26T20:56:54.228571+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6835/2361950169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":2361950169},{"type":"v1","addr":"172.21.15.103:6835","nonce":2361950169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-26T20:57:55.184 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 8 2024-04-26T20:57:55.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.184+0000 7f71e2e00640 1 -- 172.21.15.28:0/2589292854 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f71bc06c920 msgr2=0x7f71bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:55.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.184+0000 7f71e2e00640 1 --2- 172.21.15.28:0/2589292854 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f71bc06c920 0x7f71bc06ede0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f71d0034000 tx=0x7f71d002e000 comp rx=0 tx=0).stop 2024-04-26T20:57:55.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.184+0000 7f71e2e00640 1 -- 172.21.15.28:0/2589292854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71dc146670 msgr2=0x7f71dc140d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:55.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.185+0000 7f71e2e00640 1 --2- 172.21.15.28:0/2589292854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71dc146670 0x7f71dc140d50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f71c800e840 tx=0x7f71c800ed00 comp rx=0 tx=0).stop 2024-04-26T20:57:55.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.185+0000 7f71e2e00640 1 -- 172.21.15.28:0/2589292854 shutdown_connections 2024-04-26T20:57:55.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.185+0000 7f71e2e00640 1 --2- 172.21.15.28:0/2589292854 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f71bc06c920 0x7f71bc06ede0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.185+0000 7f71e2e00640 1 --2- 172.21.15.28:0/2589292854 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f71dc160ef0 0x7f71dc135130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.185+0000 7f71e2e00640 1 --2- 172.21.15.28:0/2589292854 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f71dc146670 0x7f71dc140d50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.185+0000 7f71e2e00640 1 --2- 172.21.15.28:0/2589292854 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f71dc146060 0x7f71dc0b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.185+0000 7f71e2e00640 1 -- 172.21.15.28:0/2589292854 >> 172.21.15.28:0/2589292854 conn(0x7f71dc0d0a50 msgr2=0x7f71dc0b0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:55.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.185+0000 7f71e2e00640 1 -- 172.21.15.28:0/2589292854 shutdown_connections 2024-04-26T20:57:55.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.185+0000 7f71e2e00640 1 -- 172.21.15.28:0/2589292854 wait complete. 2024-04-26T20:57:55.206 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:57:55.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.417+0000 7f3957200640 1 Processor -- start 2024-04-26T20:57:55.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.417+0000 7f3957200640 1 -- start start 2024-04-26T20:57:55.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.418+0000 7f3957200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f395815aab0 0x7f39580cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:55.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.418+0000 7f3957200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3958073d10 0x7f3958160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:55.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.418+0000 7f3957200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3958161430 con 0x7f3958161020 2024-04-26T20:57:55.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.418+0000 7f3957200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f39580cfa40 con 0x7f395815aab0 2024-04-26T20:57:55.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.418+0000 7f3957200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f3958072040 con 0x7f3958073d10 2024-04-26T20:57:55.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.419+0000 7f3955e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f395815aab0 0x7f39580cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:55.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.419+0000 7f3955e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f395815aab0 0x7f39580cd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38152/0 (socket says 172.21.15.28:38152) 2024-04-26T20:57:55.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.419+0000 7f3955e00640 1 -- 172.21.15.28:0/786361123 learned_addr learned my addr 172.21.15.28:0/786361123 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:55.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.419+0000 7f3955e00640 1 -- 172.21.15.28:0/786361123 >> v2:172.21.15.103:3300/0 conn(0x7f3958073d10 msgr2=0x7f3958160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:55.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.419+0000 7f3955e00640 1 --2- 172.21.15.28:0/786361123 >> v2:172.21.15.103:3300/0 conn(0x7f3958073d10 0x7f3958160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.419+0000 7f3955e00640 1 -- 172.21.15.28:0/786361123 >> v1:172.21.15.28:6789/0 conn(0x7f3958161020 legacy=0x7f395815d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:55.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.419+0000 7f3955e00640 1 -- 172.21.15.28:0/786361123 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39580734e0 con 0x7f395815aab0 2024-04-26T20:57:55.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.420+0000 7f3955e00640 1 --2- 172.21.15.28:0/786361123 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f395815aab0 0x7f39580cd450 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3958071ea0 tx=0x7f394802a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9145676d81ff4eb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:55.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.421+0000 7f3955400640 1 -- 172.21.15.28:0/786361123 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948037070 con 0x7f395815aab0 2024-04-26T20:57:55.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.421+0000 7f3955400640 1 -- 172.21.15.28:0/786361123 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3948032440 con 0x7f395815aab0 2024-04-26T20:57:55.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.421+0000 7f3955400640 1 -- 172.21.15.28:0/786361123 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948032810 con 0x7f395815aab0 2024-04-26T20:57:55.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.421+0000 7f3957200640 1 -- 172.21.15.28:0/786361123 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f395815aab0 msgr2=0x7f39580cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:55.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.421+0000 7f3957200640 1 --2- 172.21.15.28:0/786361123 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f395815aab0 0x7f39580cd450 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3958071ea0 tx=0x7f394802a1f0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.421+0000 7f3957200640 1 -- 172.21.15.28:0/786361123 shutdown_connections 2024-04-26T20:57:55.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.421+0000 7f3957200640 1 --2- 172.21.15.28:0/786361123 >> v2:172.21.15.103:3300/0 conn(0x7f3958073d10 0x7f3958160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.421+0000 7f3957200640 1 --2- 172.21.15.28:0/786361123 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f395815aab0 0x7f39580cd450 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.421+0000 7f3957200640 1 -- 172.21.15.28:0/786361123 >> 172.21.15.28:0/786361123 conn(0x7f39580b2720 msgr2=0x7f39580b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:55.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.421+0000 7f3957200640 1 -- 172.21.15.28:0/786361123 shutdown_connections 2024-04-26T20:57:55.423 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.421+0000 7f3957200640 1 -- 172.21.15.28:0/786361123 wait complete. 2024-04-26T20:57:55.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.423+0000 7f3957200640 1 Processor -- start 2024-04-26T20:57:55.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.423+0000 7f3957200640 1 -- start start 2024-04-26T20:57:55.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.424+0000 7f3957200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3958073d10 0x7f3958159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:55.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.424+0000 7f3957200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f395815aab0 0x7f39580b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:55.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.424+0000 7f3957200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3958161020 0x7f39580b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:55.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.424+0000 7f3957200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f39580cff70 con 0x7f3958073d10 2024-04-26T20:57:55.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.424+0000 7f3957200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f395815ae90 con 0x7f3958161020 2024-04-26T20:57:55.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.424+0000 7f3957200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f3958074160 con 0x7f395815aab0 2024-04-26T20:57:55.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.424+0000 7f3955e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3958073d10 0x7f3958159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:55.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.424+0000 7f3955e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3958073d10 0x7f3958159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38164/0 (socket says 172.21.15.28:38164) 2024-04-26T20:57:55.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.424+0000 7f3955e00640 1 -- 172.21.15.28:0/1959444792 learned_addr learned my addr 172.21.15.28:0/1959444792 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:55.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.425+0000 7f3955e00640 1 -- 172.21.15.28:0/1959444792 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f395815aab0 msgr2=0x7f39580b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:55.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.425+0000 7f3955e00640 1 --2- 172.21.15.28:0/1959444792 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f395815aab0 0x7f39580b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.425+0000 7f3955e00640 1 -- 172.21.15.28:0/1959444792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3958161020 msgr2=0x7f39580b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:55.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.425+0000 7f3956800640 1 --2- 172.21.15.28:0/1959444792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3958161020 0x7f39580b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:55.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.425+0000 7f3955e00640 1 --2- 172.21.15.28:0/1959444792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3958161020 0x7f39580b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.425+0000 7f3955e00640 1 -- 172.21.15.28:0/1959444792 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f395815a420 con 0x7f3958073d10 2024-04-26T20:57:55.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.425+0000 7f3956800640 1 --2- 172.21.15.28:0/1959444792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3958161020 0x7f39580b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:55.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.425+0000 7f3955e00640 1 --2- 172.21.15.28:0/1959444792 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3958073d10 0x7f3958159ee0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3948000c00 tx=0x7f39480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:55.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.426+0000 7f394e000640 1 -- 172.21.15.28:0/1959444792 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948041070 con 0x7f3958073d10 2024-04-26T20:57:55.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.426+0000 7f394e000640 1 -- 172.21.15.28:0/1959444792 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f394803c870 con 0x7f3958073d10 2024-04-26T20:57:55.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.426+0000 7f3957200640 1 -- 172.21.15.28:0/1959444792 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39580b56e0 con 0x7f3958073d10 2024-04-26T20:57:55.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.426+0000 7f394e000640 1 -- 172.21.15.28:0/1959444792 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948037040 con 0x7f3958073d10 2024-04-26T20:57:55.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.426+0000 7f3957200640 1 -- 172.21.15.28:0/1959444792 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f395813f650 con 0x7f3958073d10 2024-04-26T20:57:55.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.427+0000 7f3957200640 1 -- 172.21.15.28:0/1959444792 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f395806a360 con 0x7f3958073d10 2024-04-26T20:57:55.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.428+0000 7f394e000640 1 -- 172.21.15.28:0/1959444792 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3948044070 con 0x7f3958073d10 2024-04-26T20:57:55.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.429+0000 7f394e000640 1 --2- 172.21.15.28:0/1959444792 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f393406c8f0 0x7f393406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:55.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.429+0000 7f394e000640 1 -- 172.21.15.28:0/1959444792 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f39480ac8c0 con 0x7f3958073d10 2024-04-26T20:57:55.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.429+0000 7f394fe00640 1 --2- 172.21.15.28:0/1959444792 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f393406c8f0 0x7f393406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:55.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.430+0000 7f394fe00640 1 --2- 172.21.15.28:0/1959444792 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f393406c8f0 0x7f393406edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f395813a520 tx=0x7f3940006bf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:55.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.434+0000 7f394e000640 1 -- 172.21.15.28:0/1959444792 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f39580734e0 con 0x7f3958073d10 2024-04-26T20:57:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.640+0000 7f3957200640 1 -- 172.21.15.28:0/1959444792 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f395806fc80 con 0x7f3958073d10 2024-04-26T20:57:55.643 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.643+0000 7f394e000640 1 -- 172.21.15.28:0/1959444792 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16831 (secure 0 0 0) 0x7f394807ad90 con 0x7f3958073d10 2024-04-26T20:57:55.644 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:55.644 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":23,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:57:37.996297+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:57:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.645+0000 7f3957200640 1 -- 172.21.15.28:0/1959444792 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f393406c8f0 msgr2=0x7f393406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.645+0000 7f3957200640 1 --2- 172.21.15.28:0/1959444792 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f393406c8f0 0x7f393406edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f395813a520 tx=0x7f3940006bf0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.645+0000 7f3957200640 1 -- 172.21.15.28:0/1959444792 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3958073d10 msgr2=0x7f3958159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.645+0000 7f3957200640 1 --2- 172.21.15.28:0/1959444792 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3958073d10 0x7f3958159ee0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3948000c00 tx=0x7f39480046c0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.646+0000 7f3957200640 1 -- 172.21.15.28:0/1959444792 shutdown_connections 2024-04-26T20:57:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.646+0000 7f3957200640 1 --2- 172.21.15.28:0/1959444792 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f393406c8f0 0x7f393406edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.646+0000 7f3957200640 1 --2- 172.21.15.28:0/1959444792 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3958161020 0x7f39580b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.646+0000 7f3957200640 1 --2- 172.21.15.28:0/1959444792 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f395815aab0 0x7f39580b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.646+0000 7f3957200640 1 --2- 172.21.15.28:0/1959444792 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3958073d10 0x7f3958159ee0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.646+0000 7f3957200640 1 -- 172.21.15.28:0/1959444792 >> 172.21.15.28:0/1959444792 conn(0x7f39580b2720 msgr2=0x7f39580cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:55.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.646+0000 7f3957200640 1 -- 172.21.15.28:0/1959444792 shutdown_connections 2024-04-26T20:57:55.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.646+0000 7f3957200640 1 -- 172.21.15.28:0/1959444792 wait complete. 2024-04-26T20:57:55.665 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T20:57:55.665 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:57:55.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.871+0000 7f5b74a00640 1 Processor -- start 2024-04-26T20:57:55.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.871+0000 7f5b74a00640 1 -- start start 2024-04-26T20:57:55.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.873+0000 7f5b74a00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f5b70145800 0x7f5b700ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:55.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.873+0000 7f5b74a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5b70073770 0x7f5b700aee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.873+0000 7f5b74a00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b700726d0 con 0x7f5b70074050 2024-04-26T20:57:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.873+0000 7f5b74a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f5b70145be0 con 0x7f5b70073770 2024-04-26T20:57:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.873+0000 7f5b74a00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f5b700729b0 con 0x7f5b70145800 2024-04-26T20:57:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.873+0000 7f5b6d400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5b70073770 0x7f5b700aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.873+0000 7f5b6d400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5b70073770 0x7f5b700aee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38174/0 (socket says 172.21.15.28:38174) 2024-04-26T20:57:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.873+0000 7f5b6d400640 1 -- 172.21.15.28:0/1404285430 learned_addr learned my addr 172.21.15.28:0/1404285430 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.873+0000 7f5b6de00640 1 --2- 172.21.15.28:0/1404285430 >> v2:172.21.15.103:3300/0 conn(0x7f5b70145800 0x7f5b700ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.874+0000 7f5b6d400640 1 -- 172.21.15.28:0/1404285430 >> v2:172.21.15.103:3300/0 conn(0x7f5b70145800 msgr2=0x7f5b700ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.874+0000 7f5b6d400640 1 --2- 172.21.15.28:0/1404285430 >> v2:172.21.15.103:3300/0 conn(0x7f5b70145800 0x7f5b700ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.874+0000 7f5b6d400640 1 -- 172.21.15.28:0/1404285430 >> v1:172.21.15.103:6789/0 conn(0x7f5b70074050 legacy=0x7f5b700b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.874+0000 7f5b6d400640 1 -- 172.21.15.28:0/1404285430 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b70072c90 con 0x7f5b70073770 2024-04-26T20:57:55.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.875+0000 7f5b6d400640 1 --2- 172.21.15.28:0/1404285430 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b70073770 0x7f5b700aee00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5b700b0930 tx=0x7f5b5802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5919dbfdfd0df36a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:55.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.875+0000 7f5b6ca00640 1 -- 172.21.15.28:0/1404285430 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58037070 con 0x7f5b70073770 2024-04-26T20:57:55.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.875+0000 7f5b6ca00640 1 -- 172.21.15.28:0/1404285430 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b58032440 con 0x7f5b70073770 2024-04-26T20:57:55.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.875+0000 7f5b6ca00640 1 -- 172.21.15.28:0/1404285430 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58032810 con 0x7f5b70073770 2024-04-26T20:57:55.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.876+0000 7f5b74a00640 1 -- 172.21.15.28:0/1404285430 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b70073770 msgr2=0x7f5b700aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:55.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.876+0000 7f5b74a00640 1 --2- 172.21.15.28:0/1404285430 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b70073770 0x7f5b700aee00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5b700b0930 tx=0x7f5b5802a230 comp rx=0 tx=0).stop 2024-04-26T20:57:55.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.876+0000 7f5b74a00640 1 -- 172.21.15.28:0/1404285430 shutdown_connections 2024-04-26T20:57:55.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.876+0000 7f5b74a00640 1 --2- 172.21.15.28:0/1404285430 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b70073770 0x7f5b700aee00 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.876+0000 7f5b74a00640 1 --2- 172.21.15.28:0/1404285430 >> v2:172.21.15.103:3300/0 conn(0x7f5b70145800 0x7f5b700ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.876+0000 7f5b74a00640 1 -- 172.21.15.28:0/1404285430 >> 172.21.15.28:0/1404285430 conn(0x7f5b7006e220 msgr2=0x7f5b7006e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:55.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.876+0000 7f5b74a00640 1 -- 172.21.15.28:0/1404285430 shutdown_connections 2024-04-26T20:57:55.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.876+0000 7f5b74a00640 1 -- 172.21.15.28:0/1404285430 wait complete. 2024-04-26T20:57:55.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.878+0000 7f5b74a00640 1 Processor -- start 2024-04-26T20:57:55.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.878+0000 7f5b74a00640 1 -- start start 2024-04-26T20:57:55.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.879+0000 7f5b74a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5b70073770 0x7f5b70148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:55.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.879+0000 7f5b74a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b70074050 0x7f5b701385e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:55.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.879+0000 7f5b74a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b70145800 0x7f5b70155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:55.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.879+0000 7f5b74a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5b700af340 con 0x7f5b70145800 2024-04-26T20:57:55.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.879+0000 7f5b74a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5b70073250 con 0x7f5b70074050 2024-04-26T20:57:55.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.879+0000 7f5b74a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f5b700af4c0 con 0x7f5b70073770 2024-04-26T20:57:55.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.879+0000 7f5b6e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b70145800 0x7f5b70155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:55.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.879+0000 7f5b6e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b70145800 0x7f5b70155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38180/0 (socket says 172.21.15.28:38180) 2024-04-26T20:57:55.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.879+0000 7f5b6e800640 1 -- 172.21.15.28:0/3124290099 learned_addr learned my addr 172.21.15.28:0/3124290099 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:55.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.880+0000 7f5b6d400640 1 --2- 172.21.15.28:0/3124290099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b70074050 0x7f5b701385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:55.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.880+0000 7f5b6de00640 1 --2- 172.21.15.28:0/3124290099 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5b70073770 0x7f5b70148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.880+0000 7f5b6e800640 1 -- 172.21.15.28:0/3124290099 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5b70073770 msgr2=0x7f5b70148740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.880+0000 7f5b6e800640 1 --2- 172.21.15.28:0/3124290099 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5b70073770 0x7f5b70148740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.880+0000 7f5b6e800640 1 -- 172.21.15.28:0/3124290099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b70074050 msgr2=0x7f5b701385e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.880+0000 7f5b6e800640 1 --2- 172.21.15.28:0/3124290099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b70074050 0x7f5b701385e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.880+0000 7f5b6e800640 1 -- 172.21.15.28:0/3124290099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b7013b140 con 0x7f5b70145800 2024-04-26T20:57:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.880+0000 7f5b6d400640 1 --2- 172.21.15.28:0/3124290099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b70074050 0x7f5b701385e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.880+0000 7f5b6e800640 1 --2- 172.21.15.28:0/3124290099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b70145800 0x7f5b70155740 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800c780 tx=0x7f5b6800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.880+0000 7f5b66000640 1 -- 172.21.15.28:0/3124290099 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68004b60 con 0x7f5b70145800 2024-04-26T20:57:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.880+0000 7f5b66000640 1 -- 172.21.15.28:0/3124290099 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b68004d00 con 0x7f5b70145800 2024-04-26T20:57:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.880+0000 7f5b66000640 1 -- 172.21.15.28:0/3124290099 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6800aa90 con 0x7f5b70145800 2024-04-26T20:57:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.880+0000 7f5b74a00640 1 -- 172.21.15.28:0/3124290099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b7013b420 con 0x7f5b70145800 2024-04-26T20:57:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.881+0000 7f5b74a00640 1 -- 172.21.15.28:0/3124290099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b7013ad60 con 0x7f5b70145800 2024-04-26T20:57:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.882+0000 7f5b74a00640 1 -- 172.21.15.28:0/3124290099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b7006a400 con 0x7f5b70145800 2024-04-26T20:57:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.883+0000 7f5b66000640 1 -- 172.21.15.28:0/3124290099 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b6800ac30 con 0x7f5b70145800 2024-04-26T20:57:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.883+0000 7f5b66000640 1 --2- 172.21.15.28:0/3124290099 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f5b4c06c8f0 0x7f5b4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:55.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.884+0000 7f5b66000640 1 -- 172.21.15.28:0/3124290099 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f5b6808be70 con 0x7f5b70145800 2024-04-26T20:57:55.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.884+0000 7f5b6de00640 1 --2- 172.21.15.28:0/3124290099 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f5b4c06c8f0 0x7f5b4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:55.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.885+0000 7f5b6de00640 1 --2- 172.21.15.28:0/3124290099 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f5b4c06c8f0 0x7f5b4c06edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5b60006fd0 tx=0x7f5b60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:55.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:55.888+0000 7f5b66000640 1 -- 172.21.15.28:0/3124290099 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5b68020070 con 0x7f5b70145800 2024-04-26T20:57:56.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.130+0000 7f5b74a00640 1 -- 172.21.15.28:0/3124290099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b70142bb0 con 0x7f5b70145800 2024-04-26T20:57:56.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.131+0000 7f5b66000640 1 -- 172.21.15.28:0/3124290099 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7f5b6805a2b0 con 0x7f5b70145800 2024-04-26T20:57:56.133 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:56.133 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/2818824721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2818824721},{"type":"v1","addr":"172.21.15.103:6834","nonce":2818824721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/259712913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":259712913},{"type":"v1","addr":"172.21.15.28:6836","nonce":259712913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/989575415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":989575415},{"type":"v1","addr":"172.21.15.28:6837","nonce":989575415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:56:51.143688+0000","modified":"2024-04-26T20:56:54.228571+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6835/2361950169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":2361950169},{"type":"v1","addr":"172.21.15.103:6835","nonce":2361950169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-26T20:57:56.133 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 8 2024-04-26T20:57:56.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.134+0000 7f5b74a00640 1 -- 172.21.15.28:0/3124290099 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f5b4c06c8f0 msgr2=0x7f5b4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:56.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.134+0000 7f5b74a00640 1 --2- 172.21.15.28:0/3124290099 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f5b4c06c8f0 0x7f5b4c06edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5b60006fd0 tx=0x7f5b60008040 comp rx=0 tx=0).stop 2024-04-26T20:57:56.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.134+0000 7f5b74a00640 1 -- 172.21.15.28:0/3124290099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b70145800 msgr2=0x7f5b70155740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:56.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.134+0000 7f5b74a00640 1 --2- 172.21.15.28:0/3124290099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b70145800 0x7f5b70155740 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800c780 tx=0x7f5b6800cc40 comp rx=0 tx=0).stop 2024-04-26T20:57:56.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.134+0000 7f5b74a00640 1 -- 172.21.15.28:0/3124290099 shutdown_connections 2024-04-26T20:57:56.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.134+0000 7f5b74a00640 1 --2- 172.21.15.28:0/3124290099 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f5b4c06c8f0 0x7f5b4c06edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:56.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.134+0000 7f5b74a00640 1 --2- 172.21.15.28:0/3124290099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b70145800 0x7f5b70155740 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:56.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.134+0000 7f5b74a00640 1 --2- 172.21.15.28:0/3124290099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5b70074050 0x7f5b701385e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:56.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.134+0000 7f5b74a00640 1 --2- 172.21.15.28:0/3124290099 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5b70073770 0x7f5b70148740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:56.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.134+0000 7f5b74a00640 1 -- 172.21.15.28:0/3124290099 >> 172.21.15.28:0/3124290099 conn(0x7f5b7006e220 msgr2=0x7f5b700b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:56.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.134+0000 7f5b74a00640 1 -- 172.21.15.28:0/3124290099 shutdown_connections 2024-04-26T20:57:56.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.135+0000 7f5b74a00640 1 -- 172.21.15.28:0/3124290099 wait complete. 2024-04-26T20:57:56.155 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:57:56.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.358+0000 7f3d6ec00640 1 Processor -- start 2024-04-26T20:57:56.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.359+0000 7f3d6ec00640 1 -- start start 2024-04-26T20:57:56.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.360+0000 7f3d6ec00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3d68163c00 0x7f3d680b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:56.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.360+0000 7f3d6ec00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3d680b5ff0 0x7f3d68165330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:56.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.360+0000 7f3d6ec00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d680b4740 con 0x7f3d681612d0 2024-04-26T20:57:56.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.360+0000 7f3d6ec00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f3d680b4500 con 0x7f3d680b5ff0 2024-04-26T20:57:56.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.360+0000 7f3d6ec00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f3d680b4a20 con 0x7f3d68163c00 2024-04-26T20:57:56.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.361+0000 7f3d6cc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3d680b5ff0 0x7f3d68165330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:56.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.361+0000 7f3d6cc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3d680b5ff0 0x7f3d68165330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38182/0 (socket says 172.21.15.28:38182) 2024-04-26T20:57:56.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.361+0000 7f3d6cc00640 1 -- 172.21.15.28:0/306635161 learned_addr learned my addr 172.21.15.28:0/306635161 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:56.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.361+0000 7f3d67e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3d681612d0 0x7f3d680b3df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38872/0 (socket says 172.21.15.28:38872) 2024-04-26T20:57:56.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.361+0000 7f3d67400640 1 --2- 172.21.15.28:0/306635161 >> v2:172.21.15.103:3300/0 conn(0x7f3d68163c00 0x7f3d680b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:56.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.361+0000 7f3d6cc00640 1 -- 172.21.15.28:0/306635161 >> v2:172.21.15.103:3300/0 conn(0x7f3d68163c00 msgr2=0x7f3d680b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:56.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.361+0000 7f3d6cc00640 1 --2- 172.21.15.28:0/306635161 >> v2:172.21.15.103:3300/0 conn(0x7f3d68163c00 0x7f3d680b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:56.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.361+0000 7f3d6cc00640 1 -- 172.21.15.28:0/306635161 >> v1:172.21.15.28:6789/0 conn(0x7f3d681612d0 legacy=0x7f3d680b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:56.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.361+0000 7f3d66a00640 1 -- 172.21.15.28:0/306635161 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3931651903 0 0) 0x7f3d680b4740 con 0x7f3d681612d0 2024-04-26T20:57:56.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.362+0000 7f3d6cc00640 1 -- 172.21.15.28:0/306635161 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d680b4d00 con 0x7f3d680b5ff0 2024-04-26T20:57:56.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.363+0000 7f3d6cc00640 1 --2- 172.21.15.28:0/306635161 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d680b5ff0 0x7f3d68165330 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3d680b3a30 tx=0x7f3d6002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3305b637abe5ca2d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:56.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.363+0000 7f3d66a00640 1 -- 172.21.15.28:0/306635161 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60037070 con 0x7f3d680b5ff0 2024-04-26T20:57:56.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.363+0000 7f3d66a00640 1 -- 172.21.15.28:0/306635161 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d60032440 con 0x7f3d680b5ff0 2024-04-26T20:57:56.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.363+0000 7f3d66a00640 1 -- 172.21.15.28:0/306635161 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60032810 con 0x7f3d680b5ff0 2024-04-26T20:57:56.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.364+0000 7f3d6ec00640 1 -- 172.21.15.28:0/306635161 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d680b5ff0 msgr2=0x7f3d68165330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:56.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.364+0000 7f3d6ec00640 1 --2- 172.21.15.28:0/306635161 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d680b5ff0 0x7f3d68165330 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3d680b3a30 tx=0x7f3d6002a270 comp rx=0 tx=0).stop 2024-04-26T20:57:56.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.364+0000 7f3d6ec00640 1 -- 172.21.15.28:0/306635161 shutdown_connections 2024-04-26T20:57:56.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.364+0000 7f3d6ec00640 1 --2- 172.21.15.28:0/306635161 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d680b5ff0 0x7f3d68165330 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:56.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.364+0000 7f3d6ec00640 1 --2- 172.21.15.28:0/306635161 >> v2:172.21.15.103:3300/0 conn(0x7f3d68163c00 0x7f3d680b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:56.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.364+0000 7f3d6ec00640 1 -- 172.21.15.28:0/306635161 >> 172.21.15.28:0/306635161 conn(0x7f3d6806e220 msgr2=0x7f3d6806e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:56.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.364+0000 7f3d6ec00640 1 -- 172.21.15.28:0/306635161 shutdown_connections 2024-04-26T20:57:56.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.364+0000 7f3d6ec00640 1 -- 172.21.15.28:0/306635161 wait complete. 2024-04-26T20:57:56.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.366+0000 7f3d6ec00640 1 Processor -- start 2024-04-26T20:57:56.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.366+0000 7f3d6ec00640 1 -- start start 2024-04-26T20:57:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.367+0000 7f3d6ec00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d680b5ff0 0x7f3d6814adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.367+0000 7f3d6ec00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d681612d0 0x7f3d681616b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.367+0000 7f3d6ec00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d68163c00 0x7f3d680cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.367+0000 7f3d6ec00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3d681659f0 con 0x7f3d681612d0 2024-04-26T20:57:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.367+0000 7f3d6ec00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3d68165870 con 0x7f3d680b5ff0 2024-04-26T20:57:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.367+0000 7f3d6ec00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f3d68165b70 con 0x7f3d68163c00 2024-04-26T20:57:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.367+0000 7f3d67400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d681612d0 0x7f3d681616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.367+0000 7f3d67400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d681612d0 0x7f3d681616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38190/0 (socket says 172.21.15.28:38190) 2024-04-26T20:57:56.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.367+0000 7f3d67400640 1 -- 172.21.15.28:0/1767986758 learned_addr learned my addr 172.21.15.28:0/1767986758 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:56.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.367+0000 7f3d67400640 1 -- 172.21.15.28:0/1767986758 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d68163c00 msgr2=0x7f3d680cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:56.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.367+0000 7f3d67400640 1 --2- 172.21.15.28:0/1767986758 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d68163c00 0x7f3d680cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:56.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.367+0000 7f3d67400640 1 -- 172.21.15.28:0/1767986758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d680b5ff0 msgr2=0x7f3d6814adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:56.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.367+0000 7f3d67400640 1 --2- 172.21.15.28:0/1767986758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d680b5ff0 0x7f3d6814adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:56.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.367+0000 7f3d67400640 1 -- 172.21.15.28:0/1767986758 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d68162fe0 con 0x7f3d681612d0 2024-04-26T20:57:56.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.368+0000 7f3d67400640 1 --2- 172.21.15.28:0/1767986758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d681612d0 0x7f3d681616b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400d5f0 tx=0x7f3d5400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:56.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.368+0000 7f3d5fe00640 1 -- 172.21.15.28:0/1767986758 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d540125f0 con 0x7f3d681612d0 2024-04-26T20:57:56.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.368+0000 7f3d5fe00640 1 -- 172.21.15.28:0/1767986758 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d54019070 con 0x7f3d681612d0 2024-04-26T20:57:56.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.368+0000 7f3d6ec00640 1 -- 172.21.15.28:0/1767986758 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d68161f40 con 0x7f3d681612d0 2024-04-26T20:57:56.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.368+0000 7f3d5fe00640 1 -- 172.21.15.28:0/1767986758 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d540129d0 con 0x7f3d681612d0 2024-04-26T20:57:56.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.368+0000 7f3d6ec00640 1 -- 172.21.15.28:0/1767986758 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d68162500 con 0x7f3d681612d0 2024-04-26T20:57:56.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.369+0000 7f3d6ec00640 1 -- 172.21.15.28:0/1767986758 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6806a400 con 0x7f3d681612d0 2024-04-26T20:57:56.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.370+0000 7f3d5fe00640 1 -- 172.21.15.28:0/1767986758 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d54012b70 con 0x7f3d681612d0 2024-04-26T20:57:56.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.371+0000 7f3d5fe00640 1 --2- 172.21.15.28:0/1767986758 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3d4806c8f0 0x7f3d4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:56.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.371+0000 7f3d5fe00640 1 -- 172.21.15.28:0/1767986758 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f3d5408bf90 con 0x7f3d681612d0 2024-04-26T20:57:56.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.372+0000 7f3d67e00640 1 --2- 172.21.15.28:0/1767986758 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3d4806c8f0 0x7f3d4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:56.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.372+0000 7f3d67e00640 1 --2- 172.21.15.28:0/1767986758 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3d4806c8f0 0x7f3d4806edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3d58001560 tx=0x7f3d58007ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:56.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.376+0000 7f3d5fe00640 1 -- 172.21.15.28:0/1767986758 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3d5405a340 con 0x7f3d681612d0 2024-04-26T20:57:56.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.622+0000 7f3d6ec00640 1 -- 172.21.15.28:0/1767986758 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d680b0740 con 0x7f3d681612d0 2024-04-26T20:57:56.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.623+0000 7f3d5fe00640 1 -- 172.21.15.28:0/1767986758 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5175 (secure 0 0 0) 0x7f3d54059c40 con 0x7f3d681612d0 2024-04-26T20:57:56.624 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:56.624 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/2818824721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2818824721},{"type":"v1","addr":"172.21.15.103:6834","nonce":2818824721}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/259712913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":259712913},{"type":"v1","addr":"172.21.15.28:6836","nonce":259712913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/989575415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":989575415},{"type":"v1","addr":"172.21.15.28:6837","nonce":989575415}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:56:51.143688+0000","modified":"2024-04-26T20:56:54.228571+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6835/2361950169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":2361950169},{"type":"v1","addr":"172.21.15.103:6835","nonce":2361950169}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-26T20:57:56.624 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 8 2024-04-26T20:57:56.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.625+0000 7f3d6ec00640 1 -- 172.21.15.28:0/1767986758 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3d4806c8f0 msgr2=0x7f3d4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:56.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.625+0000 7f3d6ec00640 1 --2- 172.21.15.28:0/1767986758 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3d4806c8f0 0x7f3d4806edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3d58001560 tx=0x7f3d58007ca0 comp rx=0 tx=0).stop 2024-04-26T20:57:56.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.625+0000 7f3d6ec00640 1 -- 172.21.15.28:0/1767986758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d681612d0 msgr2=0x7f3d681616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:56.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.625+0000 7f3d6ec00640 1 --2- 172.21.15.28:0/1767986758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d681612d0 0x7f3d681616b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3d5400d5f0 tx=0x7f3d5400dab0 comp rx=0 tx=0).stop 2024-04-26T20:57:56.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.625+0000 7f3d6ec00640 1 -- 172.21.15.28:0/1767986758 shutdown_connections 2024-04-26T20:57:56.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.625+0000 7f3d6ec00640 1 --2- 172.21.15.28:0/1767986758 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3d4806c8f0 0x7f3d4806edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:56.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.625+0000 7f3d6ec00640 1 --2- 172.21.15.28:0/1767986758 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d68163c00 0x7f3d680cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:56.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.625+0000 7f3d6ec00640 1 --2- 172.21.15.28:0/1767986758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d681612d0 0x7f3d681616b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:56.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.625+0000 7f3d6ec00640 1 --2- 172.21.15.28:0/1767986758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d680b5ff0 0x7f3d6814adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:56.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.625+0000 7f3d6ec00640 1 -- 172.21.15.28:0/1767986758 >> 172.21.15.28:0/1767986758 conn(0x7f3d6806e220 msgr2=0x7f3d68167030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:56.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.626+0000 7f3d6ec00640 1 -- 172.21.15.28:0/1767986758 shutdown_connections 2024-04-26T20:57:56.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.626+0000 7f3d6ec00640 1 -- 172.21.15.28:0/1767986758 wait complete. 2024-04-26T20:57:56.647 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:57:56.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.858+0000 7f73e3200640 1 Processor -- start 2024-04-26T20:57:56.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.858+0000 7f73e3200640 1 -- start start 2024-04-26T20:57:56.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.860+0000 7f73e3200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f73dc15f1b0 0x7f73dc0d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:56.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.860+0000 7f73e3200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f73dc0cf700 0x7f73dc0d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:56.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.860+0000 7f73e3200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73dc0d0710 con 0x7f73dc0d5370 2024-04-26T20:57:56.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.860+0000 7f73e3200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f73dc0cff00 con 0x7f73dc0cf700 2024-04-26T20:57:56.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.860+0000 7f73e3200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f73dc0d4270 con 0x7f73dc15f1b0 2024-04-26T20:57:56.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.861+0000 7f73e1400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f73dc0cf700 0x7f73dc0d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:56.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.861+0000 7f73e1e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f73dc15f1b0 0x7f73dc0d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:56.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.861+0000 7f73e1400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f73dc0cf700 0x7f73dc0d4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38206/0 (socket says 172.21.15.28:38206) 2024-04-26T20:57:56.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.861+0000 7f73e1400640 1 -- 172.21.15.28:0/3813605611 learned_addr learned my addr 172.21.15.28:0/3813605611 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:56.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.861+0000 7f73e1400640 1 -- 172.21.15.28:0/3813605611 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f73dc15f1b0 msgr2=0x7f73dc0d4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:56.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.861+0000 7f73e1400640 1 --2- 172.21.15.28:0/3813605611 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f73dc15f1b0 0x7f73dc0d4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:56.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.861+0000 7f73e1400640 1 -- 172.21.15.28:0/3813605611 >> v1:172.21.15.103:6789/0 conn(0x7f73dc0d5370 legacy=0x7f73dc0b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:56.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.861+0000 7f73e1400640 1 -- 172.21.15.28:0/3813605611 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73dc0d0150 con 0x7f73dc0cf700 2024-04-26T20:57:56.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.862+0000 7f73e1400640 1 --2- 172.21.15.28:0/3813605611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73dc0cf700 0x7f73dc0d4e30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009870 tx=0x7f73cc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29fedbc30e20fdbe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:56.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.863+0000 7f73e0a00640 1 -- 172.21.15.28:0/3813605611 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc037070 con 0x7f73dc0cf700 2024-04-26T20:57:56.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.863+0000 7f73e0a00640 1 -- 172.21.15.28:0/3813605611 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73cc02ae50 con 0x7f73dc0cf700 2024-04-26T20:57:56.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.863+0000 7f73e0a00640 1 -- 172.21.15.28:0/3813605611 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc032860 con 0x7f73dc0cf700 2024-04-26T20:57:56.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.863+0000 7f73e3200640 1 -- 172.21.15.28:0/3813605611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73dc0cf700 msgr2=0x7f73dc0d4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:56.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.863+0000 7f73e3200640 1 --2- 172.21.15.28:0/3813605611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73dc0cf700 0x7f73dc0d4e30 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009870 tx=0x7f73cc02a300 comp rx=0 tx=0).stop 2024-04-26T20:57:56.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.863+0000 7f73e3200640 1 -- 172.21.15.28:0/3813605611 shutdown_connections 2024-04-26T20:57:56.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.863+0000 7f73e3200640 1 --2- 172.21.15.28:0/3813605611 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73dc0cf700 0x7f73dc0d4e30 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:56.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.863+0000 7f73e3200640 1 --2- 172.21.15.28:0/3813605611 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f73dc15f1b0 0x7f73dc0d4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:56.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.863+0000 7f73e3200640 1 -- 172.21.15.28:0/3813605611 >> 172.21.15.28:0/3813605611 conn(0x7f73dc06c770 msgr2=0x7f73dc06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:56.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.863+0000 7f73e3200640 1 -- 172.21.15.28:0/3813605611 shutdown_connections 2024-04-26T20:57:56.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.863+0000 7f73e3200640 1 -- 172.21.15.28:0/3813605611 wait complete. 2024-04-26T20:57:56.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.865+0000 7f73e3200640 1 Processor -- start 2024-04-26T20:57:56.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.865+0000 7f73e3200640 1 -- start start 2024-04-26T20:57:56.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.866+0000 7f73e3200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73dc0cf700 0x7f73dc14d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:56.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.866+0000 7f73e3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73dc0d5370 0x7f73dc0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:56.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.866+0000 7f73e3200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f73dc15f1b0 0x7f73dc0b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:56.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.866+0000 7f73e3200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc15f590 con 0x7f73dc0d5370 2024-04-26T20:57:56.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.866+0000 7f73e3200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f73dc0d3f00 con 0x7f73dc0cf700 2024-04-26T20:57:56.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.866+0000 7f73e3200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f73dc0b4410 con 0x7f73dc15f1b0 2024-04-26T20:57:56.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.866+0000 7f73e1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73dc0d5370 0x7f73dc0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:56.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.866+0000 7f73e1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73dc0d5370 0x7f73dc0ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38216/0 (socket says 172.21.15.28:38216) 2024-04-26T20:57:56.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.866+0000 7f73e1400640 1 -- 172.21.15.28:0/290349021 learned_addr learned my addr 172.21.15.28:0/290349021 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:56.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.867+0000 7f73e1400640 1 -- 172.21.15.28:0/290349021 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f73dc15f1b0 msgr2=0x7f73dc0b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:56.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.867+0000 7f73e1400640 1 --2- 172.21.15.28:0/290349021 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f73dc15f1b0 0x7f73dc0b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:56.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.867+0000 7f73e1400640 1 -- 172.21.15.28:0/290349021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73dc0cf700 msgr2=0x7f73dc14d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:56.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.867+0000 7f73e1400640 1 --2- 172.21.15.28:0/290349021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73dc0cf700 0x7f73dc14d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:56.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.867+0000 7f73e1400640 1 -- 172.21.15.28:0/290349021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73dc0cd0f0 con 0x7f73dc0d5370 2024-04-26T20:57:56.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.867+0000 7f73e1400640 1 --2- 172.21.15.28:0/290349021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73dc0d5370 0x7f73dc0ce190 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f73cc02a340 tx=0x7f73cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:56.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.867+0000 7f73d6000640 1 -- 172.21.15.28:0/290349021 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc041020 con 0x7f73dc0d5370 2024-04-26T20:57:56.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.868+0000 7f73d6000640 1 -- 172.21.15.28:0/290349021 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73cc03c870 con 0x7f73dc0d5370 2024-04-26T20:57:56.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.868+0000 7f73d6000640 1 -- 172.21.15.28:0/290349021 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc037040 con 0x7f73dc0d5370 2024-04-26T20:57:56.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.868+0000 7f73e3200640 1 -- 172.21.15.28:0/290349021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73dc0cd3d0 con 0x7f73dc0d5370 2024-04-26T20:57:56.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.868+0000 7f73e3200640 1 -- 172.21.15.28:0/290349021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73dc0cd990 con 0x7f73dc0d5370 2024-04-26T20:57:56.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.869+0000 7f73e3200640 1 -- 172.21.15.28:0/290349021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73dc06a360 con 0x7f73dc0d5370 2024-04-26T20:57:56.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.870+0000 7f73d6000640 1 -- 172.21.15.28:0/290349021 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f73cc044020 con 0x7f73dc0d5370 2024-04-26T20:57:56.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.871+0000 7f73d6000640 1 --2- 172.21.15.28:0/290349021 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f73c006c8f0 0x7f73c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:56.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.871+0000 7f73d6000640 1 -- 172.21.15.28:0/290349021 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f73cc0ac440 con 0x7f73dc0d5370 2024-04-26T20:57:56.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.871+0000 7f73e1e00640 1 --2- 172.21.15.28:0/290349021 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f73c006c8f0 0x7f73c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:56.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.872+0000 7f73e1e00640 1 --2- 172.21.15.28:0/290349021 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f73c006c8f0 0x7f73c006edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f73d0006a90 tx=0x7f73d0006a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:56.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:56.876+0000 7f73d6000640 1 -- 172.21.15.28:0/290349021 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f73cc04b030 con 0x7f73dc0d5370 2024-04-26T20:57:57.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.082+0000 7f73e3200640 1 -- 172.21.15.28:0/290349021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73dc0ce6d0 con 0x7f73dc0d5370 2024-04-26T20:57:57.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.085+0000 7f73d6000640 1 -- 172.21.15.28:0/290349021 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16831 (secure 0 0 0) 0x7f73cc07a880 con 0x7f73dc0d5370 2024-04-26T20:57:57.086 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:57:57.086 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":23,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:57:37.996297+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:56:48.688498+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T20:56:50.117541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2a","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:57:57.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.087+0000 7f73e3200640 1 -- 172.21.15.28:0/290349021 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f73c006c8f0 msgr2=0x7f73c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:57.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.087+0000 7f73e3200640 1 --2- 172.21.15.28:0/290349021 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f73c006c8f0 0x7f73c006edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f73d0006a90 tx=0x7f73d0006a20 comp rx=0 tx=0).stop 2024-04-26T20:57:57.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.087+0000 7f73e3200640 1 -- 172.21.15.28:0/290349021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73dc0d5370 msgr2=0x7f73dc0ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:57.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.087+0000 7f73e3200640 1 --2- 172.21.15.28:0/290349021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73dc0d5370 0x7f73dc0ce190 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f73cc02a340 tx=0x7f73cc0046c0 comp rx=0 tx=0).stop 2024-04-26T20:57:57.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.088+0000 7f73e3200640 1 -- 172.21.15.28:0/290349021 shutdown_connections 2024-04-26T20:57:57.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.088+0000 7f73e3200640 1 --2- 172.21.15.28:0/290349021 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f73c006c8f0 0x7f73c006edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:57.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.088+0000 7f73e3200640 1 --2- 172.21.15.28:0/290349021 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f73dc15f1b0 0x7f73dc0b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:57.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.088+0000 7f73e3200640 1 --2- 172.21.15.28:0/290349021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73dc0d5370 0x7f73dc0ce190 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:57.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.088+0000 7f73e3200640 1 --2- 172.21.15.28:0/290349021 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f73dc0cf700 0x7f73dc14d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:57.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.088+0000 7f73e3200640 1 -- 172.21.15.28:0/290349021 >> 172.21.15.28:0/290349021 conn(0x7f73dc06c770 msgr2=0x7f73dc15a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:57.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.088+0000 7f73e3200640 1 -- 172.21.15.28:0/290349021 shutdown_connections 2024-04-26T20:57:57.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.088+0000 7f73e3200640 1 -- 172.21.15.28:0/290349021 wait complete. 2024-04-26T20:57:57.107 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T20:57:57.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.316+0000 7f0584a00640 1 Processor -- start 2024-04-26T20:57:57.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.317+0000 7f0584a00640 1 -- start start 2024-04-26T20:57:57.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.318+0000 7f0584a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05800cb500 0x7f058006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:57.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.318+0000 7f0584a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0580145a10 con 0x7f05800cb0a0 2024-04-26T20:57:57.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.318+0000 7f0584a00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f058015f7e0 con 0x7f05800c9da0 2024-04-26T20:57:57.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.318+0000 7f0584a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f05801603f0 con 0x7f05800cb500 2024-04-26T20:57:57.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.319+0000 7f057ea00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f05800cb0a0 0x7f058006e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38886/0 (socket says 172.21.15.28:38886) 2024-04-26T20:57:57.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.319+0000 7f057ea00640 1 -- 172.21.15.28:0/865319930 learned_addr learned my addr 172.21.15.28:0/865319930 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:57.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.319+0000 7f057fe00640 1 --2- 172.21.15.28:0/865319930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05800cb500 0x7f058006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:57.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.319+0000 7f057e000640 1 -- 172.21.15.28:0/865319930 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2307429207 0 0) 0x7f0580145a10 con 0x7f05800cb0a0 2024-04-26T20:57:57.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.320+0000 7f057e000640 1 -- 172.21.15.28:0/865319930 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0564003520 con 0x7f05800cb0a0 2024-04-26T20:57:57.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.320+0000 7f057e000640 1 -- 172.21.15.28:0/865319930 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2412461215 0 0) 0x7f0564003520 con 0x7f05800cb0a0 2024-04-26T20:57:57.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.320+0000 7f057e000640 1 -- 172.21.15.28:0/865319930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05800cb500 msgr2=0x7f058006f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:57.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.320+0000 7f057e000640 1 --2- 172.21.15.28:0/865319930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05800cb500 0x7f058006f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:57.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.320+0000 7f057e000640 1 -- 172.21.15.28:0/865319930 >> v1:172.21.15.103:6789/0 conn(0x7f05800c9da0 legacy=0x7f05800ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:57.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.320+0000 7f057e000640 1 -- 172.21.15.28:0/865319930 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058015fac0 con 0x7f05800cb0a0 2024-04-26T20:57:57.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.320+0000 7f057e000640 1 -- 172.21.15.28:0/865319930 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f0568001670 con 0x7f05800cb0a0 2024-04-26T20:57:57.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.320+0000 7f057e000640 1 -- 172.21.15.28:0/865319930 >> v1:172.21.15.28:6789/0 conn(0x7f05800cb0a0 legacy=0x7f058006e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:57.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.320+0000 7f057e000640 1 --2- 172.21.15.28:0/865319930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05640039e0 0x7f0564003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:57.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.320+0000 7f057e000640 1 --2- 172.21.15.28:0/865319930 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0564004800 0x7f0564004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:57.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.320+0000 7f057e000640 1 --2- 172.21.15.28:0/865319930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05800cb500 0x7f05640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:57.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.320+0000 7f057e000640 1 -- 172.21.15.28:0/865319930 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f05801603f0 con 0x7f0564004800 2024-04-26T20:57:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.320+0000 7f057e000640 1 -- 172.21.15.28:0/865319930 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f056400baa0 con 0x7f05640039e0 2024-04-26T20:57:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.320+0000 7f057e000640 1 -- 172.21.15.28:0/865319930 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f056400bca0 con 0x7f05800cb500 2024-04-26T20:57:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.320+0000 7f057ea00640 1 --2- 172.21.15.28:0/865319930 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0564004800 0x7f0564004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.321+0000 7f057f400640 1 --2- 172.21.15.28:0/865319930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05640039e0 0x7f0564003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.321+0000 7f057fe00640 1 --2- 172.21.15.28:0/865319930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05800cb500 0x7f05640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.321+0000 7f057ea00640 1 -- 172.21.15.28:0/865319930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05800cb500 msgr2=0x7f05640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.321+0000 7f057ea00640 1 --2- 172.21.15.28:0/865319930 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f05800cb500 0x7f05640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.321+0000 7f057ea00640 1 -- 172.21.15.28:0/865319930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05640039e0 msgr2=0x7f0564003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.321+0000 7f057ea00640 1 --2- 172.21.15.28:0/865319930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f05640039e0 0x7f0564003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.321+0000 7f057ea00640 1 -- 172.21.15.28:0/865319930 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0564003520 con 0x7f0564004800 2024-04-26T20:57:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.322+0000 7f057ea00640 1 --2- 172.21.15.28:0/865319930 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0564004800 0x7f0564004c80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0568001eb0 tx=0x7f0564005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70fea312f7421d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.323+0000 7f057e000640 1 -- 172.21.15.28:0/865319930 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568007d30 con 0x7f0564004800 2024-04-26T20:57:57.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.323+0000 7f057e000640 1 -- 172.21.15.28:0/865319930 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0568005af0 con 0x7f0564004800 2024-04-26T20:57:57.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.323+0000 7f057e000640 1 -- 172.21.15.28:0/865319930 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568007070 con 0x7f0564004800 2024-04-26T20:57:57.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.323+0000 7f0584a00640 1 -- 172.21.15.28:0/865319930 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0564004800 msgr2=0x7f0564004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:57.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.323+0000 7f0584a00640 1 --2- 172.21.15.28:0/865319930 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0564004800 0x7f0564004c80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0568001eb0 tx=0x7f0564005830 comp rx=0 tx=0).stop 2024-04-26T20:57:57.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.323+0000 7f057fe00640 1 -- 172.21.15.28:0/865319930 reap_dead start 2024-04-26T20:57:57.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.323+0000 7f0584a00640 1 -- 172.21.15.28:0/865319930 shutdown_connections 2024-04-26T20:57:57.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.323+0000 7f0584a00640 1 -- 172.21.15.28:0/865319930 >> 172.21.15.28:0/865319930 conn(0x7f05800d0a50 msgr2=0x7f05800d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:57.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.323+0000 7f0584a00640 1 -- 172.21.15.28:0/865319930 shutdown_connections 2024-04-26T20:57:57.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.323+0000 7f0584a00640 1 -- 172.21.15.28:0/865319930 wait complete. 2024-04-26T20:57:57.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.325+0000 7f0584a00640 1 Processor -- start 2024-04-26T20:57:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.325+0000 7f0584a00640 1 -- start start 2024-04-26T20:57:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.326+0000 7f0584a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0564004800 0x7f058015de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.326+0000 7f0584a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f058013ebf0 0x7f058013f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.326+0000 7f0584a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f058013f9f0 0x7f0580072340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.326+0000 7f0584a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0580160080 con 0x7f058013f9f0 2024-04-26T20:57:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.326+0000 7f0584a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f058015f580 con 0x7f0564004800 2024-04-26T20:57:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.326+0000 7f0584a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f05801489a0 con 0x7f058013ebf0 2024-04-26T20:57:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.326+0000 7f057fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f058013f9f0 0x7f0580072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.327+0000 7f057fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f058013f9f0 0x7f0580072340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38230/0 (socket says 172.21.15.28:38230) 2024-04-26T20:57:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.327+0000 7f057fe00640 1 -- 172.21.15.28:0/3670018857 learned_addr learned my addr 172.21.15.28:0/3670018857 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.327+0000 7f057f400640 1 --2- 172.21.15.28:0/3670018857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0564004800 0x7f058015de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.327+0000 7f057ea00640 1 --2- 172.21.15.28:0/3670018857 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f058013ebf0 0x7f058013f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:57.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.327+0000 7f057fe00640 1 -- 172.21.15.28:0/3670018857 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f058013ebf0 msgr2=0x7f058013f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:57.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.327+0000 7f057fe00640 1 --2- 172.21.15.28:0/3670018857 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f058013ebf0 0x7f058013f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:57.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.327+0000 7f057fe00640 1 -- 172.21.15.28:0/3670018857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0564004800 msgr2=0x7f058015de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:57.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.327+0000 7f057fe00640 1 --2- 172.21.15.28:0/3670018857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0564004800 0x7f058015de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:57.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.327+0000 7f057fe00640 1 -- 172.21.15.28:0/3670018857 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f058013e150 con 0x7f058013f9f0 2024-04-26T20:57:57.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.327+0000 7f057ea00640 1 --2- 172.21.15.28:0/3670018857 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f058013ebf0 0x7f058013f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:57:57.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.327+0000 7f057fe00640 1 --2- 172.21.15.28:0/3670018857 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f058013f9f0 0x7f0580072340 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f05800cb8e0 tx=0x7f05780115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:57.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.328+0000 7f0577400640 1 -- 172.21.15.28:0/3670018857 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578003990 con 0x7f058013f9f0 2024-04-26T20:57:57.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.328+0000 7f0577400640 1 -- 172.21.15.28:0/3670018857 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f057801b070 con 0x7f058013f9f0 2024-04-26T20:57:57.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.328+0000 7f0577400640 1 -- 172.21.15.28:0/3670018857 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0578003b70 con 0x7f058013f9f0 2024-04-26T20:57:57.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.328+0000 7f0584a00640 1 -- 172.21.15.28:0/3670018857 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f058013ff30 con 0x7f058013f9f0 2024-04-26T20:57:57.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.328+0000 7f0584a00640 1 -- 172.21.15.28:0/3670018857 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f058013e6d0 con 0x7f058013f9f0 2024-04-26T20:57:57.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.329+0000 7f0584a00640 1 -- 172.21.15.28:0/3670018857 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f058006a360 con 0x7f058013f9f0 2024-04-26T20:57:57.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.330+0000 7f0577400640 1 -- 172.21.15.28:0/3670018857 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0578005d10 con 0x7f058013f9f0 2024-04-26T20:57:57.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.331+0000 7f0577400640 1 --2- 172.21.15.28:0/3670018857 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f055c06c8f0 0x7f055c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:57.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.331+0000 7f0577400640 1 -- 172.21.15.28:0/3670018857 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5825+0+0 (secure 0 0 0) 0x7f057808c080 con 0x7f058013f9f0 2024-04-26T20:57:57.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.331+0000 7f057f400640 1 --2- 172.21.15.28:0/3670018857 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f055c06c8f0 0x7f055c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:57.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.332+0000 7f057f400640 1 --2- 172.21.15.28:0/3670018857 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f055c06c8f0 0x7f055c06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0570005490 tx=0x7f0570009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:57.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.335+0000 7f0577400640 1 -- 172.21.15.28:0/3670018857 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f057805a420 con 0x7f058013f9f0 2024-04-26T20:57:57.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.580+0000 7f0584a00640 1 -- 172.21.15.28:0/3670018857 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0580074550 con 0x7f058013f9f0 2024-04-26T20:57:57.582 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-04-26T20:57:57.581+0000 7f9cbd800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:57:57.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.585+0000 7f0577400640 1 -- 172.21.15.28:0/3670018857 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f0578059d20 con 0x7f058013f9f0 2024-04-26T20:57:57.586 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T20:57:57.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.588+0000 7f0584a00640 1 -- 172.21.15.28:0/3670018857 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f055c06c8f0 msgr2=0x7f055c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:57.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.588+0000 7f0584a00640 1 --2- 172.21.15.28:0/3670018857 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f055c06c8f0 0x7f055c06edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0570005490 tx=0x7f0570009040 comp rx=0 tx=0).stop 2024-04-26T20:57:57.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.588+0000 7f0584a00640 1 -- 172.21.15.28:0/3670018857 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f058013f9f0 msgr2=0x7f0580072340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:57.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.588+0000 7f0584a00640 1 --2- 172.21.15.28:0/3670018857 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f058013f9f0 0x7f0580072340 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f05800cb8e0 tx=0x7f05780115a0 comp rx=0 tx=0).stop 2024-04-26T20:57:57.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.588+0000 7f0584a00640 1 -- 172.21.15.28:0/3670018857 shutdown_connections 2024-04-26T20:57:57.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.588+0000 7f0584a00640 1 --2- 172.21.15.28:0/3670018857 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f055c06c8f0 0x7f055c06edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:57.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.588+0000 7f0584a00640 1 --2- 172.21.15.28:0/3670018857 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f058013f9f0 0x7f0580072340 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:57.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.588+0000 7f0584a00640 1 --2- 172.21.15.28:0/3670018857 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f058013ebf0 0x7f058013f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:57.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.588+0000 7f0584a00640 1 --2- 172.21.15.28:0/3670018857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0564004800 0x7f058015de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:57.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.588+0000 7f0584a00640 1 -- 172.21.15.28:0/3670018857 >> 172.21.15.28:0/3670018857 conn(0x7f05800d0a50 msgr2=0x7f058006e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:57.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.589+0000 7f0584a00640 1 -- 172.21.15.28:0/3670018857 shutdown_connections 2024-04-26T20:57:57.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.589+0000 7f0584a00640 1 -- 172.21.15.28:0/3670018857 wait complete. 2024-04-26T20:57:57.610 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T20:57:57.614 INFO:tasks.ceph.mds.d.smithi103.stderr: -4112> 2024-04-26T20:56:54.606+0000 7f3d2ce00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:57:57.709 INFO:tasks.ceph.mds.d.smithi103.stdout:starting mds.d at 2024-04-26T20:57:57.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.807+0000 7f25fa000640 1 Processor -- start 2024-04-26T20:57:57.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.808+0000 7f25fa000640 1 -- start start 2024-04-26T20:57:57.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.809+0000 7f25fa000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f25f415f360 0x7f25f40b7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:57.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.809+0000 7f25fa000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25f40ba350 con 0x7f25f40d1de0 2024-04-26T20:57:57.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.809+0000 7f25fa000640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25f40b9ab0 con 0x7f25f40bac00 2024-04-26T20:57:57.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.809+0000 7f25fa000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f25f40b9d90 con 0x7f25f415f360 2024-04-26T20:57:57.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.810+0000 7f25f3e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f25f415f360 0x7f25f40b7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:57.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.810+0000 7f25f8c00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f25f40d1de0 0x7f25f40d21c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38902/0 (socket says 172.21.15.28:38902) 2024-04-26T20:57:57.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.810+0000 7f25f8c00640 1 -- 172.21.15.28:0/3491802959 learned_addr learned my addr 172.21.15.28:0/3491802959 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:57:57.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.810+0000 7f25f3e00640 1 -- 172.21.15.28:0/3491802959 >> v1:172.21.15.103:6789/0 conn(0x7f25f40bac00 legacy=0x7f25f40b8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:57.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.810+0000 7f25f3e00640 1 -- 172.21.15.28:0/3491802959 >> v1:172.21.15.28:6789/0 conn(0x7f25f40d1de0 legacy=0x7f25f40d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:57.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.810+0000 7f25f3e00640 1 -- 172.21.15.28:0/3491802959 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f40ba070 con 0x7f25f415f360 2024-04-26T20:57:57.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.810+0000 7f25f3400640 1 -- 172.21.15.28:0/3491802959 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2679386227 0 0) 0x7f25f40ba350 con 0x7f25f40d1de0 2024-04-26T20:57:57.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.811+0000 7f25f3e00640 1 --2- 172.21.15.28:0/3491802959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25f415f360 0x7f25f40b7fc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0097a0 tx=0x7f25dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7da9d7ca3cb98fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:57.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.812+0000 7f25f3400640 1 -- 172.21.15.28:0/3491802959 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc037070 con 0x7f25f415f360 2024-04-26T20:57:57.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.812+0000 7f25f3400640 1 -- 172.21.15.28:0/3491802959 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25dc032440 con 0x7f25f415f360 2024-04-26T20:57:57.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.812+0000 7f25f3400640 1 -- 172.21.15.28:0/3491802959 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc032810 con 0x7f25f415f360 2024-04-26T20:57:57.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.812+0000 7f25fa000640 1 -- 172.21.15.28:0/3491802959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25f415f360 msgr2=0x7f25f40b7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:57.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.812+0000 7f25fa000640 1 --2- 172.21.15.28:0/3491802959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25f415f360 0x7f25f40b7fc0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0097a0 tx=0x7f25dc02a1e0 comp rx=0 tx=0).stop 2024-04-26T20:57:57.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.812+0000 7f25fa000640 1 -- 172.21.15.28:0/3491802959 shutdown_connections 2024-04-26T20:57:57.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.812+0000 7f25fa000640 1 --2- 172.21.15.28:0/3491802959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25f415f360 0x7f25f40b7fc0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:57.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.812+0000 7f25fa000640 1 -- 172.21.15.28:0/3491802959 >> 172.21.15.28:0/3491802959 conn(0x7f25f406c770 msgr2=0x7f25f406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:57.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.813+0000 7f25fa000640 1 -- 172.21.15.28:0/3491802959 shutdown_connections 2024-04-26T20:57:57.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.813+0000 7f25fa000640 1 -- 172.21.15.28:0/3491802959 wait complete. 2024-04-26T20:57:57.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.814+0000 7f25fa000640 1 Processor -- start 2024-04-26T20:57:57.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.814+0000 7f25fa000640 1 -- start start 2024-04-26T20:57:57.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.815+0000 7f25fa000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25f40bac00 0x7f25f4153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:57.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.815+0000 7f25fa000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25f40d1de0 0x7f25f4156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:57.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.815+0000 7f25fa000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f25f415f360 0x7f25f415f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:57.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.815+0000 7f25fa000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f25f40d4620 con 0x7f25f40d1de0 2024-04-26T20:57:57.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.815+0000 7f25fa000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f25f40d44a0 con 0x7f25f40bac00 2024-04-26T20:57:57.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.815+0000 7f25fa000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f25f40d47a0 con 0x7f25f415f360 2024-04-26T20:57:57.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.816+0000 7f25f3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25f40d1de0 0x7f25f4156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:57.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.816+0000 7f25f3e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25f40d1de0 0x7f25f4156750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38246/0 (socket says 172.21.15.28:38246) 2024-04-26T20:57:57.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.816+0000 7f25f3e00640 1 -- 172.21.15.28:0/1309856439 learned_addr learned my addr 172.21.15.28:0/1309856439 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:57.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.816+0000 7f25f8c00640 1 --2- 172.21.15.28:0/1309856439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25f40bac00 0x7f25f4153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:57.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.816+0000 7f25f9600640 1 --2- 172.21.15.28:0/1309856439 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f25f415f360 0x7f25f415f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:57.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.816+0000 7f25f3e00640 1 -- 172.21.15.28:0/1309856439 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f25f415f360 msgr2=0x7f25f415f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:57.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.816+0000 7f25f3e00640 1 --2- 172.21.15.28:0/1309856439 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f25f415f360 0x7f25f415f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:57.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.816+0000 7f25f3e00640 1 -- 172.21.15.28:0/1309856439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25f40bac00 msgr2=0x7f25f4153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:57.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.816+0000 7f25f3e00640 1 --2- 172.21.15.28:0/1309856439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25f40bac00 0x7f25f4153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:57.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.816+0000 7f25f3e00640 1 -- 172.21.15.28:0/1309856439 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f4158260 con 0x7f25f40d1de0 2024-04-26T20:57:57.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.817+0000 7f25f3e00640 1 --2- 172.21.15.28:0/1309856439 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25f40d1de0 0x7f25f4156750 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0097a0 tx=0x7f25dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:57.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.817+0000 7f25f0c00640 1 -- 172.21.15.28:0/1309856439 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc037040 con 0x7f25f40d1de0 2024-04-26T20:57:57.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.817+0000 7f25f0c00640 1 -- 172.21.15.28:0/1309856439 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25dc004250 con 0x7f25f40d1de0 2024-04-26T20:57:57.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.817+0000 7f25f0c00640 1 -- 172.21.15.28:0/1309856439 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc032e20 con 0x7f25f40d1de0 2024-04-26T20:57:57.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.817+0000 7f25fa000640 1 -- 172.21.15.28:0/1309856439 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f4153c70 con 0x7f25f40d1de0 2024-04-26T20:57:57.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.817+0000 7f25fa000640 1 -- 172.21.15.28:0/1309856439 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f41574a0 con 0x7f25f40d1de0 2024-04-26T20:57:57.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.818+0000 7f25fa000640 1 -- 172.21.15.28:0/1309856439 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25f406a250 con 0x7f25f40d1de0 2024-04-26T20:57:57.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.819+0000 7f25f0c00640 1 -- 172.21.15.28:0/1309856439 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25dc030770 con 0x7f25f40d1de0 2024-04-26T20:57:57.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.820+0000 7f25f0c00640 1 --2- 172.21.15.28:0/1309856439 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f25d006c8f0 0x7f25d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:57.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.820+0000 7f25f0c00640 1 -- 172.21.15.28:0/1309856439 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5911+0+0 (secure 0 0 0) 0x7f25dc0acfc0 con 0x7f25f40d1de0 2024-04-26T20:57:57.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.820+0000 7f25f8c00640 1 --2- 172.21.15.28:0/1309856439 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f25d006c8f0 0x7f25d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:57.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.821+0000 7f25f8c00640 1 --2- 172.21.15.28:0/1309856439 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f25d006c8f0 0x7f25d006edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f25e4000f60 tx=0x7f25e4007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:57.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:57.824+0000 7f25f0c00640 1 -- 172.21.15.28:0/1309856439 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f25dc07b280 con 0x7f25f40d1de0 2024-04-26T20:57:58.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.071+0000 7f25fa000640 1 -- 172.21.15.28:0/1309856439 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f25f40d41f0 con 0x7f25f40d1de0 2024-04-26T20:57:58.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.588+0000 7f25f0c00640 1 -- 172.21.15.28:0/1309856439 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f25dc07ab80 con 0x7f25f40d1de0 2024-04-26T20:57:58.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.592+0000 7f25fa000640 1 -- 172.21.15.28:0/1309856439 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f25d006c8f0 msgr2=0x7f25d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:58.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.592+0000 7f25fa000640 1 --2- 172.21.15.28:0/1309856439 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f25d006c8f0 0x7f25d006edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f25e4000f60 tx=0x7f25e4007b50 comp rx=0 tx=0).stop 2024-04-26T20:57:58.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.592+0000 7f25fa000640 1 -- 172.21.15.28:0/1309856439 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25f40d1de0 msgr2=0x7f25f4156750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:58.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.592+0000 7f25fa000640 1 --2- 172.21.15.28:0/1309856439 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25f40d1de0 0x7f25f4156750 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0097a0 tx=0x7f25dc0046c0 comp rx=0 tx=0).stop 2024-04-26T20:57:58.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.592+0000 7f25fa000640 1 -- 172.21.15.28:0/1309856439 shutdown_connections 2024-04-26T20:57:58.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.592+0000 7f25fa000640 1 --2- 172.21.15.28:0/1309856439 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f25d006c8f0 0x7f25d006edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:58.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.592+0000 7f25fa000640 1 --2- 172.21.15.28:0/1309856439 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f25f415f360 0x7f25f415f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:58.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.592+0000 7f25fa000640 1 --2- 172.21.15.28:0/1309856439 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25f40d1de0 0x7f25f4156750 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:58.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.592+0000 7f25fa000640 1 --2- 172.21.15.28:0/1309856439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f25f40bac00 0x7f25f4153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:58.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.592+0000 7f25fa000640 1 -- 172.21.15.28:0/1309856439 >> 172.21.15.28:0/1309856439 conn(0x7f25f406c770 msgr2=0x7f25f40b84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:58.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.592+0000 7f25fa000640 1 -- 172.21.15.28:0/1309856439 shutdown_connections 2024-04-26T20:57:58.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.592+0000 7f25fa000640 1 -- 172.21.15.28:0/1309856439 wait complete. 2024-04-26T20:57:58.620 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T20:57:58.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.838+0000 7fbb20a00640 1 Processor -- start 2024-04-26T20:57:58.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.838+0000 7fbb20a00640 1 -- start start 2024-04-26T20:57:58.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.840+0000 7fbb20a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbb1c15cc90 0x7fbb1c0b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:58.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.840+0000 7fbb20a00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fbb1c15f7d0 0x7fbb1c15fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:58.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.840+0000 7fbb20a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbb1c160150 0x7fbb1c162dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:58.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.840+0000 7fbb20a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fbb1c072480 con 0x7fbb1c160150 2024-04-26T20:57:58.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.840+0000 7fbb20a00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fbb1c0b64e0 con 0x7fbb1c15f7d0 2024-04-26T20:57:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.840+0000 7fbb20a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fbb1c0b67c0 con 0x7fbb1c15cc90 2024-04-26T20:57:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.841+0000 7fbb1a800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbb1c160150 0x7fbb1c162dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.841+0000 7fbb1a800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbb1c160150 0x7fbb1c162dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38262/0 (socket says 172.21.15.28:38262) 2024-04-26T20:57:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.841+0000 7fbb1a800640 1 -- 172.21.15.28:0/2504747198 learned_addr learned my addr 172.21.15.28:0/2504747198 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.841+0000 7fbb19e00640 1 --2- 172.21.15.28:0/2504747198 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbb1c15cc90 0x7fbb1c0b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.841+0000 7fbb19400640 1 --2- 172.21.15.28:0/2504747198 >> v2:172.21.15.103:3300/0 conn(0x7fbb1c15f7d0 0x7fbb1c15fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.841+0000 7fbb1a800640 1 -- 172.21.15.28:0/2504747198 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbb1c15cc90 msgr2=0x7fbb1c0b5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.841+0000 7fbb1a800640 1 --2- 172.21.15.28:0/2504747198 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbb1c15cc90 0x7fbb1c0b5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.841+0000 7fbb1a800640 1 -- 172.21.15.28:0/2504747198 >> v2:172.21.15.103:3300/0 conn(0x7fbb1c15f7d0 msgr2=0x7fbb1c15fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.841+0000 7fbb1a800640 1 --2- 172.21.15.28:0/2504747198 >> v2:172.21.15.103:3300/0 conn(0x7fbb1c15f7d0 0x7fbb1c15fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:58.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.841+0000 7fbb1a800640 1 -- 172.21.15.28:0/2504747198 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1c0b6aa0 con 0x7fbb1c160150 2024-04-26T20:57:58.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.842+0000 7fbb1a800640 1 --2- 172.21.15.28:0/2504747198 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb1c160150 0x7fbb1c162dc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c06dec0 tx=0x7fbb1402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47f9522f635d20c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:58.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.843+0000 7fbb18a00640 1 -- 172.21.15.28:0/2504747198 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14036070 con 0x7fbb1c160150 2024-04-26T20:57:58.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.843+0000 7fbb18a00640 1 -- 172.21.15.28:0/2504747198 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb14031600 con 0x7fbb1c160150 2024-04-26T20:57:58.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.843+0000 7fbb18a00640 1 -- 172.21.15.28:0/2504747198 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb140319d0 con 0x7fbb1c160150 2024-04-26T20:57:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.843+0000 7fbb20a00640 1 -- 172.21.15.28:0/2504747198 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb1c160150 msgr2=0x7fbb1c162dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.843+0000 7fbb20a00640 1 --2- 172.21.15.28:0/2504747198 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb1c160150 0x7fbb1c162dc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c06dec0 tx=0x7fbb1402a6a0 comp rx=0 tx=0).stop 2024-04-26T20:57:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.844+0000 7fbb20a00640 1 -- 172.21.15.28:0/2504747198 shutdown_connections 2024-04-26T20:57:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.844+0000 7fbb20a00640 1 --2- 172.21.15.28:0/2504747198 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb1c160150 0x7fbb1c162dc0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.844+0000 7fbb20a00640 1 --2- 172.21.15.28:0/2504747198 >> v2:172.21.15.103:3300/0 conn(0x7fbb1c15f7d0 0x7fbb1c15fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.844+0000 7fbb20a00640 1 --2- 172.21.15.28:0/2504747198 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbb1c15cc90 0x7fbb1c0b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.844+0000 7fbb20a00640 1 -- 172.21.15.28:0/2504747198 >> 172.21.15.28:0/2504747198 conn(0x7fbb1c06e220 msgr2=0x7fbb1c06e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:58.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.844+0000 7fbb20a00640 1 -- 172.21.15.28:0/2504747198 shutdown_connections 2024-04-26T20:57:58.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.844+0000 7fbb20a00640 1 -- 172.21.15.28:0/2504747198 wait complete. 2024-04-26T20:57:58.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.846+0000 7fbb20a00640 1 Processor -- start 2024-04-26T20:57:58.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.846+0000 7fbb20a00640 1 -- start start 2024-04-26T20:57:58.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.848+0000 7fbb20a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbb1c15cc90 0x7fbb1c15bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:58.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.848+0000 7fbb20a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb1c15f7d0 0x7fbb1c06e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:58.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.848+0000 7fbb20a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb1c160150 0x7fbb1c06eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:58.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.848+0000 7fbb20a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c15d070 con 0x7fbb1c15f7d0 2024-04-26T20:57:58.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.848+0000 7fbb20a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1605b0 con 0x7fbb1c160150 2024-04-26T20:57:58.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.848+0000 7fbb20a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fbb1c0b60f0 con 0x7fbb1c15cc90 2024-04-26T20:57:58.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.848+0000 7fbb19400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb1c15f7d0 0x7fbb1c06e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:58.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.848+0000 7fbb19400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb1c15f7d0 0x7fbb1c06e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:38274/0 (socket says 172.21.15.28:38274) 2024-04-26T20:57:58.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.848+0000 7fbb19400640 1 -- 172.21.15.28:0/489232975 learned_addr learned my addr 172.21.15.28:0/489232975 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:58.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.848+0000 7fbb19400640 1 -- 172.21.15.28:0/489232975 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbb1c15cc90 msgr2=0x7fbb1c15bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:58.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.848+0000 7fbb19400640 1 --2- 172.21.15.28:0/489232975 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbb1c15cc90 0x7fbb1c15bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:58.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.848+0000 7fbb19400640 1 -- 172.21.15.28:0/489232975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb1c160150 msgr2=0x7fbb1c06eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:58.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.848+0000 7fbb19400640 1 --2- 172.21.15.28:0/489232975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb1c160150 0x7fbb1c06eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:58.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.848+0000 7fbb19400640 1 -- 172.21.15.28:0/489232975 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1c1eb7f0 con 0x7fbb1c15f7d0 2024-04-26T20:57:58.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.849+0000 7fbb19400640 1 --2- 172.21.15.28:0/489232975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb1c15f7d0 0x7fbb1c06e600 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800d800 tx=0x7fbb0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:58.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.849+0000 7fbb12000640 1 -- 172.21.15.28:0/489232975 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0800b6c0 con 0x7fbb1c15f7d0 2024-04-26T20:57:58.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.849+0000 7fbb12000640 1 -- 172.21.15.28:0/489232975 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb08019070 con 0x7fbb1c15f7d0 2024-04-26T20:57:58.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.849+0000 7fbb12000640 1 -- 172.21.15.28:0/489232975 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0800ba80 con 0x7fbb1c15f7d0 2024-04-26T20:57:58.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.849+0000 7fbb20a00640 1 -- 172.21.15.28:0/489232975 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb1c1eba90 con 0x7fbb1c15f7d0 2024-04-26T20:57:58.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.849+0000 7fbb20a00640 1 -- 172.21.15.28:0/489232975 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb1c1ec050 con 0x7fbb1c15f7d0 2024-04-26T20:57:58.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.850+0000 7fbb20a00640 1 -- 172.21.15.28:0/489232975 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb1c076520 con 0x7fbb1c15f7d0 2024-04-26T20:57:58.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.852+0000 7fbb12000640 1 -- 172.21.15.28:0/489232975 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb0800bc20 con 0x7fbb1c15f7d0 2024-04-26T20:57:58.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.852+0000 7fbb12000640 1 --2- 172.21.15.28:0/489232975 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbafc06c940 0x7fbafc06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:58.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.853+0000 7fbb12000640 1 -- 172.21.15.28:0/489232975 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5911+0+0 (secure 0 0 0) 0x7fbb0808c070 con 0x7fbb1c15f7d0 2024-04-26T20:57:58.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.853+0000 7fbb19e00640 1 --2- 172.21.15.28:0/489232975 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbafc06c940 0x7fbafc06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:58.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.853+0000 7fbb19e00640 1 --2- 172.21.15.28:0/489232975 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbafc06c940 0x7fbafc06ee00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009870 tx=0x7fbb0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:58.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:58.857+0000 7fbb12000640 1 -- 172.21.15.28:0/489232975 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb0805a440 con 0x7fbb1c15f7d0 2024-04-26T20:57:59.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.061+0000 7fbb20a00640 1 -- 172.21.15.28:0/489232975 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbb1c0b28d0 con 0x7fbb1c15f7d0 2024-04-26T20:57:59.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.598+0000 7fbb12000640 1 -- 172.21.15.28:0/489232975 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) ==== 181+0+0 (secure 0 0 0) 0x7fbb08059d40 con 0x7fbb1c15f7d0 2024-04-26T20:57:59.600 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' removed 2024-04-26T20:57:59.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.602+0000 7fbb20a00640 1 -- 172.21.15.28:0/489232975 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbafc06c940 msgr2=0x7fbafc06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:59.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.602+0000 7fbb20a00640 1 --2- 172.21.15.28:0/489232975 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbafc06c940 0x7fbafc06ee00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009870 tx=0x7fbb0c008040 comp rx=0 tx=0).stop 2024-04-26T20:57:59.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.602+0000 7fbb20a00640 1 -- 172.21.15.28:0/489232975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb1c15f7d0 msgr2=0x7fbb1c06e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:59.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.602+0000 7fbb20a00640 1 --2- 172.21.15.28:0/489232975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb1c15f7d0 0x7fbb1c06e600 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800d800 tx=0x7fbb0800dcc0 comp rx=0 tx=0).stop 2024-04-26T20:57:59.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.603+0000 7fbb20a00640 1 -- 172.21.15.28:0/489232975 shutdown_connections 2024-04-26T20:57:59.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.603+0000 7fbb20a00640 1 --2- 172.21.15.28:0/489232975 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fbafc06c940 0x7fbafc06ee00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:59.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.603+0000 7fbb20a00640 1 --2- 172.21.15.28:0/489232975 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb1c160150 0x7fbb1c06eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:59.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.603+0000 7fbb20a00640 1 --2- 172.21.15.28:0/489232975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbb1c15f7d0 0x7fbb1c06e600 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:59.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.603+0000 7fbb20a00640 1 --2- 172.21.15.28:0/489232975 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fbb1c15cc90 0x7fbb1c15bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:59.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.603+0000 7fbb20a00640 1 -- 172.21.15.28:0/489232975 >> 172.21.15.28:0/489232975 conn(0x7fbb1c06e220 msgr2=0x7fbb1c0afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:59.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.604+0000 7fbb20a00640 1 -- 172.21.15.28:0/489232975 shutdown_connections 2024-04-26T20:57:59.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.605+0000 7fbb20a00640 1 -- 172.21.15.28:0/489232975 wait complete. 2024-04-26T20:57:59.633 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T20:57:59.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.846+0000 7ff77a400640 1 Processor -- start 2024-04-26T20:57:59.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.846+0000 7ff77a400640 1 -- start start 2024-04-26T20:57:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.848+0000 7ff77a400640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff774071fb0 0x7ff7740ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.848+0000 7ff77a400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff7740ceca0 0x7ff77415d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.848+0000 7ff77a400640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff774073630 con 0x7ff7740b6100 2024-04-26T20:57:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.848+0000 7ff77a400640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7ff774073910 con 0x7ff774071fb0 2024-04-26T20:57:59.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.848+0000 7ff77a400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff774073bf0 con 0x7ff7740ceca0 2024-04-26T20:57:59.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.849+0000 7ff773e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff7740ceca0 0x7ff77415d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:59.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.849+0000 7ff772a00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff774071fb0 0x7ff7740ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:59.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.849+0000 7ff773e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff7740ceca0 0x7ff77415d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.28:55026/0 (socket says 172.21.15.28:55026) 2024-04-26T20:57:59.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.849+0000 7ff773e00640 1 -- 172.21.15.28:0/2953471818 learned_addr learned my addr 172.21.15.28:0/2953471818 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:59.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.849+0000 7ff773400640 1 --1- 172.21.15.28:0/2953471818 >> v1:172.21.15.103:6789/0 conn(0x7ff7740b6100 0x7ff7740b64e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.28:44422/0 (socket says 172.21.15.28:44422) 2024-04-26T20:57:59.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.850+0000 7ff773e00640 1 -- 172.21.15.28:0/2953471818 >> v2:172.21.15.103:3300/0 conn(0x7ff774071fb0 msgr2=0x7ff7740ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:57:59.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.850+0000 7ff773e00640 1 --2- 172.21.15.28:0/2953471818 >> v2:172.21.15.103:3300/0 conn(0x7ff774071fb0 0x7ff7740ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:59.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.850+0000 7ff773e00640 1 -- 172.21.15.28:0/2953471818 >> v1:172.21.15.103:6789/0 conn(0x7ff7740b6100 legacy=0x7ff7740b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:59.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.850+0000 7ff773e00640 1 -- 172.21.15.28:0/2953471818 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff774073ed0 con 0x7ff7740ceca0 2024-04-26T20:57:59.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.850+0000 7ff772a00640 1 --2- 172.21.15.28:0/2953471818 >> v2:172.21.15.103:3300/0 conn(0x7ff774071fb0 0x7ff7740ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T20:57:59.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.851+0000 7ff773e00640 1 --2- 172.21.15.28:0/2953471818 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff7740ceca0 0x7ff77415d5b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009e50 tx=0x7ff76c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e343f62178b319d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:59.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.851+0000 7ff772000640 1 -- 172.21.15.28:0/2953471818 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c036070 con 0x7ff7740ceca0 2024-04-26T20:57:59.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.852+0000 7ff772000640 1 -- 172.21.15.28:0/2953471818 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff76c031600 con 0x7ff7740ceca0 2024-04-26T20:57:59.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.852+0000 7ff772000640 1 -- 172.21.15.28:0/2953471818 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c0319d0 con 0x7ff7740ceca0 2024-04-26T20:57:59.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.852+0000 7ff77a400640 1 -- 172.21.15.28:0/2953471818 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff7740ceca0 msgr2=0x7ff77415d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:59.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.852+0000 7ff77a400640 1 --2- 172.21.15.28:0/2953471818 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff7740ceca0 0x7ff77415d5b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009e50 tx=0x7ff76c02a710 comp rx=0 tx=0).stop 2024-04-26T20:57:59.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.852+0000 7ff77a400640 1 -- 172.21.15.28:0/2953471818 shutdown_connections 2024-04-26T20:57:59.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.852+0000 7ff77a400640 1 --2- 172.21.15.28:0/2953471818 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff7740ceca0 0x7ff77415d5b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:59.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.852+0000 7ff77a400640 1 --2- 172.21.15.28:0/2953471818 >> v2:172.21.15.103:3300/0 conn(0x7ff774071fb0 0x7ff7740ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:59.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.852+0000 7ff77a400640 1 -- 172.21.15.28:0/2953471818 >> 172.21.15.28:0/2953471818 conn(0x7ff77406e220 msgr2=0x7ff77406e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:57:59.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.852+0000 7ff77a400640 1 -- 172.21.15.28:0/2953471818 shutdown_connections 2024-04-26T20:57:59.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.852+0000 7ff77a400640 1 -- 172.21.15.28:0/2953471818 wait complete. 2024-04-26T20:57:59.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.854+0000 7ff77a400640 1 Processor -- start 2024-04-26T20:57:59.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.854+0000 7ff77a400640 1 -- start start 2024-04-26T20:57:59.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.855+0000 7ff77a400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff774071fb0 0x7ff77415bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:59.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.855+0000 7ff77a400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7740b6100 0x7ff77415dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:59.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.855+0000 7ff77a400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7740ceca0 0x7ff7740b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:59.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.855+0000 7ff77a400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff774145fe0 con 0x7ff7740ceca0 2024-04-26T20:57:59.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.855+0000 7ff77a400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff7740cf0b0 con 0x7ff7740b6100 2024-04-26T20:57:59.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.855+0000 7ff77a400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff774146160 con 0x7ff774071fb0 2024-04-26T20:57:59.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.856+0000 7ff773e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7740ceca0 0x7ff7740b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:59.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.856+0000 7ff773e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7740ceca0 0x7ff7740b0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60842/0 (socket says 172.21.15.28:60842) 2024-04-26T20:57:59.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.856+0000 7ff773e00640 1 -- 172.21.15.28:0/2887513568 learned_addr learned my addr 172.21.15.28:0/2887513568 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:57:59.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.856+0000 7ff773400640 1 --2- 172.21.15.28:0/2887513568 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff774071fb0 0x7ff77415bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:59.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.856+0000 7ff772a00640 1 --2- 172.21.15.28:0/2887513568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7740b6100 0x7ff77415dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:59.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.856+0000 7ff773e00640 1 -- 172.21.15.28:0/2887513568 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff774071fb0 msgr2=0x7ff77415bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:59.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.856+0000 7ff773e00640 1 --2- 172.21.15.28:0/2887513568 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff774071fb0 0x7ff77415bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:59.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.856+0000 7ff773e00640 1 -- 172.21.15.28:0/2887513568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7740b6100 msgr2=0x7ff77415dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:57:59.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.856+0000 7ff773e00640 1 --2- 172.21.15.28:0/2887513568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7740b6100 0x7ff77415dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:57:59.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.856+0000 7ff773e00640 1 -- 172.21.15.28:0/2887513568 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77415c2d0 con 0x7ff7740ceca0 2024-04-26T20:57:59.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.856+0000 7ff773e00640 1 --2- 172.21.15.28:0/2887513568 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7740ceca0 0x7ff7740b0960 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff76c034040 tx=0x7ff76c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:59.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.857+0000 7ff76b400640 1 -- 172.21.15.28:0/2887513568 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c041020 con 0x7ff7740ceca0 2024-04-26T20:57:59.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.857+0000 7ff76b400640 1 -- 172.21.15.28:0/2887513568 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff76c0077a0 con 0x7ff7740ceca0 2024-04-26T20:57:59.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.857+0000 7ff76b400640 1 -- 172.21.15.28:0/2887513568 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff76c036040 con 0x7ff7740ceca0 2024-04-26T20:57:59.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.857+0000 7ff77a400640 1 -- 172.21.15.28:0/2887513568 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77412f1b0 con 0x7ff7740ceca0 2024-04-26T20:57:59.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.857+0000 7ff77a400640 1 -- 172.21.15.28:0/2887513568 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77412fa50 con 0x7ff7740ceca0 2024-04-26T20:57:59.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.858+0000 7ff77a400640 1 -- 172.21.15.28:0/2887513568 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff774076520 con 0x7ff7740ceca0 2024-04-26T20:57:59.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.859+0000 7ff76b400640 1 -- 172.21.15.28:0/2887513568 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff76c044020 con 0x7ff7740ceca0 2024-04-26T20:57:59.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.860+0000 7ff76b400640 1 --2- 172.21.15.28:0/2887513568 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff75406c8f0 0x7ff75406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:57:59.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.860+0000 7ff76b400640 1 -- 172.21.15.28:0/2887513568 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5473+0+0 (secure 0 0 0) 0x7ff76c0ac730 con 0x7ff7740ceca0 2024-04-26T20:57:59.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.860+0000 7ff773400640 1 --2- 172.21.15.28:0/2887513568 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff75406c8f0 0x7ff75406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:57:59.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.861+0000 7ff773400640 1 --2- 172.21.15.28:0/2887513568 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff75406c8f0 0x7ff75406edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff764000f60 tx=0x7ff764007cc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:57:59.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:57:59.865+0000 7ff76b400640 1 -- 172.21.15.28:0/2887513568 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff76c04b030 con 0x7ff7740ceca0 2024-04-26T20:58:00.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.068+0000 7ff77a400640 1 -- 172.21.15.28:0/2887513568 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff77406ff90 con 0x7ff7740ceca0 2024-04-26T20:58:00.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.602+0000 7ff76b400640 1 -- 172.21.15.28:0/2887513568 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) ==== 169+0+0 (secure 0 0 0) 0x7ff77415c2d0 con 0x7ff7740ceca0 2024-04-26T20:58:00.604 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' removed 2024-04-26T20:58:00.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.605+0000 7ff77a400640 1 -- 172.21.15.28:0/2887513568 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff75406c8f0 msgr2=0x7ff75406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:00.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.605+0000 7ff77a400640 1 --2- 172.21.15.28:0/2887513568 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff75406c8f0 0x7ff75406edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff764000f60 tx=0x7ff764007cc0 comp rx=0 tx=0).stop 2024-04-26T20:58:00.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.605+0000 7ff77a400640 1 -- 172.21.15.28:0/2887513568 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7740ceca0 msgr2=0x7ff7740b0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:00.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.605+0000 7ff77a400640 1 --2- 172.21.15.28:0/2887513568 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7740ceca0 0x7ff7740b0960 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff76c034040 tx=0x7ff76c004a40 comp rx=0 tx=0).stop 2024-04-26T20:58:00.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.606+0000 7ff77a400640 1 -- 172.21.15.28:0/2887513568 shutdown_connections 2024-04-26T20:58:00.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.606+0000 7ff77a400640 1 --2- 172.21.15.28:0/2887513568 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff75406c8f0 0x7ff75406edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:00.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.606+0000 7ff77a400640 1 --2- 172.21.15.28:0/2887513568 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7740ceca0 0x7ff7740b0960 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:00.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.606+0000 7ff77a400640 1 --2- 172.21.15.28:0/2887513568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7740b6100 0x7ff77415dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:00.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.606+0000 7ff77a400640 1 --2- 172.21.15.28:0/2887513568 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff774071fb0 0x7ff77415bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:00.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.606+0000 7ff77a400640 1 -- 172.21.15.28:0/2887513568 >> 172.21.15.28:0/2887513568 conn(0x7ff77406e220 msgr2=0x7ff7740aff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:00.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.608+0000 7ff77a400640 1 -- 172.21.15.28:0/2887513568 shutdown_connections 2024-04-26T20:58:00.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.608+0000 7ff77a400640 1 -- 172.21.15.28:0/2887513568 wait complete. 2024-04-26T20:58:00.639 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T20:58:00.639 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T20:58:00.639 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T20:58:00.639 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:58:00.639 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T20:58:00.640 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T20:58:00.640 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T20:58:00.640 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:58:00.640 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T20:58:00.640 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T20:58:00.640 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T20:58:00.640 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:58:00.641 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T20:58:00.641 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T20:58:00.641 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T20:58:00.641 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:58:00.659 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T20:58:00.660 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T20:58:00.662 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T20:58:00.662 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T20:58:00.664 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T20:58:00.664 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T20:58:00.668 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T20:58:00.668 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T20:58:00.707 INFO:tasks.ceph.mds.a:Started 2024-04-26T20:58:00.707 DEBUG:teuthology.parallel:result is None 2024-04-26T20:58:00.712 INFO:tasks.ceph.mds.b:Started 2024-04-26T20:58:00.713 DEBUG:teuthology.parallel:result is None 2024-04-26T20:58:00.713 INFO:tasks.ceph.mds.c:Started 2024-04-26T20:58:00.713 DEBUG:teuthology.parallel:result is None 2024-04-26T20:58:00.717 INFO:tasks.ceph.mds.d:Started 2024-04-26T20:58:00.717 DEBUG:teuthology.parallel:result is None 2024-04-26T20:58:00.717 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T20:58:00.856 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-04-26T20:58:00.859 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-04-26T20:58:00.865 INFO:tasks.ceph.mds.b.smithi103.stdout:starting mds.b at 2024-04-26T20:58:00.865 INFO:tasks.ceph.mds.d.smithi103.stdout:starting mds.d at 2024-04-26T20:58:00.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.898+0000 7fca4b600640 1 Processor -- start 2024-04-26T20:58:00.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.898+0000 7fca4b600640 1 -- start start 2024-04-26T20:58:00.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.899+0000 7fca4b600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fca440d00f0 0x7fca441607e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:00.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.899+0000 7fca4b600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fca440b6b10 0x7fca440b6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:00.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.899+0000 7fca4b600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca440b47a0 con 0x7fca440b43c0 2024-04-26T20:58:00.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.899+0000 7fca4b600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fca44160d20 con 0x7fca440d00f0 2024-04-26T20:58:00.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.899+0000 7fca4b600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fca440b74a0 con 0x7fca440b6b10 2024-04-26T20:58:00.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.899+0000 7fca48c00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fca440b43c0 0x7fca44163cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36482/0 (socket says 172.21.15.28:36482) 2024-04-26T20:58:00.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.899+0000 7fca48c00640 1 -- 172.21.15.28:0/2146453314 learned_addr learned my addr 172.21.15.28:0/2146453314 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:00.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.899+0000 7fca3be00640 1 --2- 172.21.15.28:0/2146453314 >> v2:172.21.15.28:3300/0 conn(0x7fca440d00f0 0x7fca441607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:00.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.900+0000 7fca49600640 1 --2- 172.21.15.28:0/2146453314 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fca440b6b10 0x7fca440b6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:00.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.900+0000 7fca3be00640 1 -- 172.21.15.28:0/2146453314 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fca440b6b10 msgr2=0x7fca440b6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:00.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.900+0000 7fca3be00640 1 --2- 172.21.15.28:0/2146453314 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fca440b6b10 0x7fca440b6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:00.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.900+0000 7fca3be00640 1 -- 172.21.15.28:0/2146453314 >> v1:172.21.15.28:6789/0 conn(0x7fca440b43c0 legacy=0x7fca44163cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:00.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.900+0000 7fca3be00640 1 -- 172.21.15.28:0/2146453314 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca440d0740 con 0x7fca440d00f0 2024-04-26T20:58:00.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.901+0000 7fca3be00640 1 --2- 172.21.15.28:0/2146453314 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca440d00f0 0x7fca441607e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0097a0 tx=0x7fca2c02a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e27d7199aaa597 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:00.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.901+0000 7fca43e00640 1 -- 172.21.15.28:0/2146453314 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c037070 con 0x7fca440d00f0 2024-04-26T20:58:00.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.901+0000 7fca43e00640 1 -- 172.21.15.28:0/2146453314 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca2c02ae40 con 0x7fca440d00f0 2024-04-26T20:58:00.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.901+0000 7fca43e00640 1 -- 172.21.15.28:0/2146453314 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2c032660 con 0x7fca440d00f0 2024-04-26T20:58:00.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.902+0000 7fca4b600640 1 -- 172.21.15.28:0/2146453314 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca440d00f0 msgr2=0x7fca441607e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:00.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.902+0000 7fca4b600640 1 --2- 172.21.15.28:0/2146453314 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca440d00f0 0x7fca441607e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0097a0 tx=0x7fca2c02a0f0 comp rx=0 tx=0).stop 2024-04-26T20:58:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.902+0000 7fca4b600640 1 -- 172.21.15.28:0/2146453314 shutdown_connections 2024-04-26T20:58:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.902+0000 7fca4b600640 1 --2- 172.21.15.28:0/2146453314 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fca440b6b10 0x7fca440b6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.902+0000 7fca4b600640 1 --2- 172.21.15.28:0/2146453314 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca440d00f0 0x7fca441607e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.902+0000 7fca4b600640 1 -- 172.21.15.28:0/2146453314 >> 172.21.15.28:0/2146453314 conn(0x7fca440d1b10 msgr2=0x7fca440d1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.902+0000 7fca4b600640 1 -- 172.21.15.28:0/2146453314 shutdown_connections 2024-04-26T20:58:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.902+0000 7fca4b600640 1 -- 172.21.15.28:0/2146453314 wait complete. 2024-04-26T20:58:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.903+0000 7fca4b600640 1 Processor -- start 2024-04-26T20:58:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.904+0000 7fca4b600640 1 -- start start 2024-04-26T20:58:00.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.904+0000 7fca4b600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca440b43c0 0x7fca44159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:00.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.904+0000 7fca4b600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca440b6b10 0x7fca4415ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:00.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.904+0000 7fca4b600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fca440d00f0 0x7fca441f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:00.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.904+0000 7fca4b600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fca44163790 con 0x7fca440b43c0 2024-04-26T20:58:00.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.904+0000 7fca4b600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca441634b0 con 0x7fca440b6b10 2024-04-26T20:58:00.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.904+0000 7fca4b600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fca440d3880 con 0x7fca440d00f0 2024-04-26T20:58:00.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.905+0000 7fca48c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca440b43c0 0x7fca44159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:00.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.905+0000 7fca48c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca440b43c0 0x7fca44159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60900/0 (socket says 172.21.15.28:60900) 2024-04-26T20:58:00.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.905+0000 7fca48c00640 1 -- 172.21.15.28:0/693835475 learned_addr learned my addr 172.21.15.28:0/693835475 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:00.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.905+0000 7fca3be00640 1 --2- 172.21.15.28:0/693835475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca440b6b10 0x7fca4415ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:00.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.905+0000 7fca49600640 1 --2- 172.21.15.28:0/693835475 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fca440d00f0 0x7fca441f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:00.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.905+0000 7fca48c00640 1 -- 172.21.15.28:0/693835475 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fca440d00f0 msgr2=0x7fca441f5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:00.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.905+0000 7fca48c00640 1 --2- 172.21.15.28:0/693835475 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fca440d00f0 0x7fca441f5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:00.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.905+0000 7fca48c00640 1 -- 172.21.15.28:0/693835475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca440b6b10 msgr2=0x7fca4415ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:00.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.905+0000 7fca48c00640 1 --2- 172.21.15.28:0/693835475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca440b6b10 0x7fca4415ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:00.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.905+0000 7fca48c00640 1 -- 172.21.15.28:0/693835475 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4415cfd0 con 0x7fca440b43c0 2024-04-26T20:58:00.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.906+0000 7fca48c00640 1 --2- 172.21.15.28:0/693835475 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca440b43c0 0x7fca44159830 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fca3400c400 tx=0x7fca3400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:00.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.906+0000 7fca41600640 1 -- 172.21.15.28:0/693835475 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca34017070 con 0x7fca440b43c0 2024-04-26T20:58:00.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.906+0000 7fca41600640 1 -- 172.21.15.28:0/693835475 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca34005ce0 con 0x7fca440b43c0 2024-04-26T20:58:00.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.906+0000 7fca41600640 1 -- 172.21.15.28:0/693835475 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca340072f0 con 0x7fca440b43c0 2024-04-26T20:58:00.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.906+0000 7fca4b600640 1 -- 172.21.15.28:0/693835475 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca440cd740 con 0x7fca440b43c0 2024-04-26T20:58:00.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.906+0000 7fca4b600640 1 -- 172.21.15.28:0/693835475 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca440cdd00 con 0x7fca440b43c0 2024-04-26T20:58:00.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.907+0000 7fca4b600640 1 -- 172.21.15.28:0/693835475 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca4406a400 con 0x7fca440b43c0 2024-04-26T20:58:00.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.908+0000 7fca41600640 1 -- 172.21.15.28:0/693835475 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca3401a070 con 0x7fca440b43c0 2024-04-26T20:58:00.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.909+0000 7fca41600640 1 --2- 172.21.15.28:0/693835475 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fca2006c940 0x7fca2006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:00.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.909+0000 7fca41600640 1 -- 172.21.15.28:0/693835475 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4943+0+0 (secure 0 0 0) 0x7fca3408b160 con 0x7fca440b43c0 2024-04-26T20:58:00.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.910+0000 7fca3be00640 1 --2- 172.21.15.28:0/693835475 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fca2006c940 0x7fca2006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:00.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.910+0000 7fca3be00640 1 --2- 172.21.15.28:0/693835475 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fca2006c940 0x7fca2006ee00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fca2c034000 tx=0x7fca2c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:00.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:00.912+0000 7fca41600640 1 -- 172.21.15.28:0/693835475 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca34021020 con 0x7fca440b43c0 2024-04-26T20:58:01.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.130+0000 7fca4b600640 1 -- 172.21.15.28:0/693835475 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fca440d3220 con 0x7fca440b43c0 2024-04-26T20:58:01.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.627+0000 7fca41600640 1 -- 172.21.15.28:0/693835475 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) ==== 97+0+0 (secure 0 0 0) 0x7fca340597e0 con 0x7fca440b43c0 2024-04-26T20:58:01.628 INFO:teuthology.orchestra.run.smithi028.stderr: removed all blocklist entries 2024-04-26T20:58:01.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.630+0000 7fca4b600640 1 -- 172.21.15.28:0/693835475 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fca2006c940 msgr2=0x7fca2006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:01.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.630+0000 7fca4b600640 1 --2- 172.21.15.28:0/693835475 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fca2006c940 0x7fca2006ee00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fca2c034000 tx=0x7fca2c02e000 comp rx=0 tx=0).stop 2024-04-26T20:58:01.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.630+0000 7fca4b600640 1 -- 172.21.15.28:0/693835475 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca440b43c0 msgr2=0x7fca44159830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:01.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.630+0000 7fca4b600640 1 --2- 172.21.15.28:0/693835475 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca440b43c0 0x7fca44159830 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fca3400c400 tx=0x7fca3400c8c0 comp rx=0 tx=0).stop 2024-04-26T20:58:01.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.630+0000 7fca4b600640 1 -- 172.21.15.28:0/693835475 shutdown_connections 2024-04-26T20:58:01.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.630+0000 7fca4b600640 1 --2- 172.21.15.28:0/693835475 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fca2006c940 0x7fca2006ee00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:01.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.630+0000 7fca4b600640 1 --2- 172.21.15.28:0/693835475 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fca440d00f0 0x7fca441f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:01.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.630+0000 7fca4b600640 1 --2- 172.21.15.28:0/693835475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca440b6b10 0x7fca4415ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:01.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.631+0000 7fca4b600640 1 --2- 172.21.15.28:0/693835475 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca440b43c0 0x7fca44159830 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:01.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.631+0000 7fca4b600640 1 -- 172.21.15.28:0/693835475 >> 172.21.15.28:0/693835475 conn(0x7fca440d1b10 msgr2=0x7fca440b6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:01.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.631+0000 7fca4b600640 1 -- 172.21.15.28:0/693835475 shutdown_connections 2024-04-26T20:58:01.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.631+0000 7fca4b600640 1 -- 172.21.15.28:0/693835475 wait complete. 2024-04-26T20:58:01.655 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T20:58:01.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.862+0000 7fb2bbe00640 1 Processor -- start 2024-04-26T20:58:01.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.862+0000 7fb2bbe00640 1 -- start start 2024-04-26T20:58:01.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.864+0000 7fb2bbe00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fb2bc15aab0 0x7fb2bc0cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:01.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.864+0000 7fb2bbe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb2bc073d10 0x7fb2bc160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:01.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.864+0000 7fb2bbe00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2bc161020 0x7fb2bc15d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:01.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.864+0000 7fb2bbe00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb2bc0cfa40 con 0x7fb2bc073d10 2024-04-26T20:58:01.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.864+0000 7fb2bbe00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fb2bc072040 con 0x7fb2bc15aab0 2024-04-26T20:58:01.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.864+0000 7fb2bbe00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb2bc161450 con 0x7fb2bc161020 2024-04-26T20:58:01.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.864+0000 7fb2b3e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb2bc073d10 0x7fb2bc160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:01.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.864+0000 7fb2b3e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb2bc073d10 0x7fb2bc160ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60914/0 (socket says 172.21.15.28:60914) 2024-04-26T20:58:01.866 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.864+0000 7fb2b3e00640 1 -- 172.21.15.28:0/791033868 learned_addr learned my addr 172.21.15.28:0/791033868 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:01.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.864+0000 7fb2baa00640 1 --2- 172.21.15.28:0/791033868 >> v2:172.21.15.103:3300/0 conn(0x7fb2bc15aab0 0x7fb2bc0cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:01.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.864+0000 7fb2bb400640 1 --2- 172.21.15.28:0/791033868 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2bc161020 0x7fb2bc15d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:01.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.864+0000 7fb2b3e00640 1 -- 172.21.15.28:0/791033868 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2bc161020 msgr2=0x7fb2bc15d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:01.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.865+0000 7fb2b3e00640 1 --2- 172.21.15.28:0/791033868 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2bc161020 0x7fb2bc15d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:01.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.865+0000 7fb2b3e00640 1 -- 172.21.15.28:0/791033868 >> v2:172.21.15.103:3300/0 conn(0x7fb2bc15aab0 msgr2=0x7fb2bc0cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:01.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.865+0000 7fb2b3e00640 1 --2- 172.21.15.28:0/791033868 >> v2:172.21.15.103:3300/0 conn(0x7fb2bc15aab0 0x7fb2bc0cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:01.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.865+0000 7fb2b3e00640 1 -- 172.21.15.28:0/791033868 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc15ae90 con 0x7fb2bc073d10 2024-04-26T20:58:01.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.865+0000 7fb2bb400640 1 --2- 172.21.15.28:0/791033868 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2bc161020 0x7fb2bc15d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:01.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.865+0000 7fb2baa00640 1 --2- 172.21.15.28:0/791033868 >> v2:172.21.15.103:3300/0 conn(0x7fb2bc15aab0 0x7fb2bc0cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:01.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.866+0000 7fb2b3e00640 1 --2- 172.21.15.28:0/791033868 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2bc073d10 0x7fb2bc160ae0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc076000 tx=0x7fb2a402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54f4e6046080288b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:01.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.866+0000 7fb2ba000640 1 -- 172.21.15.28:0/791033868 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4037070 con 0x7fb2bc073d10 2024-04-26T20:58:01.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.866+0000 7fb2ba000640 1 -- 172.21.15.28:0/791033868 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2a4032440 con 0x7fb2bc073d10 2024-04-26T20:58:01.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.866+0000 7fb2ba000640 1 -- 172.21.15.28:0/791033868 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4032810 con 0x7fb2bc073d10 2024-04-26T20:58:01.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.867+0000 7fb2bbe00640 1 -- 172.21.15.28:0/791033868 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2bc073d10 msgr2=0x7fb2bc160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:01.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.867+0000 7fb2bbe00640 1 --2- 172.21.15.28:0/791033868 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2bc073d10 0x7fb2bc160ae0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc076000 tx=0x7fb2a402a140 comp rx=0 tx=0).stop 2024-04-26T20:58:01.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.867+0000 7fb2bbe00640 1 -- 172.21.15.28:0/791033868 shutdown_connections 2024-04-26T20:58:01.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.867+0000 7fb2bbe00640 1 --2- 172.21.15.28:0/791033868 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2bc161020 0x7fb2bc15d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:01.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.867+0000 7fb2bbe00640 1 --2- 172.21.15.28:0/791033868 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2bc073d10 0x7fb2bc160ae0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:01.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.867+0000 7fb2bbe00640 1 --2- 172.21.15.28:0/791033868 >> v2:172.21.15.103:3300/0 conn(0x7fb2bc15aab0 0x7fb2bc0cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:01.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.867+0000 7fb2bbe00640 1 -- 172.21.15.28:0/791033868 >> 172.21.15.28:0/791033868 conn(0x7fb2bc0b2720 msgr2=0x7fb2bc0b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:01.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.867+0000 7fb2bbe00640 1 -- 172.21.15.28:0/791033868 shutdown_connections 2024-04-26T20:58:01.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.867+0000 7fb2bbe00640 1 -- 172.21.15.28:0/791033868 wait complete. 2024-04-26T20:58:01.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.869+0000 7fb2bbe00640 1 Processor -- start 2024-04-26T20:58:01.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.869+0000 7fb2bbe00640 1 -- start start 2024-04-26T20:58:01.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.870+0000 7fb2bbe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2bc073d10 0x7fb2bc142de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:01.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.870+0000 7fb2bbe00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2bc15aab0 0x7fb2bc159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:01.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.870+0000 7fb2bbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2bc161020 0x7fb2bc13fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:01.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.870+0000 7fb2bbe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc074180 con 0x7fb2bc161020 2024-04-26T20:58:01.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.870+0000 7fb2bbe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb2bc0cff70 con 0x7fb2bc073d10 2024-04-26T20:58:01.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.870+0000 7fb2bbe00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb2bc06df00 con 0x7fb2bc15aab0 2024-04-26T20:58:01.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.870+0000 7fb2bb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2bc161020 0x7fb2bc13fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:01.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.870+0000 7fb2bb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2bc161020 0x7fb2bc13fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60922/0 (socket says 172.21.15.28:60922) 2024-04-26T20:58:01.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.870+0000 7fb2bb400640 1 -- 172.21.15.28:0/351524719 learned_addr learned my addr 172.21.15.28:0/351524719 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:01.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.870+0000 7fb2baa00640 1 --2- 172.21.15.28:0/351524719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2bc073d10 0x7fb2bc142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:01.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.870+0000 7fb2b3e00640 1 --2- 172.21.15.28:0/351524719 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2bc15aab0 0x7fb2bc159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:01.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.871+0000 7fb2bb400640 1 -- 172.21.15.28:0/351524719 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2bc15aab0 msgr2=0x7fb2bc159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:01.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.871+0000 7fb2bb400640 1 --2- 172.21.15.28:0/351524719 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2bc15aab0 0x7fb2bc159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:01.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.871+0000 7fb2bb400640 1 -- 172.21.15.28:0/351524719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2bc073d10 msgr2=0x7fb2bc142de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:01.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.871+0000 7fb2bb400640 1 --2- 172.21.15.28:0/351524719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2bc073d10 0x7fb2bc142de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:01.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.871+0000 7fb2bb400640 1 -- 172.21.15.28:0/351524719 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2bc141d80 con 0x7fb2bc161020 2024-04-26T20:58:01.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.871+0000 7fb2bb400640 1 --2- 172.21.15.28:0/351524719 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2bc161020 0x7fb2bc13fc10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400cd50 tx=0x7fb2b400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:01.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.872+0000 7fb2b2a00640 1 -- 172.21.15.28:0/351524719 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b400f680 con 0x7fb2bc161020 2024-04-26T20:58:01.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.872+0000 7fb2b2a00640 1 -- 172.21.15.28:0/351524719 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2b400f840 con 0x7fb2bc161020 2024-04-26T20:58:01.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.872+0000 7fb2b2a00640 1 -- 172.21.15.28:0/351524719 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b400fc30 con 0x7fb2bc161020 2024-04-26T20:58:01.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.872+0000 7fb2bbe00640 1 -- 172.21.15.28:0/351524719 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2bc142020 con 0x7fb2bc161020 2024-04-26T20:58:01.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.872+0000 7fb2bbe00640 1 -- 172.21.15.28:0/351524719 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2bc1425e0 con 0x7fb2bc161020 2024-04-26T20:58:01.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.874+0000 7fb2bbe00640 1 -- 172.21.15.28:0/351524719 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2bc06a360 con 0x7fb2bc161020 2024-04-26T20:58:01.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.874+0000 7fb2b2a00640 1 -- 172.21.15.28:0/351524719 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb2b400e3e0 con 0x7fb2bc161020 2024-04-26T20:58:01.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.875+0000 7fb2b2a00640 1 --2- 172.21.15.28:0/351524719 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb29806c8f0 0x7fb29806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:01.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.875+0000 7fb2b2a00640 1 -- 172.21.15.28:0/351524719 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fb2b408c4f0 con 0x7fb2bc161020 2024-04-26T20:58:01.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.876+0000 7fb2baa00640 1 --2- 172.21.15.28:0/351524719 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb29806c8f0 0x7fb29806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:01.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.876+0000 7fb2baa00640 1 --2- 172.21.15.28:0/351524719 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb29806c8f0 0x7fb29806edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac006b10 tx=0x7fb2ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:01.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:01.880+0000 7fb2b2a00640 1 -- 172.21.15.28:0/351524719 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb2b4014030 con 0x7fb2bc161020 2024-04-26T20:58:02.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.162+0000 7fb2bbe00640 1 -- 172.21.15.28:0/351524719 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb2bc06fc80 con 0x7fb2bc161020 2024-04-26T20:58:02.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.163+0000 7fb2b2a00640 1 -- 172.21.15.28:0/351524719 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7fb2b405ad00 con 0x7fb2bc161020 2024-04-26T20:58:02.164 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:02.164 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T20:58:02.165 INFO:teuthology.orchestra.run.smithi028.stdout: "auth_dump": [ 2024-04-26T20:58:02.165 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.165 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.a", 2024-04-26T20:58:02.165 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDUFCxm570OOxAAwJmAfkKZtNjHl5XzgYUjSQ==", 2024-04-26T20:58:02.165 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.165 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-04-26T20:58:02.165 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-04-26T20:58:02.165 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-04-26T20:58:02.165 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:02.165 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.165 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.165 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.165 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.b", 2024-04-26T20:58:02.166 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDVFCxm5oQyFhAAhrxIFQkOub2iuwfXwrSi4g==", 2024-04-26T20:58:02.166 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.166 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-04-26T20:58:02.166 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-04-26T20:58:02.166 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-04-26T20:58:02.166 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:02.166 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.166 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.166 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.166 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.c", 2024-04-26T20:58:02.166 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDVFCxmpDSsChAAYAKo1Jj6ExMaDgsWl0NT3g==", 2024-04-26T20:58:02.166 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.167 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-04-26T20:58:02.167 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-04-26T20:58:02.167 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-04-26T20:58:02.167 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:02.167 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.167 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.167 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.167 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.d", 2024-04-26T20:58:02.167 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDVFCxm9YPLIhAAbgtxdShF5uQEQMCabsUY0A==", 2024-04-26T20:58:02.167 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.167 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-04-26T20:58:02.167 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-04-26T20:58:02.167 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-04-26T20:58:02.167 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:02.168 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.168 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.168 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.168 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.0", 2024-04-26T20:58:02.168 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDXFCxmfzl7NRAAGayH8lsv4xFqpTC/xS/l3A==", 2024-04-26T20:58:02.168 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.168 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-04-26T20:58:02.168 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-04-26T20:58:02.168 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:02.168 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.168 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.168 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.168 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.1", 2024-04-26T20:58:02.168 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDbFCxmkZiIFRAAMBKG5zWUKbjum+GOHLr4Cg==", 2024-04-26T20:58:02.168 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.168 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-04-26T20:58:02.169 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-04-26T20:58:02.169 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:02.169 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.169 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.169 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.169 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.2", 2024-04-26T20:58:02.169 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDeFCxmKVSqIRAAawTDWXhcZjzrq2dswt+H2g==", 2024-04-26T20:58:02.169 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.169 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-04-26T20:58:02.169 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-04-26T20:58:02.169 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:02.169 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.169 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.169 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.169 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.3", 2024-04-26T20:58:02.170 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDhFCxmtKLGOhAAqvk9Jg+vsKnIgems1e8Ytg==", 2024-04-26T20:58:02.170 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.170 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-04-26T20:58:02.170 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-04-26T20:58:02.170 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:02.170 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.170 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.170 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.170 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.4", 2024-04-26T20:58:02.170 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDoFCxmwx9WChAA1zzH0x5gZXgcrVUQey4dwQ==", 2024-04-26T20:58:02.170 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.170 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-04-26T20:58:02.170 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-04-26T20:58:02.171 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:02.171 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.171 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.171 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.171 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.5", 2024-04-26T20:58:02.171 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDrFCxmsGDzJBAAnhFkDmONidO/0RgaVFbDGg==", 2024-04-26T20:58:02.171 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.171 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-04-26T20:58:02.171 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-04-26T20:58:02.171 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:02.171 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.171 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.171 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.171 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.6", 2024-04-26T20:58:02.171 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDuFCxmkj9/LxAA+8IvdhuCnsvEM56+y9stnQ==", 2024-04-26T20:58:02.171 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.172 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-04-26T20:58:02.172 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-04-26T20:58:02.172 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:02.172 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.172 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.172 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.172 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.7", 2024-04-26T20:58:02.172 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDxFCxmvsb6OhAA50znn/4LbyZqxI1xgMJIEw==", 2024-04-26T20:58:02.172 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.172 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-04-26T20:58:02.172 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-04-26T20:58:02.172 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:02.172 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.172 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.172 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.173 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.0", 2024-04-26T20:58:02.173 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDVFCxm/rQRLhAAGsnaSU0YseRGCeDvxvvp/A==", 2024-04-26T20:58:02.173 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.173 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-04-26T20:58:02.173 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-04-26T20:58:02.173 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-04-26T20:58:02.173 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-04-26T20:58:02.173 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.173 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.173 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.173 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.1", 2024-04-26T20:58:02.173 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDWFCxmFhfoAhAA3JGoo80M5CvuNjJyFg3akw==", 2024-04-26T20:58:02.173 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.173 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-04-26T20:58:02.173 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-04-26T20:58:02.174 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-04-26T20:58:02.174 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-04-26T20:58:02.174 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.174 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.174 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.174 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.2", 2024-04-26T20:58:02.174 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDVFCxm+ntKNRAABSG+8Hggkp8p4fQhi9UudA==", 2024-04-26T20:58:02.174 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.174 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-04-26T20:58:02.174 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-04-26T20:58:02.174 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-04-26T20:58:02.174 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-04-26T20:58:02.174 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.174 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.175 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.175 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.3", 2024-04-26T20:58:02.175 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDWFCxmur+LCxAAqUus6BuHeNXG2R5zM2j00A==", 2024-04-26T20:58:02.175 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.175 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-04-26T20:58:02.175 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-04-26T20:58:02.175 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-04-26T20:58:02.175 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-04-26T20:58:02.175 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.175 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.175 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.175 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.admin", 2024-04-26T20:58:02.175 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDUFCxmHtVlAxAAd3pFr6eAayMMfaQSnXoTwg==", 2024-04-26T20:58:02.175 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.175 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-04-26T20:58:02.175 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-04-26T20:58:02.176 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-04-26T20:58:02.176 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:02.176 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.176 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.176 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.176 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.bootstrap-mds", 2024-04-26T20:58:02.176 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQD/FCxmTuNzGhAADFpymEBtepT0ssnIElTouQ==", 2024-04-26T20:58:02.176 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.176 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile bootstrap-mds" 2024-04-26T20:58:02.176 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.176 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.176 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.176 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.bootstrap-mgr", 2024-04-26T20:58:02.176 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQD/FCxmAF90GhAAAZ/IQH3u4I05XZ6zf/t/rg==", 2024-04-26T20:58:02.177 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.177 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-26T20:58:02.177 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.177 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.177 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.177 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.bootstrap-osd", 2024-04-26T20:58:02.177 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQD/FCxmH3p0GhAAVgW9vwQeaCwMDk0FaJCiXg==", 2024-04-26T20:58:02.177 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.177 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile bootstrap-osd" 2024-04-26T20:58:02.177 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.177 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.177 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.177 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.bootstrap-rbd", 2024-04-26T20:58:02.177 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQD/FCxm3ZR0GhAAFq08GnS4cc4cJuiHfkiEUA==", 2024-04-26T20:58:02.178 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.178 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-26T20:58:02.178 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.178 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.178 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.178 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-26T20:58:02.178 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQD/FCxmA7B0GhAALogM0M2er+u1YmWOTmKlqg==", 2024-04-26T20:58:02.178 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.178 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-26T20:58:02.178 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.178 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.178 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.178 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.bootstrap-rgw", 2024-04-26T20:58:02.179 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQD/FCxm7st0GhAAWBy73ha5tDc2lSwrnReZgg==", 2024-04-26T20:58:02.179 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.179 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-26T20:58:02.179 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.179 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.179 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.179 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.x", 2024-04-26T20:58:02.179 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDUFCxmh49VMxAAYQbMibMZBy6hsiO53AZYXw==", 2024-04-26T20:58:02.179 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.179 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-04-26T20:58:02.179 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2024-04-26T20:58:02.179 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:02.179 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.179 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:02.179 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:02.180 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.y", 2024-04-26T20:58:02.180 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDUFCxmod8TKxAA5myHdFdSXxwOb3F4fMcyiA==", 2024-04-26T20:58:02.180 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:02.180 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-04-26T20:58:02.180 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2024-04-26T20:58:02.180 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:02.180 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.180 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:02.180 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T20:58:02.180 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T20:58:02.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.165+0000 7fb2bbe00640 1 -- 172.21.15.28:0/351524719 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb29806c8f0 msgr2=0x7fb29806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:02.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.165+0000 7fb2bbe00640 1 --2- 172.21.15.28:0/351524719 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb29806c8f0 0x7fb29806edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac006b10 tx=0x7fb2ac008040 comp rx=0 tx=0).stop 2024-04-26T20:58:02.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.165+0000 7fb2bbe00640 1 -- 172.21.15.28:0/351524719 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2bc161020 msgr2=0x7fb2bc13fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:02.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.165+0000 7fb2bbe00640 1 --2- 172.21.15.28:0/351524719 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2bc161020 0x7fb2bc13fc10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fb2b400cd50 tx=0x7fb2b400a650 comp rx=0 tx=0).stop 2024-04-26T20:58:02.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.166+0000 7fb2bbe00640 1 -- 172.21.15.28:0/351524719 shutdown_connections 2024-04-26T20:58:02.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.166+0000 7fb2bbe00640 1 --2- 172.21.15.28:0/351524719 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb29806c8f0 0x7fb29806edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:02.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.166+0000 7fb2bbe00640 1 --2- 172.21.15.28:0/351524719 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2bc161020 0x7fb2bc13fc10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:02.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.166+0000 7fb2bbe00640 1 --2- 172.21.15.28:0/351524719 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2bc15aab0 0x7fb2bc159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:02.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.166+0000 7fb2bbe00640 1 --2- 172.21.15.28:0/351524719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb2bc073d10 0x7fb2bc142de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:02.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.166+0000 7fb2bbe00640 1 -- 172.21.15.28:0/351524719 >> 172.21.15.28:0/351524719 conn(0x7fb2bc0b2720 msgr2=0x7fb2bc073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:02.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.166+0000 7fb2bbe00640 1 -- 172.21.15.28:0/351524719 shutdown_connections 2024-04-26T20:58:02.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.166+0000 7fb2bbe00640 1 -- 172.21.15.28:0/351524719 wait complete. 2024-04-26T20:58:02.186 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-26T20:58:02.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.390+0000 7f2b43400640 1 Processor -- start 2024-04-26T20:58:02.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.391+0000 7f2b43400640 1 -- start start 2024-04-26T20:58:02.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.392+0000 7f2b43400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2b3c147c90 0x7f2b3c06e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:02.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.392+0000 7f2b43400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b3c0b5480 0x7f2b3c0b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:02.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.392+0000 7f2b43400640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b3c074390 con 0x7f2b3c073160 2024-04-26T20:58:02.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.392+0000 7f2b43400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f2b3c0b5e20 con 0x7f2b3c147c90 2024-04-26T20:58:02.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.392+0000 7f2b43400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f2b3c073dd0 con 0x7f2b3c0b5480 2024-04-26T20:58:02.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.392+0000 7f2b40a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2b3c147c90 0x7f2b3c06e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:02.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.393+0000 7f2b40a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2b3c147c90 0x7f2b3c06e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60926/0 (socket says 172.21.15.28:60926) 2024-04-26T20:58:02.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.393+0000 7f2b40a00640 1 -- 172.21.15.28:0/2382619923 learned_addr learned my addr 172.21.15.28:0/2382619923 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:02.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.393+0000 7f2b3be00640 1 --2- 172.21.15.28:0/2382619923 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b3c0b5480 0x7f2b3c0b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:02.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.393+0000 7f2b40a00640 1 -- 172.21.15.28:0/2382619923 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b3c0b5480 msgr2=0x7f2b3c0b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:02.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.393+0000 7f2b40a00640 1 --2- 172.21.15.28:0/2382619923 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b3c0b5480 0x7f2b3c0b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:02.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.393+0000 7f2b40a00640 1 -- 172.21.15.28:0/2382619923 >> v1:172.21.15.103:6789/0 conn(0x7f2b3c073160 legacy=0x7f2b3c0735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:02.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.393+0000 7f2b40a00640 1 -- 172.21.15.28:0/2382619923 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3c0740b0 con 0x7f2b3c147c90 2024-04-26T20:58:02.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.393+0000 7f2b3be00640 1 --2- 172.21.15.28:0/2382619923 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b3c0b5480 0x7f2b3c0b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:02.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.394+0000 7f2b40a00640 1 --2- 172.21.15.28:0/2382619923 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b3c147c90 0x7f2b3c06e610 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2b30009870 tx=0x7f2b3002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1131c5b970916769 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:02.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.394+0000 7f2b3b400640 1 -- 172.21.15.28:0/2382619923 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b30037070 con 0x7f2b3c147c90 2024-04-26T20:58:02.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.395+0000 7f2b3b400640 1 -- 172.21.15.28:0/2382619923 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b30032440 con 0x7f2b3c147c90 2024-04-26T20:58:02.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.395+0000 7f2b3b400640 1 -- 172.21.15.28:0/2382619923 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b30032810 con 0x7f2b3c147c90 2024-04-26T20:58:02.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.395+0000 7f2b43400640 1 -- 172.21.15.28:0/2382619923 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b3c147c90 msgr2=0x7f2b3c06e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:02.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.395+0000 7f2b43400640 1 --2- 172.21.15.28:0/2382619923 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b3c147c90 0x7f2b3c06e610 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2b30009870 tx=0x7f2b3002a180 comp rx=0 tx=0).stop 2024-04-26T20:58:02.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.395+0000 7f2b43400640 1 -- 172.21.15.28:0/2382619923 shutdown_connections 2024-04-26T20:58:02.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.395+0000 7f2b43400640 1 --2- 172.21.15.28:0/2382619923 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b3c0b5480 0x7f2b3c0b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:02.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.395+0000 7f2b43400640 1 --2- 172.21.15.28:0/2382619923 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b3c147c90 0x7f2b3c06e610 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:02.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.395+0000 7f2b43400640 1 -- 172.21.15.28:0/2382619923 >> 172.21.15.28:0/2382619923 conn(0x7f2b3c06c770 msgr2=0x7f2b3c06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:02.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.395+0000 7f2b43400640 1 -- 172.21.15.28:0/2382619923 shutdown_connections 2024-04-26T20:58:02.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.395+0000 7f2b43400640 1 -- 172.21.15.28:0/2382619923 wait complete. 2024-04-26T20:58:02.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.397+0000 7f2b43400640 1 Processor -- start 2024-04-26T20:58:02.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.397+0000 7f2b43400640 1 -- start start 2024-04-26T20:58:02.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.398+0000 7f2b43400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b3c073160 0x7f2b3c13f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:02.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.398+0000 7f2b43400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b3c0b5480 0x7f2b3c0cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:02.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.398+0000 7f2b43400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b3c147c90 0x7f2b3c1622c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:02.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.398+0000 7f2b43400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c0af7b0 con 0x7f2b3c0b5480 2024-04-26T20:58:02.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.398+0000 7f2b43400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c06eb50 con 0x7f2b3c147c90 2024-04-26T20:58:02.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.398+0000 7f2b43400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f2b3c148070 con 0x7f2b3c073160 2024-04-26T20:58:02.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.398+0000 7f2b3be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b3c0b5480 0x7f2b3c0cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:02.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.398+0000 7f2b3be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b3c0b5480 0x7f2b3c0cf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60938/0 (socket says 172.21.15.28:60938) 2024-04-26T20:58:02.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.398+0000 7f2b3be00640 1 -- 172.21.15.28:0/2738174191 learned_addr learned my addr 172.21.15.28:0/2738174191 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:02.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.399+0000 7f2b41400640 1 --2- 172.21.15.28:0/2738174191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b3c147c90 0x7f2b3c1622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:02.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.399+0000 7f2b40a00640 1 --2- 172.21.15.28:0/2738174191 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b3c073160 0x7f2b3c13f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:02.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.399+0000 7f2b3be00640 1 -- 172.21.15.28:0/2738174191 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b3c073160 msgr2=0x7f2b3c13f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:02.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.399+0000 7f2b3be00640 1 --2- 172.21.15.28:0/2738174191 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b3c073160 0x7f2b3c13f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:02.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.399+0000 7f2b3be00640 1 -- 172.21.15.28:0/2738174191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b3c147c90 msgr2=0x7f2b3c1622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:02.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.399+0000 7f2b3be00640 1 --2- 172.21.15.28:0/2738174191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b3c147c90 0x7f2b3c1622c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:02.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.399+0000 7f2b3be00640 1 -- 172.21.15.28:0/2738174191 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3c0cf810 con 0x7f2b3c0b5480 2024-04-26T20:58:02.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.399+0000 7f2b3be00640 1 --2- 172.21.15.28:0/2738174191 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b3c0b5480 0x7f2b3c0cf1c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800c820 tx=0x7f2b2800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:02.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.400+0000 7f2b38c00640 1 -- 172.21.15.28:0/2738174191 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b28015070 con 0x7f2b3c0b5480 2024-04-26T20:58:02.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.400+0000 7f2b38c00640 1 -- 172.21.15.28:0/2738174191 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b28007a30 con 0x7f2b3c0b5480 2024-04-26T20:58:02.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.400+0000 7f2b38c00640 1 -- 172.21.15.28:0/2738174191 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b28007de0 con 0x7f2b3c0b5480 2024-04-26T20:58:02.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.400+0000 7f2b43400640 1 -- 172.21.15.28:0/2738174191 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b3c0cfaf0 con 0x7f2b3c0b5480 2024-04-26T20:58:02.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.400+0000 7f2b43400640 1 -- 172.21.15.28:0/2738174191 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b3c0d00b0 con 0x7f2b3c0b5480 2024-04-26T20:58:02.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.401+0000 7f2b43400640 1 -- 172.21.15.28:0/2738174191 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b3c06a400 con 0x7f2b3c0b5480 2024-04-26T20:58:02.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.402+0000 7f2b38c00640 1 -- 172.21.15.28:0/2738174191 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b28018070 con 0x7f2b3c0b5480 2024-04-26T20:58:02.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.403+0000 7f2b38c00640 1 --2- 172.21.15.28:0/2738174191 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2b1c06c920 0x7f2b1c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:02.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.403+0000 7f2b38c00640 1 -- 172.21.15.28:0/2738174191 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f2b2808b680 con 0x7f2b3c0b5480 2024-04-26T20:58:02.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.403+0000 7f2b40a00640 1 --2- 172.21.15.28:0/2738174191 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2b1c06c920 0x7f2b1c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:02.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.404+0000 7f2b40a00640 1 --2- 172.21.15.28:0/2738174191 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2b1c06c920 0x7f2b1c06ede0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2b30034000 tx=0x7f2b3002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:02.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.407+0000 7f2b38c00640 1 -- 172.21.15.28:0/2738174191 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2b28059df0 con 0x7f2b3c0b5480 2024-04-26T20:58:02.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.689+0000 7f2b43400640 1 -- 172.21.15.28:0/2738174191 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f2b3c0d0920 con 0x7f2b3c0b5480 2024-04-26T20:58:02.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.693+0000 7f2b38c00640 1 -- 172.21.15.28:0/2738174191 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7f2b280596f0 con 0x7f2b3c0b5480 2024-04-26T20:58:02.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.696+0000 7f2b43400640 1 -- 172.21.15.28:0/2738174191 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2b1c06c920 msgr2=0x7f2b1c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:02.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.696+0000 7f2b43400640 1 --2- 172.21.15.28:0/2738174191 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2b1c06c920 0x7f2b1c06ede0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2b30034000 tx=0x7f2b3002e000 comp rx=0 tx=0).stop 2024-04-26T20:58:02.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.696+0000 7f2b43400640 1 -- 172.21.15.28:0/2738174191 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b3c0b5480 msgr2=0x7f2b3c0cf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:02.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.696+0000 7f2b43400640 1 --2- 172.21.15.28:0/2738174191 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b3c0b5480 0x7f2b3c0cf1c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800c820 tx=0x7f2b2800cce0 comp rx=0 tx=0).stop 2024-04-26T20:58:02.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.696+0000 7f2b43400640 1 -- 172.21.15.28:0/2738174191 shutdown_connections 2024-04-26T20:58:02.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.696+0000 7f2b43400640 1 --2- 172.21.15.28:0/2738174191 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2b1c06c920 0x7f2b1c06ede0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:02.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.696+0000 7f2b43400640 1 --2- 172.21.15.28:0/2738174191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2b3c147c90 0x7f2b3c1622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:02.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.696+0000 7f2b43400640 1 --2- 172.21.15.28:0/2738174191 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b3c0b5480 0x7f2b3c0cf1c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:02.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.696+0000 7f2b43400640 1 --2- 172.21.15.28:0/2738174191 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2b3c073160 0x7f2b3c13f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:02.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.696+0000 7f2b43400640 1 -- 172.21.15.28:0/2738174191 >> 172.21.15.28:0/2738174191 conn(0x7f2b3c06c770 msgr2=0x7f2b3c0b46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:02.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.696+0000 7f2b43400640 1 -- 172.21.15.28:0/2738174191 shutdown_connections 2024-04-26T20:58:02.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.696+0000 7f2b43400640 1 -- 172.21.15.28:0/2738174191 wait complete. 2024-04-26T20:58:02.715 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-26T20:58:02.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.921+0000 7fa3e6e00640 1 Processor -- start 2024-04-26T20:58:02.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.921+0000 7fa3e6e00640 1 -- start start 2024-04-26T20:58:02.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.923+0000 7fa3e6e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa3e0073bf0 0x7fa3e006e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:02.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.923+0000 7fa3e6e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3e00721d0 con 0x7fa3e00737a0 2024-04-26T20:58:02.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.923+0000 7fa3e6e00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3e00724b0 con 0x7fa3e0156180 2024-04-26T20:58:02.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.923+0000 7fa3e6e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fa3e0072de0 con 0x7fa3e0073bf0 2024-04-26T20:58:02.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.923+0000 7fa3e5000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa3e00737a0 0x7fa3e006e110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36496/0 (socket says 172.21.15.28:36496) 2024-04-26T20:58:02.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.923+0000 7fa3e5000640 1 -- 172.21.15.28:0/2073691279 learned_addr learned my addr 172.21.15.28:0/2073691279 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:02.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.924+0000 7fa3dfe00640 1 -- 172.21.15.28:0/2073691279 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2887739878 0 0) 0x7fa3e00721d0 con 0x7fa3e00737a0 2024-04-26T20:58:02.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.924+0000 7fa3dfe00640 1 -- 172.21.15.28:0/2073691279 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3c8003520 con 0x7fa3e00737a0 2024-04-26T20:58:02.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.924+0000 7fa3e6400640 1 --2- 172.21.15.28:0/2073691279 >> v2:172.21.15.103:3300/0 conn(0x7fa3e0073bf0 0x7fa3e006e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:02.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.924+0000 7fa3dfe00640 1 -- 172.21.15.28:0/2073691279 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1149263449 0 0) 0x7fa3c8003520 con 0x7fa3e00737a0 2024-04-26T20:58:02.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.924+0000 7fa3dfe00640 1 -- 172.21.15.28:0/2073691279 >> v2:172.21.15.103:3300/0 conn(0x7fa3e0073bf0 msgr2=0x7fa3e006e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:02.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.924+0000 7fa3dfe00640 1 --2- 172.21.15.28:0/2073691279 >> v2:172.21.15.103:3300/0 conn(0x7fa3e0073bf0 0x7fa3e006e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:02.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.924+0000 7fa3dfe00640 1 -- 172.21.15.28:0/2073691279 >> v1:172.21.15.103:6789/0 conn(0x7fa3e0156180 legacy=0x7fa3e00b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:02.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.924+0000 7fa3dfe00640 1 -- 172.21.15.28:0/2073691279 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e0072790 con 0x7fa3e00737a0 2024-04-26T20:58:02.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.924+0000 7fa3dfe00640 1 -- 172.21.15.28:0/2073691279 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fa3cc001670 con 0x7fa3e00737a0 2024-04-26T20:58:02.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.924+0000 7fa3dfe00640 1 -- 172.21.15.28:0/2073691279 >> v1:172.21.15.28:6789/0 conn(0x7fa3e00737a0 legacy=0x7fa3e006e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:02.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.924+0000 7fa3dfe00640 1 --2- 172.21.15.28:0/2073691279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3c8003a00 0x7fa3c8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:02.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.924+0000 7fa3dfe00640 1 --2- 172.21.15.28:0/2073691279 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3c8004800 0x7fa3c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:02.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.924+0000 7fa3dfe00640 1 --2- 172.21.15.28:0/2073691279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3c8005210 0x7fa3c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:02.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.924+0000 7fa3dfe00640 1 -- 172.21.15.28:0/2073691279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e0072de0 con 0x7fa3c8005210 2024-04-26T20:58:02.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.924+0000 7fa3dfe00640 1 -- 172.21.15.28:0/2073691279 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c800c150 con 0x7fa3c8003a00 2024-04-26T20:58:02.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.924+0000 7fa3dfe00640 1 -- 172.21.15.28:0/2073691279 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c800c350 con 0x7fa3c8004800 2024-04-26T20:58:02.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.925+0000 7fa3e6400640 1 --2- 172.21.15.28:0/2073691279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3c8005210 0x7fa3c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:02.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.925+0000 7fa3e5a00640 1 --2- 172.21.15.28:0/2073691279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3c8003a00 0x7fa3c8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:02.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.925+0000 7fa3e5000640 1 --2- 172.21.15.28:0/2073691279 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3c8004800 0x7fa3c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:02.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.925+0000 7fa3e6400640 1 -- 172.21.15.28:0/2073691279 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3c8004800 msgr2=0x7fa3c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:02.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.925+0000 7fa3e6400640 1 --2- 172.21.15.28:0/2073691279 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3c8004800 0x7fa3c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:02.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.925+0000 7fa3e6400640 1 -- 172.21.15.28:0/2073691279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3c8003a00 msgr2=0x7fa3c8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:02.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.925+0000 7fa3e6400640 1 --2- 172.21.15.28:0/2073691279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3c8003a00 0x7fa3c8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:02.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.925+0000 7fa3e6400640 1 -- 172.21.15.28:0/2073691279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c8003520 con 0x7fa3c8005210 2024-04-26T20:58:02.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.926+0000 7fa3e6400640 1 -- 172.21.15.28:0/2073691279 reap_dead start 2024-04-26T20:58:02.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.926+0000 7fa3e6400640 1 --2- 172.21.15.28:0/2073691279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3c8005210 0x7fa3c800b9c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8007360 tx=0x7fa3c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca08d06ee95c53a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:02.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.926+0000 7fa3dfe00640 1 -- 172.21.15.28:0/2073691279 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8009be0 con 0x7fa3c8005210 2024-04-26T20:58:02.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.926+0000 7fa3dfe00640 1 -- 172.21.15.28:0/2073691279 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3d8003bd0 con 0x7fa3c8005210 2024-04-26T20:58:02.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.926+0000 7fa3dfe00640 1 -- 172.21.15.28:0/2073691279 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8004020 con 0x7fa3c8005210 2024-04-26T20:58:02.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.927+0000 7fa3e6e00640 1 -- 172.21.15.28:0/2073691279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3c8005210 msgr2=0x7fa3c800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:02.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.927+0000 7fa3e6e00640 1 --2- 172.21.15.28:0/2073691279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3c8005210 0x7fa3c800b9c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa3d8007360 tx=0x7fa3c8005830 comp rx=0 tx=0).stop 2024-04-26T20:58:02.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.927+0000 7fa3e6e00640 1 -- 172.21.15.28:0/2073691279 shutdown_connections 2024-04-26T20:58:02.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.927+0000 7fa3e6e00640 1 --2- 172.21.15.28:0/2073691279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3c8005210 0x7fa3c800b9c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:02.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.927+0000 7fa3e6e00640 1 -- 172.21.15.28:0/2073691279 >> 172.21.15.28:0/2073691279 conn(0x7fa3e00b2720 msgr2=0x7fa3e00b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:02.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.927+0000 7fa3e6e00640 1 -- 172.21.15.28:0/2073691279 shutdown_connections 2024-04-26T20:58:02.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.927+0000 7fa3e6e00640 1 -- 172.21.15.28:0/2073691279 wait complete. 2024-04-26T20:58:02.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.929+0000 7fa3e6e00640 1 Processor -- start 2024-04-26T20:58:02.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.929+0000 7fa3e6e00640 1 -- start start 2024-04-26T20:58:02.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.930+0000 7fa3e6e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3c8005210 0x7fa3e014c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:02.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.930+0000 7fa3e6e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3e014de30 0x7fa3e014a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:02.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.930+0000 7fa3e6e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3e015c750 0x7fa3e006e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:02.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.930+0000 7fa3e6e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e00b3f50 con 0x7fa3e014de30 2024-04-26T20:58:02.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.930+0000 7fa3e6e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e0074060 con 0x7fa3c8005210 2024-04-26T20:58:02.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.930+0000 7fa3e6e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa3e0073270 con 0x7fa3e015c750 2024-04-26T20:58:02.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.930+0000 7fa3e5000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3e014de30 0x7fa3e014a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:02.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.930+0000 7fa3e5000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3e014de30 0x7fa3e014a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60952/0 (socket says 172.21.15.28:60952) 2024-04-26T20:58:02.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.930+0000 7fa3e5000640 1 -- 172.21.15.28:0/909139427 learned_addr learned my addr 172.21.15.28:0/909139427 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:02.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.931+0000 7fa3e6400640 1 --2- 172.21.15.28:0/909139427 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3e015c750 0x7fa3e006e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:02.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.931+0000 7fa3e5000640 1 -- 172.21.15.28:0/909139427 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3e015c750 msgr2=0x7fa3e006e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:02.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.931+0000 7fa3e5a00640 1 --2- 172.21.15.28:0/909139427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3c8005210 0x7fa3e014c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:02.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.931+0000 7fa3e5000640 1 --2- 172.21.15.28:0/909139427 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3e015c750 0x7fa3e006e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:02.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.931+0000 7fa3e5000640 1 -- 172.21.15.28:0/909139427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3c8005210 msgr2=0x7fa3e014c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:02.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.931+0000 7fa3e5000640 1 --2- 172.21.15.28:0/909139427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3c8005210 0x7fa3e014c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:02.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.931+0000 7fa3e5000640 1 -- 172.21.15.28:0/909139427 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e014cd90 con 0x7fa3e014de30 2024-04-26T20:58:02.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.931+0000 7fa3e6400640 1 --2- 172.21.15.28:0/909139427 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3e015c750 0x7fa3e006e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:02.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.931+0000 7fa3e5000640 1 --2- 172.21.15.28:0/909139427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3e014de30 0x7fa3e014a0e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00e700 tx=0x7fa3cc00ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:02.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.932+0000 7fa3dd600640 1 -- 172.21.15.28:0/909139427 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00ee10 con 0x7fa3e014de30 2024-04-26T20:58:02.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.932+0000 7fa3dd600640 1 -- 172.21.15.28:0/909139427 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3cc018070 con 0x7fa3e014de30 2024-04-26T20:58:02.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.932+0000 7fa3dd600640 1 -- 172.21.15.28:0/909139427 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00cb10 con 0x7fa3e014de30 2024-04-26T20:58:02.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.932+0000 7fa3e6e00640 1 -- 172.21.15.28:0/909139427 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e014d070 con 0x7fa3e014de30 2024-04-26T20:58:02.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.932+0000 7fa3e6e00640 1 -- 172.21.15.28:0/909139427 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e014d630 con 0x7fa3e014de30 2024-04-26T20:58:02.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.933+0000 7fa3e6e00640 1 -- 172.21.15.28:0/909139427 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e006a360 con 0x7fa3e014de30 2024-04-26T20:58:02.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.934+0000 7fa3dd600640 1 -- 172.21.15.28:0/909139427 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3cc00ccb0 con 0x7fa3e014de30 2024-04-26T20:58:02.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.935+0000 7fa3dd600640 1 --2- 172.21.15.28:0/909139427 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa3c006c8f0 0x7fa3c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:02.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.935+0000 7fa3dd600640 1 -- 172.21.15.28:0/909139427 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fa3cc08b850 con 0x7fa3e014de30 2024-04-26T20:58:02.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.935+0000 7fa3e5a00640 1 --2- 172.21.15.28:0/909139427 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa3c006c8f0 0x7fa3c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:02.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.936+0000 7fa3e5a00640 1 --2- 172.21.15.28:0/909139427 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa3c006c8f0 0x7fa3c006edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40014d0 tx=0x7fa3d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:02.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:02.940+0000 7fa3dd600640 1 -- 172.21.15.28:0/909139427 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa3cc059ea0 con 0x7fa3e014de30 2024-04-26T20:58:03.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.212+0000 7fa3e6e00640 1 -- 172.21.15.28:0/909139427 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fa3e00b34b0 con 0x7fa3e014de30 2024-04-26T20:58:03.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.217+0000 7fa3dd600640 1 -- 172.21.15.28:0/909139427 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7fa3cc0597a0 con 0x7fa3e014de30 2024-04-26T20:58:03.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.219+0000 7fa3e6e00640 1 -- 172.21.15.28:0/909139427 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa3c006c8f0 msgr2=0x7fa3c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:03.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.219+0000 7fa3e6e00640 1 --2- 172.21.15.28:0/909139427 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa3c006c8f0 0x7fa3c006edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40014d0 tx=0x7fa3d4009040 comp rx=0 tx=0).stop 2024-04-26T20:58:03.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.219+0000 7fa3e6e00640 1 -- 172.21.15.28:0/909139427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3e014de30 msgr2=0x7fa3e014a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:03.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.219+0000 7fa3e6e00640 1 --2- 172.21.15.28:0/909139427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3e014de30 0x7fa3e014a0e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00e700 tx=0x7fa3cc00ebc0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.220+0000 7fa3e6e00640 1 -- 172.21.15.28:0/909139427 shutdown_connections 2024-04-26T20:58:03.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.220+0000 7fa3e6e00640 1 --2- 172.21.15.28:0/909139427 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa3c006c8f0 0x7fa3c006edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.220+0000 7fa3e6e00640 1 --2- 172.21.15.28:0/909139427 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3e015c750 0x7fa3e006e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.220+0000 7fa3e6e00640 1 --2- 172.21.15.28:0/909139427 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3e014de30 0x7fa3e014a0e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.220+0000 7fa3e6e00640 1 --2- 172.21.15.28:0/909139427 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3c8005210 0x7fa3e014c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.220+0000 7fa3e6e00640 1 -- 172.21.15.28:0/909139427 >> 172.21.15.28:0/909139427 conn(0x7fa3e00b2720 msgr2=0x7fa3e0162770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:03.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.220+0000 7fa3e6e00640 1 -- 172.21.15.28:0/909139427 shutdown_connections 2024-04-26T20:58:03.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.220+0000 7fa3e6e00640 1 -- 172.21.15.28:0/909139427 wait complete. 2024-04-26T20:58:03.238 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-26T20:58:03.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.441+0000 7f3986800640 1 Processor -- start 2024-04-26T20:58:03.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.441+0000 7f3986800640 1 -- start start 2024-04-26T20:58:03.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.443+0000 7f3986800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f398015aab0 0x7f39800cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:03.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.443+0000 7f3986800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3980073d10 0x7f3980160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:03.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.443+0000 7f3986800640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3980161430 con 0x7f3980161020 2024-04-26T20:58:03.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.443+0000 7f3986800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f39800cfa40 con 0x7f3980073d10 2024-04-26T20:58:03.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.443+0000 7f3986800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f3980072040 con 0x7f398015aab0 2024-04-26T20:58:03.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.443+0000 7f3984a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3980073d10 0x7f3980160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:03.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.444+0000 7f3984a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3980073d10 0x7f3980160ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60960/0 (socket says 172.21.15.28:60960) 2024-04-26T20:58:03.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.444+0000 7f3984a00640 1 -- 172.21.15.28:0/4236385316 learned_addr learned my addr 172.21.15.28:0/4236385316 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:03.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.444+0000 7f3985400640 1 --2- 172.21.15.28:0/4236385316 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f398015aab0 0x7f39800cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:03.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.444+0000 7f3984a00640 1 -- 172.21.15.28:0/4236385316 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f398015aab0 msgr2=0x7f39800cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:03.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.444+0000 7f3984a00640 1 --2- 172.21.15.28:0/4236385316 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f398015aab0 0x7f39800cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.444+0000 7f3984a00640 1 -- 172.21.15.28:0/4236385316 >> v1:172.21.15.103:6789/0 conn(0x7f3980161020 legacy=0x7f398015d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:03.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.444+0000 7f3984a00640 1 -- 172.21.15.28:0/4236385316 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39800734e0 con 0x7f3980073d10 2024-04-26T20:58:03.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.445+0000 7f3984a00640 1 --2- 172.21.15.28:0/4236385316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3980073d10 0x7f3980160ae0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f396c009870 tx=0x7f396c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f4edb6ee7423bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:03.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.445+0000 7f397fe00640 1 -- 172.21.15.28:0/4236385316 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c037070 con 0x7f3980073d10 2024-04-26T20:58:03.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.445+0000 7f397fe00640 1 -- 172.21.15.28:0/4236385316 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f396c032440 con 0x7f3980073d10 2024-04-26T20:58:03.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.445+0000 7f397fe00640 1 -- 172.21.15.28:0/4236385316 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c032810 con 0x7f3980073d10 2024-04-26T20:58:03.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.446+0000 7f3986800640 1 -- 172.21.15.28:0/4236385316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3980073d10 msgr2=0x7f3980160ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:03.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.446+0000 7f3986800640 1 --2- 172.21.15.28:0/4236385316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3980073d10 0x7f3980160ae0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f396c009870 tx=0x7f396c02a180 comp rx=0 tx=0).stop 2024-04-26T20:58:03.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.446+0000 7f3986800640 1 -- 172.21.15.28:0/4236385316 shutdown_connections 2024-04-26T20:58:03.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.446+0000 7f3986800640 1 --2- 172.21.15.28:0/4236385316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3980073d10 0x7f3980160ae0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.446+0000 7f3986800640 1 --2- 172.21.15.28:0/4236385316 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f398015aab0 0x7f39800cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.446+0000 7f3986800640 1 -- 172.21.15.28:0/4236385316 >> 172.21.15.28:0/4236385316 conn(0x7f39800b2720 msgr2=0x7f39800b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:03.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.446+0000 7f3986800640 1 -- 172.21.15.28:0/4236385316 shutdown_connections 2024-04-26T20:58:03.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.446+0000 7f3986800640 1 -- 172.21.15.28:0/4236385316 wait complete. 2024-04-26T20:58:03.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.448+0000 7f3986800640 1 Processor -- start 2024-04-26T20:58:03.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.448+0000 7f3986800640 1 -- start start 2024-04-26T20:58:03.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.450+0000 7f3986800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3980073d10 0x7f3980159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:03.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.450+0000 7f3986800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f398015aab0 0x7f39800b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:03.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.450+0000 7f3986800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3980161020 0x7f39800b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:03.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.450+0000 7f3986800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f39800cff70 con 0x7f3980161020 2024-04-26T20:58:03.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.450+0000 7f3986800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f398015ae90 con 0x7f3980073d10 2024-04-26T20:58:03.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.450+0000 7f3986800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f398006df00 con 0x7f398015aab0 2024-04-26T20:58:03.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.450+0000 7f3985e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3980161020 0x7f39800b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:03.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.450+0000 7f3985e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3980161020 0x7f39800b14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60966/0 (socket says 172.21.15.28:60966) 2024-04-26T20:58:03.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.450+0000 7f3985e00640 1 -- 172.21.15.28:0/750009521 learned_addr learned my addr 172.21.15.28:0/750009521 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:03.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.450+0000 7f3985e00640 1 -- 172.21.15.28:0/750009521 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f398015aab0 msgr2=0x7f39800b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:03.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.450+0000 7f3985e00640 1 --2- 172.21.15.28:0/750009521 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f398015aab0 0x7f39800b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.450+0000 7f3985e00640 1 -- 172.21.15.28:0/750009521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3980073d10 msgr2=0x7f3980159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:03.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.450+0000 7f3985e00640 1 --2- 172.21.15.28:0/750009521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3980073d10 0x7f3980159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.450+0000 7f3985e00640 1 -- 172.21.15.28:0/750009521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39800ae8c0 con 0x7f3980161020 2024-04-26T20:58:03.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.451+0000 7f3985e00640 1 --2- 172.21.15.28:0/750009521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3980161020 0x7f39800b14f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f397800b410 tx=0x7f397800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:03.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.451+0000 7f397d600640 1 -- 172.21.15.28:0/750009521 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978015030 con 0x7f3980161020 2024-04-26T20:58:03.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.451+0000 7f397d600640 1 -- 172.21.15.28:0/750009521 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3978004020 con 0x7f3980161020 2024-04-26T20:58:03.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.451+0000 7f397d600640 1 -- 172.21.15.28:0/750009521 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3978004410 con 0x7f3980161020 2024-04-26T20:58:03.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.451+0000 7f3986800640 1 -- 172.21.15.28:0/750009521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39800b5750 con 0x7f3980161020 2024-04-26T20:58:03.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.451+0000 7f3986800640 1 -- 172.21.15.28:0/750009521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39800aeb80 con 0x7f3980161020 2024-04-26T20:58:03.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.452+0000 7f3986800640 1 -- 172.21.15.28:0/750009521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f398006a360 con 0x7f3980161020 2024-04-26T20:58:03.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.454+0000 7f397d600640 1 -- 172.21.15.28:0/750009521 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3978018070 con 0x7f3980161020 2024-04-26T20:58:03.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.454+0000 7f397d600640 1 --2- 172.21.15.28:0/750009521 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f396006c8f0 0x7f396006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:03.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.455+0000 7f397d600640 1 -- 172.21.15.28:0/750009521 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f397808b2d0 con 0x7f3980161020 2024-04-26T20:58:03.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.455+0000 7f3985400640 1 --2- 172.21.15.28:0/750009521 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f396006c8f0 0x7f396006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:03.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.455+0000 7f3985400640 1 --2- 172.21.15.28:0/750009521 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f396006c8f0 0x7f396006edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f39740070b0 tx=0x7f3974007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:03.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.459+0000 7f397d600640 1 -- 172.21.15.28:0/750009521 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3978059a40 con 0x7f3980161020 2024-04-26T20:58:03.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.738+0000 7f3986800640 1 -- 172.21.15.28:0/750009521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f398006fc80 con 0x7f3980161020 2024-04-26T20:58:03.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.742+0000 7f397d600640 1 -- 172.21.15.28:0/750009521 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f3978059340 con 0x7f3980161020 2024-04-26T20:58:03.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.744+0000 7f3986800640 1 -- 172.21.15.28:0/750009521 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f396006c8f0 msgr2=0x7f396006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:03.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.744+0000 7f3986800640 1 --2- 172.21.15.28:0/750009521 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f396006c8f0 0x7f396006edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f39740070b0 tx=0x7f3974007040 comp rx=0 tx=0).stop 2024-04-26T20:58:03.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.744+0000 7f3986800640 1 -- 172.21.15.28:0/750009521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3980161020 msgr2=0x7f39800b14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:03.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.744+0000 7f3986800640 1 --2- 172.21.15.28:0/750009521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3980161020 0x7f39800b14f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f397800b410 tx=0x7f397800b8d0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.744+0000 7f3986800640 1 -- 172.21.15.28:0/750009521 shutdown_connections 2024-04-26T20:58:03.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.744+0000 7f3986800640 1 --2- 172.21.15.28:0/750009521 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f396006c8f0 0x7f396006edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.744+0000 7f3986800640 1 --2- 172.21.15.28:0/750009521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3980161020 0x7f39800b14f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.744+0000 7f3986800640 1 --2- 172.21.15.28:0/750009521 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f398015aab0 0x7f39800b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.744+0000 7f3986800640 1 --2- 172.21.15.28:0/750009521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3980073d10 0x7f3980159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.744+0000 7f3986800640 1 -- 172.21.15.28:0/750009521 >> 172.21.15.28:0/750009521 conn(0x7f39800b2720 msgr2=0x7f3980145120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:03.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.745+0000 7f3986800640 1 -- 172.21.15.28:0/750009521 shutdown_connections 2024-04-26T20:58:03.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.745+0000 7f3986800640 1 -- 172.21.15.28:0/750009521 wait complete. 2024-04-26T20:58:03.764 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-26T20:58:03.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.959+0000 7f1ace600640 1 Processor -- start 2024-04-26T20:58:03.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.959+0000 7f1ace600640 1 -- start start 2024-04-26T20:58:03.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.961+0000 7f1ace600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1ac8147c90 0x7f1ac806e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:03.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.961+0000 7f1ace600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1ac80b5480 0x7f1ac80b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:03.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.961+0000 7f1ace600640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f1ac8073160 0x7f1ac80735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:03.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.961+0000 7f1ace600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f1ac80b5e20 con 0x7f1ac8147c90 2024-04-26T20:58:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.961+0000 7f1ace600640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f1ac8074390 con 0x7f1ac8073160 2024-04-26T20:58:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.961+0000 7f1ace600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac8073b10 con 0x7f1ac80b5480 2024-04-26T20:58:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.962+0000 7f1ac7400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1ac8147c90 0x7f1ac806e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.962+0000 7f1ac7400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1ac8147c90 0x7f1ac806e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60972/0 (socket says 172.21.15.28:60972) 2024-04-26T20:58:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.962+0000 7f1ac7400640 1 -- 172.21.15.28:0/1056145256 learned_addr learned my addr 172.21.15.28:0/1056145256 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.962+0000 7f1abfe00640 1 --2- 172.21.15.28:0/1056145256 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1ac80b5480 0x7f1ac80b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.962+0000 7f1ac7e00640 1 --2- 172.21.15.28:0/1056145256 >> v2:172.21.15.103:3300/0 conn(0x7f1ac8073160 0x7f1ac80735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.962+0000 7f1ac7400640 1 -- 172.21.15.28:0/1056145256 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1ac80b5480 msgr2=0x7f1ac80b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.962+0000 7f1ac7400640 1 --2- 172.21.15.28:0/1056145256 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1ac80b5480 0x7f1ac80b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.962+0000 7f1ac7400640 1 -- 172.21.15.28:0/1056145256 >> v2:172.21.15.103:3300/0 conn(0x7f1ac8073160 msgr2=0x7f1ac80735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.962+0000 7f1ac7400640 1 --2- 172.21.15.28:0/1056145256 >> v2:172.21.15.103:3300/0 conn(0x7f1ac8073160 0x7f1ac80735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.962+0000 7f1ac7400640 1 -- 172.21.15.28:0/1056145256 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac8073dd0 con 0x7f1ac8147c90 2024-04-26T20:58:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.964+0000 7f1ac7400640 1 --2- 172.21.15.28:0/1056145256 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ac8147c90 0x7f1ac806e610 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8163db0 tx=0x7f1ab802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda387bf9fbf9f47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.964+0000 7f1ac6a00640 1 -- 172.21.15.28:0/1056145256 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8037070 con 0x7f1ac8147c90 2024-04-26T20:58:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.964+0000 7f1ac6a00640 1 -- 172.21.15.28:0/1056145256 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab8032440 con 0x7f1ac8147c90 2024-04-26T20:58:03.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.964+0000 7f1ac6a00640 1 -- 172.21.15.28:0/1056145256 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8032810 con 0x7f1ac8147c90 2024-04-26T20:58:03.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.965+0000 7f1ace600640 1 -- 172.21.15.28:0/1056145256 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ac8147c90 msgr2=0x7f1ac806e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.965+0000 7f1ace600640 1 --2- 172.21.15.28:0/1056145256 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ac8147c90 0x7f1ac806e610 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8163db0 tx=0x7f1ab802a140 comp rx=0 tx=0).stop 2024-04-26T20:58:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.965+0000 7f1ace600640 1 -- 172.21.15.28:0/1056145256 shutdown_connections 2024-04-26T20:58:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.965+0000 7f1ace600640 1 --2- 172.21.15.28:0/1056145256 >> v2:172.21.15.103:3300/0 conn(0x7f1ac8073160 0x7f1ac80735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.965+0000 7f1ace600640 1 --2- 172.21.15.28:0/1056145256 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1ac80b5480 0x7f1ac80b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.965+0000 7f1ace600640 1 --2- 172.21.15.28:0/1056145256 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ac8147c90 0x7f1ac806e610 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.965+0000 7f1ace600640 1 -- 172.21.15.28:0/1056145256 >> 172.21.15.28:0/1056145256 conn(0x7f1ac806c770 msgr2=0x7f1ac806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.965+0000 7f1ace600640 1 -- 172.21.15.28:0/1056145256 shutdown_connections 2024-04-26T20:58:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.965+0000 7f1ace600640 1 -- 172.21.15.28:0/1056145256 wait complete. 2024-04-26T20:58:03.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.967+0000 7f1ace600640 1 Processor -- start 2024-04-26T20:58:03.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.968+0000 7f1ace600640 1 -- start start 2024-04-26T20:58:03.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.969+0000 7f1ace600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1ac8073160 0x7f1ac81f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.969+0000 7f1ace600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ac80b5480 0x7f1ac80ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.969+0000 7f1ace600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ac8147c90 0x7f1ac80ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.969+0000 7f1ace600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac806eb50 con 0x7f1ac80b5480 2024-04-26T20:58:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.969+0000 7f1ace600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac80b3eb0 con 0x7f1ac8147c90 2024-04-26T20:58:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.969+0000 7f1ace600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac80af7b0 con 0x7f1ac8073160 2024-04-26T20:58:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.969+0000 7f1abfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ac80b5480 0x7f1ac80ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.969+0000 7f1abfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ac80b5480 0x7f1ac80ccab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60974/0 (socket says 172.21.15.28:60974) 2024-04-26T20:58:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.969+0000 7f1abfe00640 1 -- 172.21.15.28:0/1903415517 learned_addr learned my addr 172.21.15.28:0/1903415517 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.970+0000 7f1ac7400640 1 --2- 172.21.15.28:0/1903415517 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1ac8073160 0x7f1ac81f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.970+0000 7f1abfe00640 1 -- 172.21.15.28:0/1903415517 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1ac8073160 msgr2=0x7f1ac81f8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.970+0000 7f1abfe00640 1 --2- 172.21.15.28:0/1903415517 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1ac8073160 0x7f1ac81f8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.970+0000 7f1abfe00640 1 -- 172.21.15.28:0/1903415517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ac8147c90 msgr2=0x7f1ac80ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.970+0000 7f1abfe00640 1 --2- 172.21.15.28:0/1903415517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ac8147c90 0x7f1ac80ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:03.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.970+0000 7f1abfe00640 1 -- 172.21.15.28:0/1903415517 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac80cea50 con 0x7f1ac80b5480 2024-04-26T20:58:03.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.970+0000 7f1ac7400640 1 --2- 172.21.15.28:0/1903415517 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1ac8073160 0x7f1ac81f8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:03.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.970+0000 7f1abfe00640 1 --2- 172.21.15.28:0/1903415517 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ac80b5480 0x7f1ac80ccab0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000d830 tx=0x7f1ab000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:03.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.970+0000 7f1abf400640 1 -- 172.21.15.28:0/1903415517 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab000b6c0 con 0x7f1ac80b5480 2024-04-26T20:58:03.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.970+0000 7f1abf400640 1 -- 172.21.15.28:0/1903415517 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ab0019070 con 0x7f1ac80b5480 2024-04-26T20:58:03.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.970+0000 7f1abf400640 1 -- 172.21.15.28:0/1903415517 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab000ba80 con 0x7f1ac80b5480 2024-04-26T20:58:03.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.970+0000 7f1ace600640 1 -- 172.21.15.28:0/1903415517 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac80ced30 con 0x7f1ac80b5480 2024-04-26T20:58:03.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.971+0000 7f1ace600640 1 -- 172.21.15.28:0/1903415517 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac80cf2f0 con 0x7f1ac80b5480 2024-04-26T20:58:03.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.972+0000 7f1ace600640 1 -- 172.21.15.28:0/1903415517 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac806a400 con 0x7f1ac80b5480 2024-04-26T20:58:03.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.973+0000 7f1abf400640 1 -- 172.21.15.28:0/1903415517 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ab001e460 con 0x7f1ac80b5480 2024-04-26T20:58:03.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.974+0000 7f1abf400640 1 --2- 172.21.15.28:0/1903415517 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1aa406c8f0 0x7f1aa406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:03.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.974+0000 7f1abf400640 1 -- 172.21.15.28:0/1903415517 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f1ab008bef0 con 0x7f1ac80b5480 2024-04-26T20:58:03.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.974+0000 7f1ac7400640 1 --2- 172.21.15.28:0/1903415517 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1aa406c8f0 0x7f1aa406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:03.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.975+0000 7f1ac7400640 1 --2- 172.21.15.28:0/1903415517 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1aa406c8f0 0x7f1aa406edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8034000 tx=0x7f1ab802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:03.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:03.978+0000 7f1abf400640 1 -- 172.21.15.28:0/1903415517 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ab005a660 con 0x7f1ac80b5480 2024-04-26T20:58:04.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.245+0000 7f1ace600640 1 -- 172.21.15.28:0/1903415517 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f1ac80cf690 con 0x7f1ac80b5480 2024-04-26T20:58:04.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.250+0000 7f1abf400640 1 -- 172.21.15.28:0/1903415517 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f1ab0059f60 con 0x7f1ac80b5480 2024-04-26T20:58:04.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.253+0000 7f1ace600640 1 -- 172.21.15.28:0/1903415517 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1aa406c8f0 msgr2=0x7f1aa406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:04.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.253+0000 7f1ace600640 1 --2- 172.21.15.28:0/1903415517 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1aa406c8f0 0x7f1aa406edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8034000 tx=0x7f1ab802e000 comp rx=0 tx=0).stop 2024-04-26T20:58:04.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.253+0000 7f1ace600640 1 -- 172.21.15.28:0/1903415517 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ac80b5480 msgr2=0x7f1ac80ccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:04.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.253+0000 7f1ace600640 1 --2- 172.21.15.28:0/1903415517 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ac80b5480 0x7f1ac80ccab0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000d830 tx=0x7f1ab000dcf0 comp rx=0 tx=0).stop 2024-04-26T20:58:04.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.253+0000 7f1ace600640 1 -- 172.21.15.28:0/1903415517 shutdown_connections 2024-04-26T20:58:04.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.253+0000 7f1ace600640 1 --2- 172.21.15.28:0/1903415517 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1aa406c8f0 0x7f1aa406edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:04.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.253+0000 7f1ace600640 1 --2- 172.21.15.28:0/1903415517 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1ac8147c90 0x7f1ac80ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:04.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.253+0000 7f1ace600640 1 --2- 172.21.15.28:0/1903415517 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ac80b5480 0x7f1ac80ccab0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:04.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.253+0000 7f1ace600640 1 --2- 172.21.15.28:0/1903415517 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1ac8073160 0x7f1ac81f8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:04.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.253+0000 7f1ace600640 1 -- 172.21.15.28:0/1903415517 >> 172.21.15.28:0/1903415517 conn(0x7f1ac806c770 msgr2=0x7f1ac8146f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:04.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.253+0000 7f1ace600640 1 -- 172.21.15.28:0/1903415517 shutdown_connections 2024-04-26T20:58:04.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.254+0000 7f1ace600640 1 -- 172.21.15.28:0/1903415517 wait complete. 2024-04-26T20:58:04.272 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-26T20:58:04.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.477+0000 7f5327e00640 1 Processor -- start 2024-04-26T20:58:04.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.477+0000 7f5327e00640 1 -- start start 2024-04-26T20:58:04.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.479+0000 7f5327e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f53280737a0 0x7f5328073bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:04.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.479+0000 7f5327e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f532806e110 0x7f532806e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:04.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.479+0000 7f5327e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53280721d0 con 0x7f5328156180 2024-04-26T20:58:04.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.479+0000 7f5327e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f5328074130 con 0x7f53280737a0 2024-04-26T20:58:04.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.479+0000 7f5327e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f5328072de0 con 0x7f532806e110 2024-04-26T20:58:04.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.479+0000 7f5326a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f5328156180 0x7f53280b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36510/0 (socket says 172.21.15.28:36510) 2024-04-26T20:58:04.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.479+0000 7f5326a00640 1 -- 172.21.15.28:0/2185263562 learned_addr learned my addr 172.21.15.28:0/2185263562 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:04.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.479+0000 7f5326000640 1 --2- 172.21.15.28:0/2185263562 >> v2:172.21.15.28:3300/0 conn(0x7f53280737a0 0x7f5328073bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:04.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.479+0000 7f5325600640 1 -- 172.21.15.28:0/2185263562 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3303033042 0 0) 0x7f53280721d0 con 0x7f5328156180 2024-04-26T20:58:04.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.480+0000 7f5327400640 1 --2- 172.21.15.28:0/2185263562 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f532806e110 0x7f532806e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:04.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.480+0000 7f5325600640 1 -- 172.21.15.28:0/2185263562 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f530c003520 con 0x7f5328156180 2024-04-26T20:58:04.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.480+0000 7f5326000640 1 -- 172.21.15.28:0/2185263562 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f532806e110 msgr2=0x7f532806e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:04.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.480+0000 7f5326000640 1 --2- 172.21.15.28:0/2185263562 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f532806e110 0x7f532806e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:04.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.480+0000 7f5326000640 1 -- 172.21.15.28:0/2185263562 >> v1:172.21.15.28:6789/0 conn(0x7f5328156180 legacy=0x7f53280b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:04.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.480+0000 7f5326000640 1 -- 172.21.15.28:0/2185263562 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53280724b0 con 0x7f53280737a0 2024-04-26T20:58:04.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.481+0000 7f5326000640 1 --2- 172.21.15.28:0/2185263562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53280737a0 0x7f5328073bf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f53140097a0 tx=0x7f531402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82dbc368c887477a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:04.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.481+0000 7f5325600640 1 -- 172.21.15.28:0/2185263562 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5314037070 con 0x7f53280737a0 2024-04-26T20:58:04.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.481+0000 7f5325600640 1 -- 172.21.15.28:0/2185263562 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5314032440 con 0x7f53280737a0 2024-04-26T20:58:04.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.481+0000 7f5325600640 1 -- 172.21.15.28:0/2185263562 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53140327f0 con 0x7f53280737a0 2024-04-26T20:58:04.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.482+0000 7f5327e00640 1 -- 172.21.15.28:0/2185263562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53280737a0 msgr2=0x7f5328073bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:04.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.482+0000 7f5327e00640 1 --2- 172.21.15.28:0/2185263562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53280737a0 0x7f5328073bf0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f53140097a0 tx=0x7f531402a210 comp rx=0 tx=0).stop 2024-04-26T20:58:04.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.482+0000 7f5327e00640 1 -- 172.21.15.28:0/2185263562 shutdown_connections 2024-04-26T20:58:04.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.482+0000 7f5327e00640 1 --2- 172.21.15.28:0/2185263562 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f532806e110 0x7f532806e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:04.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.482+0000 7f5327e00640 1 --2- 172.21.15.28:0/2185263562 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53280737a0 0x7f5328073bf0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:04.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.482+0000 7f5327e00640 1 -- 172.21.15.28:0/2185263562 >> 172.21.15.28:0/2185263562 conn(0x7f53280b2720 msgr2=0x7f53280b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:04.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.482+0000 7f5327e00640 1 -- 172.21.15.28:0/2185263562 shutdown_connections 2024-04-26T20:58:04.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.482+0000 7f5327e00640 1 -- 172.21.15.28:0/2185263562 wait complete. 2024-04-26T20:58:04.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.484+0000 7f5327e00640 1 Processor -- start 2024-04-26T20:58:04.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.484+0000 7f5327e00640 1 -- start start 2024-04-26T20:58:04.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.485+0000 7f5327e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f532806e110 0x7f532814d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:04.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.485+0000 7f5327e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53280737a0 0x7f5328153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:04.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.485+0000 7f5327e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5328156180 0x7f5328154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:04.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.485+0000 7f5327e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f53280b4a70 con 0x7f532806e110 2024-04-26T20:58:04.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.485+0000 7f5327e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5328072a70 con 0x7f53280737a0 2024-04-26T20:58:04.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.485+0000 7f5327e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f53280b3f50 con 0x7f5328156180 2024-04-26T20:58:04.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.485+0000 7f5326a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f532806e110 0x7f532814d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:04.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.485+0000 7f5326a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f532806e110 0x7f532814d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:60994/0 (socket says 172.21.15.28:60994) 2024-04-26T20:58:04.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.485+0000 7f5326a00640 1 -- 172.21.15.28:0/2045924891 learned_addr learned my addr 172.21.15.28:0/2045924891 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:04.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.485+0000 7f5326a00640 1 -- 172.21.15.28:0/2045924891 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5328156180 msgr2=0x7f5328154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:04.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.485+0000 7f5326a00640 1 --2- 172.21.15.28:0/2045924891 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5328156180 0x7f5328154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:04.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.485+0000 7f5326a00640 1 -- 172.21.15.28:0/2045924891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53280737a0 msgr2=0x7f5328153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:04.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.485+0000 7f5326a00640 1 --2- 172.21.15.28:0/2045924891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53280737a0 0x7f5328153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:04.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.485+0000 7f5326a00640 1 -- 172.21.15.28:0/2045924891 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5328154690 con 0x7f532806e110 2024-04-26T20:58:04.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.486+0000 7f5326a00640 1 --2- 172.21.15.28:0/2045924891 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f532806e110 0x7f532814d2e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f531800c3c0 tx=0x7f531800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:04.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.486+0000 7f531f400640 1 -- 172.21.15.28:0/2045924891 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5318017070 con 0x7f532806e110 2024-04-26T20:58:04.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.486+0000 7f531f400640 1 -- 172.21.15.28:0/2045924891 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5318005ce0 con 0x7f532806e110 2024-04-26T20:58:04.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.486+0000 7f531f400640 1 -- 172.21.15.28:0/2045924891 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53180072f0 con 0x7f532806e110 2024-04-26T20:58:04.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.486+0000 7f5327e00640 1 -- 172.21.15.28:0/2045924891 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f532814bef0 con 0x7f532806e110 2024-04-26T20:58:04.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.486+0000 7f5327e00640 1 -- 172.21.15.28:0/2045924891 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f532814c4b0 con 0x7f532806e110 2024-04-26T20:58:04.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.487+0000 7f5327e00640 1 -- 172.21.15.28:0/2045924891 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f532814cbc0 con 0x7f532806e110 2024-04-26T20:58:04.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.488+0000 7f531f400640 1 -- 172.21.15.28:0/2045924891 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f531801a070 con 0x7f532806e110 2024-04-26T20:58:04.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.489+0000 7f531f400640 1 --2- 172.21.15.28:0/2045924891 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f530806c8f0 0x7f530806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:04.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.490+0000 7f531f400640 1 -- 172.21.15.28:0/2045924891 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f531808acd0 con 0x7f532806e110 2024-04-26T20:58:04.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.490+0000 7f5326000640 1 --2- 172.21.15.28:0/2045924891 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f530806c8f0 0x7f530806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:04.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.491+0000 7f5326000640 1 --2- 172.21.15.28:0/2045924891 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f530806c8f0 0x7f530806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5314034000 tx=0x7f531402c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:04.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.494+0000 7f531f400640 1 -- 172.21.15.28:0/2045924891 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5328154690 con 0x7f532806e110 2024-04-26T20:58:04.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.774+0000 7f5327e00640 1 -- 172.21.15.28:0/2045924891 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f53280b34b0 con 0x7f532806e110 2024-04-26T20:58:04.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.779+0000 7f531f400640 1 -- 172.21.15.28:0/2045924891 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7f5318059320 con 0x7f532806e110 2024-04-26T20:58:04.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.781+0000 7f5327e00640 1 -- 172.21.15.28:0/2045924891 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f530806c8f0 msgr2=0x7f530806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:04.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.781+0000 7f5327e00640 1 --2- 172.21.15.28:0/2045924891 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f530806c8f0 0x7f530806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5314034000 tx=0x7f531402c000 comp rx=0 tx=0).stop 2024-04-26T20:58:04.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.781+0000 7f5327e00640 1 -- 172.21.15.28:0/2045924891 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f532806e110 msgr2=0x7f532814d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:04.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.781+0000 7f5327e00640 1 --2- 172.21.15.28:0/2045924891 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f532806e110 0x7f532814d2e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f531800c3c0 tx=0x7f531800c880 comp rx=0 tx=0).stop 2024-04-26T20:58:04.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.781+0000 7f5327e00640 1 -- 172.21.15.28:0/2045924891 shutdown_connections 2024-04-26T20:58:04.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.781+0000 7f5327e00640 1 --2- 172.21.15.28:0/2045924891 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f530806c8f0 0x7f530806edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:04.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.781+0000 7f5327e00640 1 --2- 172.21.15.28:0/2045924891 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5328156180 0x7f5328154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:04.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.781+0000 7f5327e00640 1 --2- 172.21.15.28:0/2045924891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53280737a0 0x7f5328153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:04.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.781+0000 7f5327e00640 1 --2- 172.21.15.28:0/2045924891 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f532806e110 0x7f532814d2e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:04.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.781+0000 7f5327e00640 1 -- 172.21.15.28:0/2045924891 >> 172.21.15.28:0/2045924891 conn(0x7f53280b2720 msgr2=0x7f5328160750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:04.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.781+0000 7f5327e00640 1 -- 172.21.15.28:0/2045924891 shutdown_connections 2024-04-26T20:58:04.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:04.781+0000 7f5327e00640 1 -- 172.21.15.28:0/2045924891 wait complete. 2024-04-26T20:58:04.801 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-26T20:58:05.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.004+0000 7f89ef200640 1 Processor -- start 2024-04-26T20:58:05.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.004+0000 7f89ef200640 1 -- start start 2024-04-26T20:58:05.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.005+0000 7f89ef200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f89f0072920 0x7f89f01638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.005+0000 7f89ef200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89f0072e70 con 0x7f89f015c310 2024-04-26T20:58:05.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.005+0000 7f89ef200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89f0073150 con 0x7f89f015e850 2024-04-26T20:58:05.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.005+0000 7f89ef200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f89f00739f0 con 0x7f89f0072920 2024-04-26T20:58:05.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.005+0000 7f89ede00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f89f015c310 0x7f89f015c6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36518/0 (socket says 172.21.15.28:36518) 2024-04-26T20:58:05.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.006+0000 7f89ede00640 1 -- 172.21.15.28:0/689603363 learned_addr learned my addr 172.21.15.28:0/689603363 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:05.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.006+0000 7f89e7e00640 1 --2- 172.21.15.28:0/689603363 >> v2:172.21.15.103:3300/0 conn(0x7f89f0072920 0x7f89f01638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:05.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.006+0000 7f89ed400640 1 -- 172.21.15.28:0/689603363 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4144301465 0 0) 0x7f89f0072e70 con 0x7f89f015c310 2024-04-26T20:58:05.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.006+0000 7f89ed400640 1 -- 172.21.15.28:0/689603363 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89d4003520 con 0x7f89f015c310 2024-04-26T20:58:05.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.006+0000 7f89ed400640 1 -- 172.21.15.28:0/689603363 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3389152864 0 0) 0x7f89d4003520 con 0x7f89f015c310 2024-04-26T20:58:05.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.006+0000 7f89ed400640 1 -- 172.21.15.28:0/689603363 >> v2:172.21.15.103:3300/0 conn(0x7f89f0072920 msgr2=0x7f89f01638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:05.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.006+0000 7f89ed400640 1 --2- 172.21.15.28:0/689603363 >> v2:172.21.15.103:3300/0 conn(0x7f89f0072920 0x7f89f01638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.006+0000 7f89ed400640 1 -- 172.21.15.28:0/689603363 >> v1:172.21.15.103:6789/0 conn(0x7f89f015e850 legacy=0x7f89f0165e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:05.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.006+0000 7f89ed400640 1 -- 172.21.15.28:0/689603363 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f0073430 con 0x7f89f015c310 2024-04-26T20:58:05.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.006+0000 7f89ed400640 1 -- 172.21.15.28:0/689603363 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f89e0001670 con 0x7f89f015c310 2024-04-26T20:58:05.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.006+0000 7f89ed400640 1 -- 172.21.15.28:0/689603363 >> v1:172.21.15.28:6789/0 conn(0x7f89f015c310 legacy=0x7f89f015c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:05.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.007+0000 7f89ed400640 1 --2- 172.21.15.28:0/689603363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89d4003a00 0x7f89d4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.007+0000 7f89ed400640 1 --2- 172.21.15.28:0/689603363 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89d4004800 0x7f89d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.007+0000 7f89ed400640 1 --2- 172.21.15.28:0/689603363 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f89d4005210 0x7f89d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.007+0000 7f89ed400640 1 -- 172.21.15.28:0/689603363 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f89f00739f0 con 0x7f89d4004800 2024-04-26T20:58:05.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.007+0000 7f89ed400640 1 -- 172.21.15.28:0/689603363 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f89d400c160 con 0x7f89d4003a00 2024-04-26T20:58:05.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.007+0000 7f89ed400640 1 -- 172.21.15.28:0/689603363 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f89d400c360 con 0x7f89d4005210 2024-04-26T20:58:05.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.007+0000 7f89e7e00640 1 --2- 172.21.15.28:0/689603363 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89d4004800 0x7f89d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:05.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.007+0000 7f89ede00640 1 --2- 172.21.15.28:0/689603363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89d4003a00 0x7f89d4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:05.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.007+0000 7f89e7e00640 1 -- 172.21.15.28:0/689603363 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f89d4005210 msgr2=0x7f89d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:05.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.007+0000 7f89e7e00640 1 --2- 172.21.15.28:0/689603363 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f89d4005210 0x7f89d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.007+0000 7f89e7e00640 1 -- 172.21.15.28:0/689603363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89d4003a00 msgr2=0x7f89d4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:05.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.007+0000 7f89e7e00640 1 --2- 172.21.15.28:0/689603363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89d4003a00 0x7f89d4003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.007+0000 7f89e7e00640 1 -- 172.21.15.28:0/689603363 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d4003520 con 0x7f89d4004800 2024-04-26T20:58:05.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.007+0000 7f89ee800640 1 -- 172.21.15.28:0/689603363 reap_dead start 2024-04-26T20:58:05.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.008+0000 7f89e7e00640 1 --2- 172.21.15.28:0/689603363 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89d4004800 0x7f89d4004c80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009a90 tx=0x7f89d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e831ba5d10c13631 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:05.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.009+0000 7f89ed400640 1 -- 172.21.15.28:0/689603363 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d8036070 con 0x7f89d4004800 2024-04-26T20:58:05.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.009+0000 7f89ed400640 1 -- 172.21.15.28:0/689603363 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89d8031400 con 0x7f89d4004800 2024-04-26T20:58:05.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.009+0000 7f89ed400640 1 -- 172.21.15.28:0/689603363 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d8009690 con 0x7f89d4004800 2024-04-26T20:58:05.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.009+0000 7f89ef200640 1 -- 172.21.15.28:0/689603363 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89d4004800 msgr2=0x7f89d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:05.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.009+0000 7f89ef200640 1 --2- 172.21.15.28:0/689603363 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89d4004800 0x7f89d4004c80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f89d8009a90 tx=0x7f89d4005830 comp rx=0 tx=0).stop 2024-04-26T20:58:05.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.009+0000 7f89ef200640 1 -- 172.21.15.28:0/689603363 shutdown_connections 2024-04-26T20:58:05.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.009+0000 7f89ef200640 1 --2- 172.21.15.28:0/689603363 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89d4004800 0x7f89d4004c80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.009+0000 7f89ef200640 1 -- 172.21.15.28:0/689603363 >> 172.21.15.28:0/689603363 conn(0x7f89f00b2720 msgr2=0x7f89f00b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:05.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.009+0000 7f89ef200640 1 -- 172.21.15.28:0/689603363 shutdown_connections 2024-04-26T20:58:05.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.009+0000 7f89ef200640 1 -- 172.21.15.28:0/689603363 wait complete. 2024-04-26T20:58:05.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.011+0000 7f89ef200640 1 Processor -- start 2024-04-26T20:58:05.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.011+0000 7f89ef200640 1 -- start start 2024-04-26T20:58:05.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.012+0000 7f89ef200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f89d4004800 0x7f89f01f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.012+0000 7f89ef200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89f00b15e0 0x7f89f00b1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.012+0000 7f89ef200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89f00ae930 0x7f89f00aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.012+0000 7f89ef200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f89f006fe10 con 0x7f89f00ae930 2024-04-26T20:58:05.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.012+0000 7f89ef200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f89f006fc90 con 0x7f89f00b15e0 2024-04-26T20:58:05.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.012+0000 7f89ef200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f89f006df00 con 0x7f89d4004800 2024-04-26T20:58:05.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.013+0000 7f89ee800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89f00ae930 0x7f89f00aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:05.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.013+0000 7f89ee800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89f00ae930 0x7f89f00aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:32786/0 (socket says 172.21.15.28:32786) 2024-04-26T20:58:05.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.013+0000 7f89ee800640 1 -- 172.21.15.28:0/4240710941 learned_addr learned my addr 172.21.15.28:0/4240710941 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:05.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.013+0000 7f89ede00640 1 --2- 172.21.15.28:0/4240710941 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f89d4004800 0x7f89f01f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:05.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.013+0000 7f89e7e00640 1 --2- 172.21.15.28:0/4240710941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89f00b15e0 0x7f89f00b1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:05.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.013+0000 7f89ee800640 1 -- 172.21.15.28:0/4240710941 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f89d4004800 msgr2=0x7f89f01f48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:05.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.013+0000 7f89ee800640 1 --2- 172.21.15.28:0/4240710941 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f89d4004800 0x7f89f01f48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.013+0000 7f89ee800640 1 -- 172.21.15.28:0/4240710941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89f00b15e0 msgr2=0x7f89f00b1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:05.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.013+0000 7f89ee800640 1 --2- 172.21.15.28:0/4240710941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89f00b15e0 0x7f89f00b1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.013+0000 7f89ee800640 1 -- 172.21.15.28:0/4240710941 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f00b3640 con 0x7f89f00ae930 2024-04-26T20:58:05.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.013+0000 7f89e7e00640 1 --2- 172.21.15.28:0/4240710941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89f00b15e0 0x7f89f00b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:05.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.013+0000 7f89ee800640 1 --2- 172.21.15.28:0/4240710941 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89f00ae930 0x7f89f00aedb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f89f00cdab0 tx=0x7f89e800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:05.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.014+0000 7f89e6000640 1 -- 172.21.15.28:0/4240710941 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e8017070 con 0x7f89f00ae930 2024-04-26T20:58:05.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.014+0000 7f89e6000640 1 -- 172.21.15.28:0/4240710941 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89e8005d60 con 0x7f89f00ae930 2024-04-26T20:58:05.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.014+0000 7f89e6000640 1 -- 172.21.15.28:0/4240710941 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89e800a850 con 0x7f89f00ae930 2024-04-26T20:58:05.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.014+0000 7f89ef200640 1 -- 172.21.15.28:0/4240710941 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89f00af6a0 con 0x7f89f00ae930 2024-04-26T20:58:05.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.014+0000 7f89ef200640 1 -- 172.21.15.28:0/4240710941 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89f013f480 con 0x7f89f00ae930 2024-04-26T20:58:05.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.015+0000 7f89ef200640 1 -- 172.21.15.28:0/4240710941 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89f00b1fa0 con 0x7f89f00ae930 2024-04-26T20:58:05.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.016+0000 7f89e6000640 1 -- 172.21.15.28:0/4240710941 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89e801a070 con 0x7f89f00ae930 2024-04-26T20:58:05.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.017+0000 7f89e6000640 1 --2- 172.21.15.28:0/4240710941 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f89cc06c8f0 0x7f89cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.017+0000 7f89e6000640 1 -- 172.21.15.28:0/4240710941 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f89e808a990 con 0x7f89f00ae930 2024-04-26T20:58:05.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.017+0000 7f89ede00640 1 --2- 172.21.15.28:0/4240710941 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f89cc06c8f0 0x7f89cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:05.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.018+0000 7f89ede00640 1 --2- 172.21.15.28:0/4240710941 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f89cc06c8f0 0x7f89cc06edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f89f00ccd00 tx=0x7f89e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:05.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.021+0000 7f89e6000640 1 -- 172.21.15.28:0/4240710941 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f89f00b3640 con 0x7f89f00ae930 2024-04-26T20:58:05.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.297+0000 7f89ef200640 1 -- 172.21.15.28:0/4240710941 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f89f006a360 con 0x7f89f00ae930 2024-04-26T20:58:05.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.302+0000 7f89e6000640 1 -- 172.21.15.28:0/4240710941 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f89e8059100 con 0x7f89f00ae930 2024-04-26T20:58:05.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.304+0000 7f89ef200640 1 -- 172.21.15.28:0/4240710941 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f89cc06c8f0 msgr2=0x7f89cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:05.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.304+0000 7f89ef200640 1 --2- 172.21.15.28:0/4240710941 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f89cc06c8f0 0x7f89cc06edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f89f00ccd00 tx=0x7f89e0009040 comp rx=0 tx=0).stop 2024-04-26T20:58:05.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.304+0000 7f89ef200640 1 -- 172.21.15.28:0/4240710941 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89f00ae930 msgr2=0x7f89f00aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:05.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.304+0000 7f89ef200640 1 --2- 172.21.15.28:0/4240710941 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89f00ae930 0x7f89f00aedb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f89f00cdab0 tx=0x7f89e800cb90 comp rx=0 tx=0).stop 2024-04-26T20:58:05.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.305+0000 7f89ef200640 1 -- 172.21.15.28:0/4240710941 shutdown_connections 2024-04-26T20:58:05.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.305+0000 7f89ef200640 1 --2- 172.21.15.28:0/4240710941 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f89cc06c8f0 0x7f89cc06edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.305+0000 7f89ef200640 1 --2- 172.21.15.28:0/4240710941 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f89f00ae930 0x7f89f00aedb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.305+0000 7f89ef200640 1 --2- 172.21.15.28:0/4240710941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89f00b15e0 0x7f89f00b1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.305+0000 7f89ef200640 1 --2- 172.21.15.28:0/4240710941 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f89d4004800 0x7f89f01f48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.305+0000 7f89ef200640 1 -- 172.21.15.28:0/4240710941 >> 172.21.15.28:0/4240710941 conn(0x7f89f00b2720 msgr2=0x7f89f014b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:05.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.305+0000 7f89ef200640 1 -- 172.21.15.28:0/4240710941 shutdown_connections 2024-04-26T20:58:05.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.305+0000 7f89ef200640 1 -- 172.21.15.28:0/4240710941 wait complete. 2024-04-26T20:58:05.327 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:58:05.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.532+0000 7fba6fe00640 1 Processor -- start 2024-04-26T20:58:05.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.532+0000 7fba6fe00640 1 -- start start 2024-04-26T20:58:05.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.534+0000 7fba6fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fba70146670 0x7fba70071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.534+0000 7fba6fe00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba70146440 con 0x7fba701610c0 2024-04-26T20:58:05.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.534+0000 7fba6fe00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba7006f010 con 0x7fba70146060 2024-04-26T20:58:05.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.534+0000 7fba6fe00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fba7006f510 con 0x7fba70146670 2024-04-26T20:58:05.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.534+0000 7fba6e000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fba70146670 0x7fba70071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:05.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.534+0000 7fba6e000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fba70146670 0x7fba70071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:32802/0 (socket says 172.21.15.28:32802) 2024-04-26T20:58:05.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.534+0000 7fba6e000640 1 -- 172.21.15.28:0/1934839659 learned_addr learned my addr 172.21.15.28:0/1934839659 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:05.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.535+0000 7fba6e000640 1 -- 172.21.15.28:0/1934839659 >> v1:172.21.15.103:6789/0 conn(0x7fba70146060 legacy=0x7fba701609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:05.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.535+0000 7fba6e000640 1 -- 172.21.15.28:0/1934839659 >> v1:172.21.15.28:6789/0 conn(0x7fba701610c0 legacy=0x7fba700cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:05.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.535+0000 7fba6e000640 1 -- 172.21.15.28:0/1934839659 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba701614d0 con 0x7fba70146670 2024-04-26T20:58:05.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.536+0000 7fba6e000640 1 --2- 172.21.15.28:0/1934839659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba70146670 0x7fba70071e20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009870 tx=0x7fba5c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5ef2328963a1ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:05.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.536+0000 7fba6d600640 1 -- 172.21.15.28:0/1934839659 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c037070 con 0x7fba70146670 2024-04-26T20:58:05.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.536+0000 7fba6d600640 1 -- 172.21.15.28:0/1934839659 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba5c032440 con 0x7fba70146670 2024-04-26T20:58:05.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.536+0000 7fba6d600640 1 -- 172.21.15.28:0/1934839659 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba5c032810 con 0x7fba70146670 2024-04-26T20:58:05.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.537+0000 7fba6fe00640 1 -- 172.21.15.28:0/1934839659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba70146670 msgr2=0x7fba70071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:05.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.537+0000 7fba6fe00640 1 --2- 172.21.15.28:0/1934839659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba70146670 0x7fba70071e20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009870 tx=0x7fba5c02a210 comp rx=0 tx=0).stop 2024-04-26T20:58:05.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.537+0000 7fba6fe00640 1 -- 172.21.15.28:0/1934839659 shutdown_connections 2024-04-26T20:58:05.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.537+0000 7fba6fe00640 1 --2- 172.21.15.28:0/1934839659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba70146670 0x7fba70071e20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.537+0000 7fba6fe00640 1 -- 172.21.15.28:0/1934839659 >> 172.21.15.28:0/1934839659 conn(0x7fba700d0a50 msgr2=0x7fba700d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:05.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.537+0000 7fba6fe00640 1 -- 172.21.15.28:0/1934839659 shutdown_connections 2024-04-26T20:58:05.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.537+0000 7fba6fe00640 1 -- 172.21.15.28:0/1934839659 wait complete. 2024-04-26T20:58:05.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.539+0000 7fba6fe00640 1 Processor -- start 2024-04-26T20:58:05.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.539+0000 7fba6fe00640 1 -- start start 2024-04-26T20:58:05.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.540+0000 7fba6fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba70146060 0x7fba701427b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.540+0000 7fba6fe00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fba70146670 0x7fba701f23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.540+0000 7fba6fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba701610c0 0x7fba700b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.540+0000 7fba6fe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fba7006ff60 con 0x7fba701610c0 2024-04-26T20:58:05.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.540+0000 7fba6fe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fba7006fc80 con 0x7fba70146060 2024-04-26T20:58:05.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.540+0000 7fba6fe00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fba700cb7b0 con 0x7fba70146670 2024-04-26T20:58:05.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.540+0000 7fba6f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba701610c0 0x7fba700b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:05.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.540+0000 7fba6f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba701610c0 0x7fba700b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:32806/0 (socket says 172.21.15.28:32806) 2024-04-26T20:58:05.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.540+0000 7fba6f400640 1 -- 172.21.15.28:0/3484860618 learned_addr learned my addr 172.21.15.28:0/3484860618 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:05.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.540+0000 7fba6ea00640 1 --2- 172.21.15.28:0/3484860618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba70146060 0x7fba701427b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:05.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.541+0000 7fba6e000640 1 --2- 172.21.15.28:0/3484860618 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fba70146670 0x7fba701f23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:05.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.541+0000 7fba6f400640 1 -- 172.21.15.28:0/3484860618 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fba70146670 msgr2=0x7fba701f23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:05.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.541+0000 7fba6f400640 1 --2- 172.21.15.28:0/3484860618 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fba70146670 0x7fba701f23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.541+0000 7fba6f400640 1 -- 172.21.15.28:0/3484860618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba70146060 msgr2=0x7fba701427b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:05.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.541+0000 7fba6f400640 1 --2- 172.21.15.28:0/3484860618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba70146060 0x7fba701427b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.541+0000 7fba6f400640 1 -- 172.21.15.28:0/3484860618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba701f2930 con 0x7fba701610c0 2024-04-26T20:58:05.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.541+0000 7fba6ea00640 1 --2- 172.21.15.28:0/3484860618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba70146060 0x7fba701427b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:05.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.541+0000 7fba6e000640 1 --2- 172.21.15.28:0/3484860618 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fba70146670 0x7fba701f23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:05.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.541+0000 7fba6f400640 1 --2- 172.21.15.28:0/3484860618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba701610c0 0x7fba700b4860 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fba6800c6e0 tx=0x7fba6800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:05.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.541+0000 7fba66a00640 1 -- 172.21.15.28:0/3484860618 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68003b70 con 0x7fba701610c0 2024-04-26T20:58:05.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.541+0000 7fba66a00640 1 -- 172.21.15.28:0/3484860618 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba68003d30 con 0x7fba701610c0 2024-04-26T20:58:05.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.541+0000 7fba66a00640 1 -- 172.21.15.28:0/3484860618 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68005310 con 0x7fba701610c0 2024-04-26T20:58:05.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.541+0000 7fba6fe00640 1 -- 172.21.15.28:0/3484860618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba700b4da0 con 0x7fba701610c0 2024-04-26T20:58:05.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.541+0000 7fba6fe00640 1 -- 172.21.15.28:0/3484860618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba700b5320 con 0x7fba701610c0 2024-04-26T20:58:05.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.542+0000 7fba6fe00640 1 -- 172.21.15.28:0/3484860618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba7006a360 con 0x7fba701610c0 2024-04-26T20:58:05.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.543+0000 7fba66a00640 1 -- 172.21.15.28:0/3484860618 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba680054b0 con 0x7fba701610c0 2024-04-26T20:58:05.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.544+0000 7fba66a00640 1 --2- 172.21.15.28:0/3484860618 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fba5006c8f0 0x7fba5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.544+0000 7fba66a00640 1 -- 172.21.15.28:0/3484860618 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fba6808b710 con 0x7fba701610c0 2024-04-26T20:58:05.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.545+0000 7fba6ea00640 1 --2- 172.21.15.28:0/3484860618 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fba5006c8f0 0x7fba5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:05.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.545+0000 7fba6ea00640 1 --2- 172.21.15.28:0/3484860618 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fba5006c8f0 0x7fba5006edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fba60003eb0 tx=0x7fba60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:05.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.549+0000 7fba66a00640 1 -- 172.21.15.28:0/3484860618 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fba68059df0 con 0x7fba701610c0 2024-04-26T20:58:05.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.754+0000 7fba6fe00640 1 -- 172.21.15.28:0/3484860618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba70072f40 con 0x7fba701610c0 2024-04-26T20:58:05.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.755+0000 7fba66a00640 1 -- 172.21.15.28:0/3484860618 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+12867 (secure 0 0 0) 0x7fba680596f0 con 0x7fba701610c0 2024-04-26T20:58:05.757 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:05.757 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":30,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:58:01.639624+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:58:05.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.758+0000 7fba6fe00640 1 -- 172.21.15.28:0/3484860618 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fba5006c8f0 msgr2=0x7fba5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:05.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.758+0000 7fba6fe00640 1 --2- 172.21.15.28:0/3484860618 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fba5006c8f0 0x7fba5006edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fba60003eb0 tx=0x7fba60009040 comp rx=0 tx=0).stop 2024-04-26T20:58:05.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.758+0000 7fba6fe00640 1 -- 172.21.15.28:0/3484860618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba701610c0 msgr2=0x7fba700b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:05.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.758+0000 7fba6fe00640 1 --2- 172.21.15.28:0/3484860618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba701610c0 0x7fba700b4860 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fba6800c6e0 tx=0x7fba6800cba0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.758+0000 7fba6fe00640 1 -- 172.21.15.28:0/3484860618 shutdown_connections 2024-04-26T20:58:05.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.758+0000 7fba6fe00640 1 --2- 172.21.15.28:0/3484860618 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fba5006c8f0 0x7fba5006edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.758+0000 7fba6fe00640 1 --2- 172.21.15.28:0/3484860618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba701610c0 0x7fba700b4860 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.758+0000 7fba6fe00640 1 --2- 172.21.15.28:0/3484860618 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fba70146670 0x7fba701f23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.758+0000 7fba6fe00640 1 --2- 172.21.15.28:0/3484860618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fba70146060 0x7fba701427b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.758+0000 7fba6fe00640 1 -- 172.21.15.28:0/3484860618 >> 172.21.15.28:0/3484860618 conn(0x7fba700d0a50 msgr2=0x7fba7015ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:05.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.759+0000 7fba6fe00640 1 -- 172.21.15.28:0/3484860618 shutdown_connections 2024-04-26T20:58:05.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.759+0000 7fba6fe00640 1 -- 172.21.15.28:0/3484860618 wait complete. 2024-04-26T20:58:05.777 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:58:05.777 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:58:05.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.988+0000 7f776aa00640 1 Processor -- start 2024-04-26T20:58:05.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.988+0000 7f776aa00640 1 -- start start 2024-04-26T20:58:05.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.989+0000 7f776aa00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f77640d1de0 0x7f77640d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.989+0000 7f776aa00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f776415f360 0x7f77640d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.990+0000 7f776aa00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77640d2700 con 0x7f77640bac00 2024-04-26T20:58:05.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.990+0000 7f776aa00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f77640ba350 con 0x7f77640d1de0 2024-04-26T20:58:05.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.990+0000 7f776aa00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f77640b9ab0 con 0x7f776415f360 2024-04-26T20:58:05.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.990+0000 7f776a000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f77640bac00 0x7f77640d39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36530/0 (socket says 172.21.15.28:36530) 2024-04-26T20:58:05.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.990+0000 7f776a000640 1 -- 172.21.15.28:0/3631947265 learned_addr learned my addr 172.21.15.28:0/3631947265 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:05.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.990+0000 7f7768c00640 1 --2- 172.21.15.28:0/3631947265 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f776415f360 0x7f77640d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:05.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.990+0000 7f7769600640 1 --2- 172.21.15.28:0/3631947265 >> v2:172.21.15.103:3300/0 conn(0x7f77640d1de0 0x7f77640d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:05.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.990+0000 7f7763e00640 1 -- 172.21.15.28:0/3631947265 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2117614813 0 0) 0x7f77640d2700 con 0x7f77640bac00 2024-04-26T20:58:05.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.991+0000 7f7763e00640 1 -- 172.21.15.28:0/3631947265 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f774c003520 con 0x7f77640bac00 2024-04-26T20:58:05.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.991+0000 7f7763e00640 1 -- 172.21.15.28:0/3631947265 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3301727568 0 0) 0x7f774c003520 con 0x7f77640bac00 2024-04-26T20:58:05.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.991+0000 7f7763e00640 1 -- 172.21.15.28:0/3631947265 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f776415f360 msgr2=0x7f77640d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:05.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.991+0000 7f7763e00640 1 --2- 172.21.15.28:0/3631947265 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f776415f360 0x7f77640d3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.991+0000 7f7763e00640 1 -- 172.21.15.28:0/3631947265 >> v2:172.21.15.103:3300/0 conn(0x7f77640d1de0 msgr2=0x7f77640d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:05.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.991+0000 7f7763e00640 1 --2- 172.21.15.28:0/3631947265 >> v2:172.21.15.103:3300/0 conn(0x7f77640d1de0 0x7f77640d21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.991+0000 7f7763e00640 1 -- 172.21.15.28:0/3631947265 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77640b9d90 con 0x7f77640bac00 2024-04-26T20:58:05.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.991+0000 7f7763e00640 1 -- 172.21.15.28:0/3631947265 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f775c0019f0 con 0x7f77640bac00 2024-04-26T20:58:05.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.991+0000 7f7763e00640 1 -- 172.21.15.28:0/3631947265 >> v1:172.21.15.28:6789/0 conn(0x7f77640bac00 legacy=0x7f77640d39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:05.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.991+0000 7f7763e00640 1 --2- 172.21.15.28:0/3631947265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f774c0039b0 0x7f774c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.991+0000 7f7763e00640 1 --2- 172.21.15.28:0/3631947265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f774c004800 0x7f774c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.991+0000 7f7763e00640 1 --2- 172.21.15.28:0/3631947265 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f776415f360 0x7f774c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.991+0000 7f7763e00640 1 -- 172.21.15.28:0/3631947265 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f77640ba350 con 0x7f774c004800 2024-04-26T20:58:05.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.991+0000 7f7763e00640 1 -- 172.21.15.28:0/3631947265 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77640b9ab0 con 0x7f774c0039b0 2024-04-26T20:58:05.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.991+0000 7f7763e00640 1 -- 172.21.15.28:0/3631947265 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f774c00b9c0 con 0x7f776415f360 2024-04-26T20:58:05.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.991+0000 7f7768c00640 1 --2- 172.21.15.28:0/3631947265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f774c004800 0x7f774c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:05.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.992+0000 7f7769600640 1 --2- 172.21.15.28:0/3631947265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f774c0039b0 0x7f774c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:05.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.992+0000 7f7768c00640 1 -- 172.21.15.28:0/3631947265 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f776415f360 msgr2=0x7f774c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:05.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.992+0000 7f7768c00640 1 --2- 172.21.15.28:0/3631947265 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f776415f360 0x7f774c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.992+0000 7f7768c00640 1 -- 172.21.15.28:0/3631947265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f774c0039b0 msgr2=0x7f774c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:05.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.992+0000 7f7768c00640 1 --2- 172.21.15.28:0/3631947265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f774c0039b0 0x7f774c003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:05.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.992+0000 7f7768c00640 1 -- 172.21.15.28:0/3631947265 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774c003520 con 0x7f774c004800 2024-04-26T20:58:05.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.993+0000 7f7768c00640 1 --2- 172.21.15.28:0/3631947265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f774c004800 0x7f774c004c80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7750009a90 tx=0x7f774c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c915b514bbf3342d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:05.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.993+0000 7f7763e00640 1 -- 172.21.15.28:0/3631947265 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750036070 con 0x7f774c004800 2024-04-26T20:58:05.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.993+0000 7f7763e00640 1 -- 172.21.15.28:0/3631947265 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7750031400 con 0x7f774c004800 2024-04-26T20:58:05.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.993+0000 7f7763e00640 1 -- 172.21.15.28:0/3631947265 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7750009690 con 0x7f774c004800 2024-04-26T20:58:05.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.994+0000 7f776aa00640 1 -- 172.21.15.28:0/3631947265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f774c004800 msgr2=0x7f774c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:05.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.994+0000 7f776aa00640 1 --2- 172.21.15.28:0/3631947265 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f774c004800 0x7f774c004c80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7750009a90 tx=0x7f774c005830 comp rx=0 tx=0).stop 2024-04-26T20:58:05.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.994+0000 7f776a000640 1 -- 172.21.15.28:0/3631947265 reap_dead start 2024-04-26T20:58:05.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.994+0000 7f776aa00640 1 -- 172.21.15.28:0/3631947265 shutdown_connections 2024-04-26T20:58:05.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.994+0000 7f776aa00640 1 -- 172.21.15.28:0/3631947265 >> 172.21.15.28:0/3631947265 conn(0x7f776406c770 msgr2=0x7f776406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:05.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.994+0000 7f776aa00640 1 -- 172.21.15.28:0/3631947265 shutdown_connections 2024-04-26T20:58:05.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.994+0000 7f776aa00640 1 -- 172.21.15.28:0/3631947265 wait complete. 2024-04-26T20:58:05.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.996+0000 7f776aa00640 1 Processor -- start 2024-04-26T20:58:05.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.996+0000 7f776aa00640 1 -- start start 2024-04-26T20:58:05.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.997+0000 7f776aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f774c004800 0x7f77641565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.997+0000 7f776aa00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f77640ce290 0x7f77640ce710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.997+0000 7f776aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7764161380 0x7f7764161830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:05.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.997+0000 7f776aa00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f77640d3490 con 0x7f774c004800 2024-04-26T20:58:05.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.997+0000 7f776aa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77640d2f70 con 0x7f7764161380 2024-04-26T20:58:05.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.997+0000 7f776aa00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f77640d3610 con 0x7f77640ce290 2024-04-26T20:58:05.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.997+0000 7f7769600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f774c004800 0x7f77641565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:05.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.997+0000 7f7769600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f774c004800 0x7f77641565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:32828/0 (socket says 172.21.15.28:32828) 2024-04-26T20:58:05.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.997+0000 7f7769600640 1 -- 172.21.15.28:0/737174098 learned_addr learned my addr 172.21.15.28:0/737174098 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:06.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.998+0000 7f7769600640 1 -- 172.21.15.28:0/737174098 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f77640ce290 msgr2=0x7f77640ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:06.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.998+0000 7f7768c00640 1 --2- 172.21.15.28:0/737174098 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f77640ce290 0x7f77640ce710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:06.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.998+0000 7f776a000640 1 --2- 172.21.15.28:0/737174098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7764161380 0x7f7764161830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:06.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.998+0000 7f7769600640 1 --2- 172.21.15.28:0/737174098 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f77640ce290 0x7f77640ce710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.998+0000 7f7769600640 1 -- 172.21.15.28:0/737174098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7764161380 msgr2=0x7f7764161830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:06.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.998+0000 7f7769600640 1 --2- 172.21.15.28:0/737174098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7764161380 0x7f7764161830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.998+0000 7f7769600640 1 -- 172.21.15.28:0/737174098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776415f770 con 0x7f774c004800 2024-04-26T20:58:06.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.998+0000 7f7769600640 1 --2- 172.21.15.28:0/737174098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f774c004800 0x7f77641565f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7758010910 tx=0x7f7758010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:06.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.998+0000 7f7761600640 1 -- 172.21.15.28:0/737174098 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775800f040 con 0x7f774c004800 2024-04-26T20:58:06.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.998+0000 7f7761600640 1 -- 172.21.15.28:0/737174098 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f775800ec20 con 0x7f774c004800 2024-04-26T20:58:06.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.999+0000 7f776aa00640 1 -- 172.21.15.28:0/737174098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776415fa50 con 0x7f774c004800 2024-04-26T20:58:06.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.999+0000 7f7761600640 1 -- 172.21.15.28:0/737174098 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775801c400 con 0x7f774c004800 2024-04-26T20:58:06.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:05.999+0000 7f776aa00640 1 -- 172.21.15.28:0/737174098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7764160010 con 0x7f774c004800 2024-04-26T20:58:06.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.000+0000 7f776aa00640 1 -- 172.21.15.28:0/737174098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f776406a250 con 0x7f774c004800 2024-04-26T20:58:06.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.001+0000 7f7761600640 1 -- 172.21.15.28:0/737174098 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f775801c5a0 con 0x7f774c004800 2024-04-26T20:58:06.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.002+0000 7f7761600640 1 --2- 172.21.15.28:0/737174098 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f774406c920 0x7f774406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:06.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.002+0000 7f7761600640 1 -- 172.21.15.28:0/737174098 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f775808bed0 con 0x7f774c004800 2024-04-26T20:58:06.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.002+0000 7f7768c00640 1 --2- 172.21.15.28:0/737174098 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f774406c920 0x7f774406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:06.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.003+0000 7f7768c00640 1 --2- 172.21.15.28:0/737174098 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f774406c920 0x7f774406ede0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7750033000 tx=0x7f7750008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:06.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.006+0000 7f7761600640 1 -- 172.21.15.28:0/737174098 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7758018030 con 0x7f774c004800 2024-04-26T20:58:06.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.212+0000 7f776aa00640 1 -- 172.21.15.28:0/737174098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f77640b6380 con 0x7f774c004800 2024-04-26T20:58:06.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.212+0000 7f7761600640 1 -- 172.21.15.28:0/737174098 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) ==== 93+0+10 (secure 0 0 0) 0x7f775805a640 con 0x7f774c004800 2024-04-26T20:58:06.214 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:58:06.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.215+0000 7f776aa00640 1 -- 172.21.15.28:0/737174098 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f774406c920 msgr2=0x7f774406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:06.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.215+0000 7f776aa00640 1 --2- 172.21.15.28:0/737174098 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f774406c920 0x7f774406ede0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7750033000 tx=0x7f7750008030 comp rx=0 tx=0).stop 2024-04-26T20:58:06.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.215+0000 7f776aa00640 1 -- 172.21.15.28:0/737174098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f774c004800 msgr2=0x7f77641565f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:06.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.215+0000 7f776aa00640 1 --2- 172.21.15.28:0/737174098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f774c004800 0x7f77641565f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f7758010910 tx=0x7f7758010dd0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.215+0000 7f776aa00640 1 -- 172.21.15.28:0/737174098 shutdown_connections 2024-04-26T20:58:06.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.215+0000 7f776aa00640 1 --2- 172.21.15.28:0/737174098 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f774406c920 0x7f774406ede0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.215+0000 7f776aa00640 1 --2- 172.21.15.28:0/737174098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7764161380 0x7f7764161830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.215+0000 7f776aa00640 1 --2- 172.21.15.28:0/737174098 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f77640ce290 0x7f77640ce710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.215+0000 7f776aa00640 1 --2- 172.21.15.28:0/737174098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f774c004800 0x7f77641565f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.215+0000 7f776aa00640 1 -- 172.21.15.28:0/737174098 >> 172.21.15.28:0/737174098 conn(0x7f776406c770 msgr2=0x7f77640b7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:06.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.215+0000 7f776aa00640 1 -- 172.21.15.28:0/737174098 shutdown_connections 2024-04-26T20:58:06.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.215+0000 7f776aa00640 1 -- 172.21.15.28:0/737174098 wait complete. 2024-04-26T20:58:06.235 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T20:58:06.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.441+0000 7fefb6400640 1 Processor -- start 2024-04-26T20:58:06.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.441+0000 7fefb6400640 1 -- start start 2024-04-26T20:58:06.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.443+0000 7fefb6400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fefb006f020 0x7fefb00731f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:06.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.443+0000 7fefb6400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefb0074050 con 0x7fefb01467a0 2024-04-26T20:58:06.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.443+0000 7fefb6400640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefb015d430 con 0x7fefb0073730 2024-04-26T20:58:06.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.443+0000 7fefb6400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fefb006f550 con 0x7fefb006f020 2024-04-26T20:58:06.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.443+0000 7fefaf400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fefb01467a0 0x7fefb00d0540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36542/0 (socket says 172.21.15.28:36542) 2024-04-26T20:58:06.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.443+0000 7fefaf400640 1 -- 172.21.15.28:0/3896704380 learned_addr learned my addr 172.21.15.28:0/3896704380 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:06.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.443+0000 7fefaea00640 1 --2- 172.21.15.28:0/3896704380 >> v2:172.21.15.28:3300/0 conn(0x7fefb006f020 0x7fefb00731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:06.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.444+0000 7fefaea00640 1 -- 172.21.15.28:0/3896704380 >> v1:172.21.15.103:6789/0 conn(0x7fefb0073730 legacy=0x7fefb015d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:06.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.444+0000 7fefaea00640 1 -- 172.21.15.28:0/3896704380 >> v1:172.21.15.28:6789/0 conn(0x7fefb01467a0 legacy=0x7fefb00d0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:06.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.444+0000 7fefaea00640 1 -- 172.21.15.28:0/3896704380 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb006f830 con 0x7fefb006f020 2024-04-26T20:58:06.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.445+0000 7fefaea00640 1 --2- 172.21.15.28:0/3896704380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb006f020 0x7fefb00731f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0097a0 tx=0x7fef9c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ae9e6af38b4858 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:06.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.445+0000 7fefae000640 1 -- 172.21.15.28:0/3896704380 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c037070 con 0x7fefb006f020 2024-04-26T20:58:06.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.445+0000 7fefae000640 1 -- 172.21.15.28:0/3896704380 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef9c032440 con 0x7fefb006f020 2024-04-26T20:58:06.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.445+0000 7fefae000640 1 -- 172.21.15.28:0/3896704380 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c032810 con 0x7fefb006f020 2024-04-26T20:58:06.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.445+0000 7fefb6400640 1 -- 172.21.15.28:0/3896704380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb006f020 msgr2=0x7fefb00731f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:06.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.445+0000 7fefb6400640 1 --2- 172.21.15.28:0/3896704380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb006f020 0x7fefb00731f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0097a0 tx=0x7fef9c02a1e0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.446+0000 7fefb6400640 1 -- 172.21.15.28:0/3896704380 shutdown_connections 2024-04-26T20:58:06.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.446+0000 7fefb6400640 1 --2- 172.21.15.28:0/3896704380 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb006f020 0x7fefb00731f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.446+0000 7fefb6400640 1 -- 172.21.15.28:0/3896704380 >> 172.21.15.28:0/3896704380 conn(0x7fefb006c770 msgr2=0x7fefb006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:06.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.446+0000 7fefb6400640 1 -- 172.21.15.28:0/3896704380 shutdown_connections 2024-04-26T20:58:06.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.446+0000 7fefb6400640 1 -- 172.21.15.28:0/3896704380 wait complete. 2024-04-26T20:58:06.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.447+0000 7fefb6400640 1 Processor -- start 2024-04-26T20:58:06.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.448+0000 7fefb6400640 1 -- start start 2024-04-26T20:58:06.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.448+0000 7fefb6400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb006f020 0x7fefb013a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:06.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.448+0000 7fefb6400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb0073730 0x7fefb013e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:06.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.448+0000 7fefb6400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb01467a0 0x7fefb00b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:06.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.448+0000 7fefb6400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fefb015e070 con 0x7fefb0073730 2024-04-26T20:58:06.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.448+0000 7fefb6400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fefb006fdf0 con 0x7fefb01467a0 2024-04-26T20:58:06.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.448+0000 7fefb6400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fefb015e1f0 con 0x7fefb006f020 2024-04-26T20:58:06.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.449+0000 7fefaea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb0073730 0x7fefb013e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:06.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.449+0000 7fefaea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb0073730 0x7fefb013e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:32838/0 (socket says 172.21.15.28:32838) 2024-04-26T20:58:06.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.449+0000 7fefaea00640 1 -- 172.21.15.28:0/557185982 learned_addr learned my addr 172.21.15.28:0/557185982 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:06.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.449+0000 7fefaea00640 1 -- 172.21.15.28:0/557185982 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb006f020 msgr2=0x7fefb013a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:06.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.449+0000 7fefaea00640 1 --2- 172.21.15.28:0/557185982 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb006f020 0x7fefb013a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.449+0000 7fefaea00640 1 -- 172.21.15.28:0/557185982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb01467a0 msgr2=0x7fefb00b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:06.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.449+0000 7fefafe00640 1 --2- 172.21.15.28:0/557185982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb01467a0 0x7fefb00b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:06.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.449+0000 7fefaea00640 1 --2- 172.21.15.28:0/557185982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb01467a0 0x7fefb00b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.449+0000 7fefaea00640 1 -- 172.21.15.28:0/557185982 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb013edf0 con 0x7fefb0073730 2024-04-26T20:58:06.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.449+0000 7fefafe00640 1 --2- 172.21.15.28:0/557185982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb01467a0 0x7fefb00b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:06.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.450+0000 7fefaea00640 1 --2- 172.21.15.28:0/557185982 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb0073730 0x7fefb013e410 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0098c0 tx=0x7fef9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:06.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.450+0000 7fefa7400640 1 -- 172.21.15.28:0/557185982 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c037040 con 0x7fefb0073730 2024-04-26T20:58:06.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.450+0000 7fefa7400640 1 -- 172.21.15.28:0/557185982 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef9c042070 con 0x7fefb0073730 2024-04-26T20:58:06.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.450+0000 7fefa7400640 1 -- 172.21.15.28:0/557185982 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c030a70 con 0x7fefb0073730 2024-04-26T20:58:06.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.450+0000 7fefb6400640 1 -- 172.21.15.28:0/557185982 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb013f0d0 con 0x7fefb0073730 2024-04-26T20:58:06.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.450+0000 7fefb6400640 1 -- 172.21.15.28:0/557185982 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb013f690 con 0x7fefb0073730 2024-04-26T20:58:06.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.452+0000 7fefb6400640 1 -- 172.21.15.28:0/557185982 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefb006a400 con 0x7fefb0073730 2024-04-26T20:58:06.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.453+0000 7fefa7400640 1 -- 172.21.15.28:0/557185982 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef9c030c10 con 0x7fefb0073730 2024-04-26T20:58:06.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.453+0000 7fefa7400640 1 --2- 172.21.15.28:0/557185982 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fef9006c8f0 0x7fef9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:06.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.453+0000 7fefa7400640 1 -- 172.21.15.28:0/557185982 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fef9c0ac250 con 0x7fefb0073730 2024-04-26T20:58:06.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.454+0000 7fefaf400640 1 --2- 172.21.15.28:0/557185982 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fef9006c8f0 0x7fef9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:06.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.455+0000 7fefaf400640 1 --2- 172.21.15.28:0/557185982 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fef9006c8f0 0x7fef9006edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fefa0000f60 tx=0x7fefa0007a80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:06.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.459+0000 7fefa7400640 1 -- 172.21.15.28:0/557185982 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fef9c045360 con 0x7fefb0073730 2024-04-26T20:58:06.666 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.665+0000 7fefb6400640 1 -- 172.21.15.28:0/557185982 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fefb006d360 con 0x7fefb0073730 2024-04-26T20:58:06.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.666+0000 7fefa7400640 1 -- 172.21.15.28:0/557185982 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) ==== 84+0+65 (secure 0 0 0) 0x7fef9c07aa60 con 0x7fefb0073730 2024-04-26T20:58:06.667 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:06.667 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2024-04-26T20:58:06.667 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:06.667 INFO:teuthology.orchestra.run.smithi028.stdout: "poolnum": 1, 2024-04-26T20:58:06.667 INFO:teuthology.orchestra.run.smithi028.stdout: "poolname": ".mgr" 2024-04-26T20:58:06.667 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:06.667 INFO:teuthology.orchestra.run.smithi028.stdout:] 2024-04-26T20:58:06.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.668+0000 7fefb6400640 1 -- 172.21.15.28:0/557185982 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fef9006c8f0 msgr2=0x7fef9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:06.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.668+0000 7fefb6400640 1 --2- 172.21.15.28:0/557185982 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fef9006c8f0 0x7fef9006edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fefa0000f60 tx=0x7fefa0007a80 comp rx=0 tx=0).stop 2024-04-26T20:58:06.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.668+0000 7fefb6400640 1 -- 172.21.15.28:0/557185982 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb0073730 msgr2=0x7fefb013e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:06.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.668+0000 7fefb6400640 1 --2- 172.21.15.28:0/557185982 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb0073730 0x7fefb013e410 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0098c0 tx=0x7fef9c0046c0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.669+0000 7fefb6400640 1 -- 172.21.15.28:0/557185982 shutdown_connections 2024-04-26T20:58:06.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.669+0000 7fefb6400640 1 --2- 172.21.15.28:0/557185982 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fef9006c8f0 0x7fef9006edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.669+0000 7fefb6400640 1 --2- 172.21.15.28:0/557185982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb01467a0 0x7fefb00b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.669+0000 7fefb6400640 1 --2- 172.21.15.28:0/557185982 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb0073730 0x7fefb013e410 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.669+0000 7fefb6400640 1 --2- 172.21.15.28:0/557185982 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb006f020 0x7fefb013a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.669+0000 7fefb6400640 1 -- 172.21.15.28:0/557185982 >> 172.21.15.28:0/557185982 conn(0x7fefb006c770 msgr2=0x7fefb00ccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:06.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.669+0000 7fefb6400640 1 -- 172.21.15.28:0/557185982 shutdown_connections 2024-04-26T20:58:06.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.669+0000 7fefb6400640 1 -- 172.21.15.28:0/557185982 wait complete. 2024-04-26T20:58:06.690 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T20:58:06.690 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T20:58:06.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.906+0000 7fc5aea00640 1 Processor -- start 2024-04-26T20:58:06.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.906+0000 7fc5aea00640 1 -- start start 2024-04-26T20:58:06.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.907+0000 7fc5aea00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5a815f0f0 0x7fc5a80b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:06.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.908+0000 7fc5aea00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5a80b8420 con 0x7fc5a80b4140 2024-04-26T20:58:06.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.908+0000 7fc5aea00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5a80bade0 con 0x7fc5a80cfb00 2024-04-26T20:58:06.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.908+0000 7fc5aea00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a80d3bf0 con 0x7fc5a815f0f0 2024-04-26T20:58:06.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.908+0000 7fc5acc00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc5a80b4140 0x7fc5a80ba6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36556/0 (socket says 172.21.15.28:36556) 2024-04-26T20:58:06.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.908+0000 7fc5acc00640 1 -- 172.21.15.28:0/743318953 learned_addr learned my addr 172.21.15.28:0/743318953 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:06.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.908+0000 7fc5ad600640 1 --2- 172.21.15.28:0/743318953 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5a815f0f0 0x7fc5a80b7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:06.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.908+0000 7fc5a7e00640 1 -- 172.21.15.28:0/743318953 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1197254024 0 0) 0x7fc598001110 con 0x7fc5a80b4140 2024-04-26T20:58:06.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.908+0000 7fc5a7e00640 1 -- 172.21.15.28:0/743318953 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc590003520 con 0x7fc5a80b4140 2024-04-26T20:58:06.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.909+0000 7fc5a7e00640 1 -- 172.21.15.28:0/743318953 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1634368416 0 0) 0x7fc590003520 con 0x7fc5a80b4140 2024-04-26T20:58:06.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.909+0000 7fc5a7e00640 1 -- 172.21.15.28:0/743318953 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5a815f0f0 msgr2=0x7fc5a80b7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:06.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.909+0000 7fc5a7e00640 1 --2- 172.21.15.28:0/743318953 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5a815f0f0 0x7fc5a80b7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.909+0000 7fc5a7e00640 1 -- 172.21.15.28:0/743318953 >> v1:172.21.15.103:6789/0 conn(0x7fc5a80cfb00 legacy=0x7fc5a80d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:06.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.909+0000 7fc5a7e00640 1 -- 172.21.15.28:0/743318953 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a80d2fe0 con 0x7fc5a80b4140 2024-04-26T20:58:06.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.909+0000 7fc5a7e00640 1 -- 172.21.15.28:0/743318953 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fc598001840 con 0x7fc5a80b4140 2024-04-26T20:58:06.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.909+0000 7fc5a7e00640 1 -- 172.21.15.28:0/743318953 >> v1:172.21.15.28:6789/0 conn(0x7fc5a80b4140 legacy=0x7fc5a80ba6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:06.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.909+0000 7fc5a7e00640 1 --2- 172.21.15.28:0/743318953 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5a815f0f0 0x7fc590003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:06.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.909+0000 7fc5a7e00640 1 --2- 172.21.15.28:0/743318953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc590003fd0 0x7fc590004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:06.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.909+0000 7fc5a7e00640 1 --2- 172.21.15.28:0/743318953 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc590004d40 0x7fc5900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:06.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.909+0000 7fc5a7e00640 1 -- 172.21.15.28:0/743318953 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a80d3bf0 con 0x7fc590004d40 2024-04-26T20:58:06.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.909+0000 7fc5a7e00640 1 -- 172.21.15.28:0/743318953 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc59000bbb0 con 0x7fc590003fd0 2024-04-26T20:58:06.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.909+0000 7fc5a7e00640 1 -- 172.21.15.28:0/743318953 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc59000bdb0 con 0x7fc5a815f0f0 2024-04-26T20:58:06.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.909+0000 7fc5ae000640 1 --2- 172.21.15.28:0/743318953 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc590004d40 0x7fc5900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:06.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.910+0000 7fc5ad600640 1 --2- 172.21.15.28:0/743318953 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5a815f0f0 0x7fc590003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:06.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.910+0000 7fc5ae000640 1 -- 172.21.15.28:0/743318953 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5a815f0f0 msgr2=0x7fc590003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:06.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.910+0000 7fc5ae000640 1 --2- 172.21.15.28:0/743318953 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5a815f0f0 0x7fc590003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.910+0000 7fc5ae000640 1 -- 172.21.15.28:0/743318953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc590003fd0 msgr2=0x7fc590004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:06.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.910+0000 7fc5acc00640 1 --2- 172.21.15.28:0/743318953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc590003fd0 0x7fc590004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:06.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.910+0000 7fc5ae000640 1 --2- 172.21.15.28:0/743318953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc590003fd0 0x7fc590004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.910+0000 7fc5ae000640 1 -- 172.21.15.28:0/743318953 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc590003520 con 0x7fc590004d40 2024-04-26T20:58:06.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.910+0000 7fc5acc00640 1 --2- 172.21.15.28:0/743318953 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc590003fd0 0x7fc590004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:06.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.911+0000 7fc5ae000640 1 --2- 172.21.15.28:0/743318953 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc590004d40 0x7fc5900051c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0001e50 tx=0x7fc590005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1ffcbe1f94f97b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:06.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.911+0000 7fc5a7e00640 1 -- 172.21.15.28:0/743318953 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0001a70 con 0x7fc590004d40 2024-04-26T20:58:06.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.911+0000 7fc5a7e00640 1 -- 172.21.15.28:0/743318953 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a0003970 con 0x7fc590004d40 2024-04-26T20:58:06.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.911+0000 7fc5a7e00640 1 -- 172.21.15.28:0/743318953 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0003d60 con 0x7fc590004d40 2024-04-26T20:58:06.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.912+0000 7fc5aea00640 1 -- 172.21.15.28:0/743318953 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc590004d40 msgr2=0x7fc5900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:06.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.912+0000 7fc5aea00640 1 --2- 172.21.15.28:0/743318953 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc590004d40 0x7fc5900051c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0001e50 tx=0x7fc590005830 comp rx=0 tx=0).stop 2024-04-26T20:58:06.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.912+0000 7fc5ae000640 1 -- 172.21.15.28:0/743318953 reap_dead start 2024-04-26T20:58:06.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.912+0000 7fc5aea00640 1 -- 172.21.15.28:0/743318953 shutdown_connections 2024-04-26T20:58:06.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.912+0000 7fc5aea00640 1 -- 172.21.15.28:0/743318953 >> 172.21.15.28:0/743318953 conn(0x7fc5a806c980 msgr2=0x7fc5a8072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:06.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.912+0000 7fc5aea00640 1 -- 172.21.15.28:0/743318953 shutdown_connections 2024-04-26T20:58:06.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.912+0000 7fc5aea00640 1 -- 172.21.15.28:0/743318953 wait complete. 2024-04-26T20:58:06.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.914+0000 7fc5aea00640 1 Processor -- start 2024-04-26T20:58:06.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.914+0000 7fc5aea00640 1 -- start start 2024-04-26T20:58:06.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.915+0000 7fc5aea00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5a814e330 0x7fc5a80c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:06.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.915+0000 7fc5aea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5a80cea40 0x7fc5a80ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:06.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.915+0000 7fc5aea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5a81609f0 0x7fc5a8160e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:06.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.915+0000 7fc5aea00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a80d3880 con 0x7fc5a81609f0 2024-04-26T20:58:06.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.915+0000 7fc5aea00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a80d35a0 con 0x7fc5a80cea40 2024-04-26T20:58:06.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.915+0000 7fc5aea00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc5a80b7ee0 con 0x7fc5a814e330 2024-04-26T20:58:06.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.915+0000 7fc5ae000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5a81609f0 0x7fc5a8160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:06.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.915+0000 7fc5ae000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5a81609f0 0x7fc5a8160e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:32866/0 (socket says 172.21.15.28:32866) 2024-04-26T20:58:06.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.915+0000 7fc5ae000640 1 -- 172.21.15.28:0/2522846312 learned_addr learned my addr 172.21.15.28:0/2522846312 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:06.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.915+0000 7fc5acc00640 1 --2- 172.21.15.28:0/2522846312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5a80cea40 0x7fc5a80ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:06.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.915+0000 7fc5ad600640 1 --2- 172.21.15.28:0/2522846312 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5a814e330 0x7fc5a80c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:06.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.916+0000 7fc5ae000640 1 -- 172.21.15.28:0/2522846312 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5a814e330 msgr2=0x7fc5a80c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:06.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.916+0000 7fc5ae000640 1 --2- 172.21.15.28:0/2522846312 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5a814e330 0x7fc5a80c9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.916+0000 7fc5ae000640 1 -- 172.21.15.28:0/2522846312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5a80cea40 msgr2=0x7fc5a80ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:06.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.916+0000 7fc5ae000640 1 --2- 172.21.15.28:0/2522846312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5a80cea40 0x7fc5a80ceec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:06.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.916+0000 7fc5ae000640 1 -- 172.21.15.28:0/2522846312 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a80cc1e0 con 0x7fc5a81609f0 2024-04-26T20:58:06.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.916+0000 7fc5ae000640 1 --2- 172.21.15.28:0/2522846312 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5a81609f0 0x7fc5a8160e70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0001f40 tx=0x7fc5a00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:06.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.916+0000 7fc5a5600640 1 -- 172.21.15.28:0/2522846312 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0041070 con 0x7fc5a81609f0 2024-04-26T20:58:06.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.916+0000 7fc5a5600640 1 -- 172.21.15.28:0/2522846312 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5a0006d60 con 0x7fc5a81609f0 2024-04-26T20:58:06.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.916+0000 7fc5aea00640 1 -- 172.21.15.28:0/2522846312 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a81f8730 con 0x7fc5a81609f0 2024-04-26T20:58:06.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.916+0000 7fc5a5600640 1 -- 172.21.15.28:0/2522846312 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0001a70 con 0x7fc5a81609f0 2024-04-26T20:58:06.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.916+0000 7fc5aea00640 1 -- 172.21.15.28:0/2522846312 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a81f8cf0 con 0x7fc5a81609f0 2024-04-26T20:58:06.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.917+0000 7fc5aea00640 1 -- 172.21.15.28:0/2522846312 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a8050760 con 0x7fc5a81609f0 2024-04-26T20:58:06.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.919+0000 7fc5a5600640 1 -- 172.21.15.28:0/2522846312 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5a0044070 con 0x7fc5a81609f0 2024-04-26T20:58:06.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.920+0000 7fc5a5600640 1 --2- 172.21.15.28:0/2522846312 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc58c06c8f0 0x7fc58c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:06.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.920+0000 7fc5a5600640 1 -- 172.21.15.28:0/2522846312 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fc5a00aacb0 con 0x7fc5a81609f0 2024-04-26T20:58:06.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.920+0000 7fc5ad600640 1 --2- 172.21.15.28:0/2522846312 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc58c06c8f0 0x7fc58c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:06.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.921+0000 7fc5ad600640 1 --2- 172.21.15.28:0/2522846312 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc58c06c8f0 0x7fc58c06edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009950 tx=0x7fc59c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:06.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:06.924+0000 7fc5a5600640 1 -- 172.21.15.28:0/2522846312 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc5a8050760 con 0x7fc5a81609f0 2024-04-26T20:58:07.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.128+0000 7fc5aea00640 1 -- 172.21.15.28:0/2522846312 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc5a806a3d0 con 0x7fc5a81609f0 2024-04-26T20:58:07.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.727+0000 7fc5a5600640 1 -- 172.21.15.28:0/2522846312 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) ==== 138+0+0 (secure 0 0 0) 0x7fc5a80cc1e0 con 0x7fc5a81609f0 2024-04-26T20:58:07.729 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' created 2024-04-26T20:58:07.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.732+0000 7fc5aea00640 1 -- 172.21.15.28:0/2522846312 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc58c06c8f0 msgr2=0x7fc58c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:07.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.732+0000 7fc5aea00640 1 --2- 172.21.15.28:0/2522846312 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc58c06c8f0 0x7fc58c06edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009950 tx=0x7fc59c008040 comp rx=0 tx=0).stop 2024-04-26T20:58:07.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.732+0000 7fc5aea00640 1 -- 172.21.15.28:0/2522846312 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5a81609f0 msgr2=0x7fc5a8160e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:07.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.732+0000 7fc5aea00640 1 --2- 172.21.15.28:0/2522846312 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5a81609f0 0x7fc5a8160e70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0001f40 tx=0x7fc5a00014a0 comp rx=0 tx=0).stop 2024-04-26T20:58:07.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.732+0000 7fc5aea00640 1 -- 172.21.15.28:0/2522846312 shutdown_connections 2024-04-26T20:58:07.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.732+0000 7fc5aea00640 1 --2- 172.21.15.28:0/2522846312 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc58c06c8f0 0x7fc58c06edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:07.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.732+0000 7fc5aea00640 1 --2- 172.21.15.28:0/2522846312 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc5a81609f0 0x7fc5a8160e70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:07.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.732+0000 7fc5aea00640 1 --2- 172.21.15.28:0/2522846312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc5a80cea40 0x7fc5a80ceec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:07.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.732+0000 7fc5aea00640 1 --2- 172.21.15.28:0/2522846312 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc5a814e330 0x7fc5a80c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:07.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.732+0000 7fc5aea00640 1 -- 172.21.15.28:0/2522846312 >> 172.21.15.28:0/2522846312 conn(0x7fc5a806c980 msgr2=0x7fc5a80b5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:07.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.733+0000 7fc5aea00640 1 -- 172.21.15.28:0/2522846312 shutdown_connections 2024-04-26T20:58:07.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.733+0000 7fc5aea00640 1 -- 172.21.15.28:0/2522846312 wait complete. 2024-04-26T20:58:07.760 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T20:58:07.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.951+0000 7f453fe00640 1 Processor -- start 2024-04-26T20:58:07.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.952+0000 7f453fe00640 1 -- start start 2024-04-26T20:58:07.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.953+0000 7f453fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4540148a90 0x7f45400d1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:07.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.953+0000 7f453fe00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45400b74c0 0x7f45400d43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:07.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.954+0000 7f453fe00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45400d2010 con 0x7f45400b9730 2024-04-26T20:58:07.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.954+0000 7f453fe00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f45400b8b40 con 0x7f4540148a90 2024-04-26T20:58:07.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.954+0000 7f453fe00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f45400d22f0 con 0x7f45400b74c0 2024-04-26T20:58:07.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.954+0000 7f453ea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4540148a90 0x7f45400d1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:07.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.954+0000 7f453ea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4540148a90 0x7f45400d1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:32870/0 (socket says 172.21.15.28:32870) 2024-04-26T20:58:07.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.954+0000 7f453ea00640 1 -- 172.21.15.28:0/1433523922 learned_addr learned my addr 172.21.15.28:0/1433523922 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:07.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.954+0000 7f453f400640 1 --2- 172.21.15.28:0/1433523922 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45400b74c0 0x7f45400d43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:07.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.955+0000 7f453ea00640 1 -- 172.21.15.28:0/1433523922 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45400b74c0 msgr2=0x7f45400d43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:07.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.955+0000 7f453ea00640 1 --2- 172.21.15.28:0/1433523922 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45400b74c0 0x7f45400d43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:07.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.955+0000 7f453ea00640 1 -- 172.21.15.28:0/1433523922 >> v1:172.21.15.103:6789/0 conn(0x7f45400b9730 legacy=0x7f45400b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:07.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.955+0000 7f453ea00640 1 -- 172.21.15.28:0/1433523922 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45400d25d0 con 0x7f4540148a90 2024-04-26T20:58:07.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.956+0000 7f453ea00640 1 --2- 172.21.15.28:0/1433523922 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4540148a90 0x7f45400d1910 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4530009e50 tx=0x7f453002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9963cf0c6d7c47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:07.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.956+0000 7f453d600640 1 -- 172.21.15.28:0/1433523922 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530036070 con 0x7f4540148a90 2024-04-26T20:58:07.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.956+0000 7f453d600640 1 -- 172.21.15.28:0/1433523922 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4530031600 con 0x7f4540148a90 2024-04-26T20:58:07.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.956+0000 7f453d600640 1 -- 172.21.15.28:0/1433523922 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45300319d0 con 0x7f4540148a90 2024-04-26T20:58:07.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.957+0000 7f453fe00640 1 -- 172.21.15.28:0/1433523922 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4540148a90 msgr2=0x7f45400d1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:07.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.957+0000 7f453fe00640 1 --2- 172.21.15.28:0/1433523922 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4540148a90 0x7f45400d1910 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4530009e50 tx=0x7f453002a6c0 comp rx=0 tx=0).stop 2024-04-26T20:58:07.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.957+0000 7f453fe00640 1 -- 172.21.15.28:0/1433523922 shutdown_connections 2024-04-26T20:58:07.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.957+0000 7f453fe00640 1 --2- 172.21.15.28:0/1433523922 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f45400b74c0 0x7f45400d43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:07.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.957+0000 7f453fe00640 1 --2- 172.21.15.28:0/1433523922 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4540148a90 0x7f45400d1910 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:07.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.957+0000 7f453fe00640 1 -- 172.21.15.28:0/1433523922 >> 172.21.15.28:0/1433523922 conn(0x7f454006c530 msgr2=0x7f454006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:07.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.957+0000 7f453fe00640 1 -- 172.21.15.28:0/1433523922 shutdown_connections 2024-04-26T20:58:07.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.957+0000 7f453fe00640 1 -- 172.21.15.28:0/1433523922 wait complete. 2024-04-26T20:58:07.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.959+0000 7f453fe00640 1 Processor -- start 2024-04-26T20:58:07.960 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.959+0000 7f453fe00640 1 -- start start 2024-04-26T20:58:07.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.960+0000 7f453fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45400b74c0 0x7f454013cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:07.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.960+0000 7f453fe00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45400b9730 0x7f454013e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:07.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.960+0000 7f453fe00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4540148a90 0x7f4540143680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:07.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.960+0000 7f453fe00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4540148e70 con 0x7f45400b74c0 2024-04-26T20:58:07.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.960+0000 7f453fe00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45400d2b90 con 0x7f45400b9730 2024-04-26T20:58:07.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.960+0000 7f453fe00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f45400d48f0 con 0x7f4540148a90 2024-04-26T20:58:07.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.960+0000 7f453ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45400b74c0 0x7f454013cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:07.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.960+0000 7f453ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45400b74c0 0x7f454013cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:32882/0 (socket says 172.21.15.28:32882) 2024-04-26T20:58:07.961 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.960+0000 7f453ea00640 1 -- 172.21.15.28:0/2580185196 learned_addr learned my addr 172.21.15.28:0/2580185196 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:07.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.960+0000 7f453ea00640 1 -- 172.21.15.28:0/2580185196 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4540148a90 msgr2=0x7f4540143680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:07.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.960+0000 7f453ea00640 1 --2- 172.21.15.28:0/2580185196 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4540148a90 0x7f4540143680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:07.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.961+0000 7f453ea00640 1 -- 172.21.15.28:0/2580185196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45400b9730 msgr2=0x7f454013e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:07.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.961+0000 7f453ea00640 1 --2- 172.21.15.28:0/2580185196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45400b9730 0x7f454013e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:07.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.961+0000 7f453ea00640 1 -- 172.21.15.28:0/2580185196 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454013d900 con 0x7f45400b74c0 2024-04-26T20:58:07.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.961+0000 7f453ea00640 1 --2- 172.21.15.28:0/2580185196 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45400b74c0 0x7f454013cf20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4530034040 tx=0x7f4530004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:07.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.961+0000 7f4536a00640 1 -- 172.21.15.28:0/2580185196 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530036050 con 0x7f45400b74c0 2024-04-26T20:58:07.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.961+0000 7f4536a00640 1 -- 172.21.15.28:0/2580185196 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4530042070 con 0x7f45400b74c0 2024-04-26T20:58:07.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.961+0000 7f453fe00640 1 -- 172.21.15.28:0/2580185196 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f454013dbe0 con 0x7f45400b74c0 2024-04-26T20:58:07.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.961+0000 7f4536a00640 1 -- 172.21.15.28:0/2580185196 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530009510 con 0x7f45400b74c0 2024-04-26T20:58:07.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.961+0000 7f453fe00640 1 -- 172.21.15.28:0/2580185196 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f454013e1a0 con 0x7f45400b74c0 2024-04-26T20:58:07.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.963+0000 7f453fe00640 1 -- 172.21.15.28:0/2580185196 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f454006a2c0 con 0x7f45400b74c0 2024-04-26T20:58:07.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.964+0000 7f4536a00640 1 -- 172.21.15.28:0/2580185196 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4530007c50 con 0x7f45400b74c0 2024-04-26T20:58:07.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.965+0000 7f4536a00640 1 --2- 172.21.15.28:0/2580185196 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f451c06c8f0 0x7f451c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:07.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.965+0000 7f4536a00640 1 -- 172.21.15.28:0/2580185196 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5227+0+0 (secure 0 0 0) 0x7f45300acb30 con 0x7f45400b74c0 2024-04-26T20:58:07.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.965+0000 7f453e000640 1 --2- 172.21.15.28:0/2580185196 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f451c06c8f0 0x7f451c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:07.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.966+0000 7f453e000640 1 --2- 172.21.15.28:0/2580185196 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f451c06c8f0 0x7f451c06edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4540140a50 tx=0x7f45280079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:07.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:07.970+0000 7f4536a00640 1 -- 172.21.15.28:0/2580185196 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4530045360 con 0x7f45400b74c0 2024-04-26T20:58:08.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.177+0000 7f453fe00640 1 -- 172.21.15.28:0/2580185196 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4540068380 con 0x7f45400b74c0 2024-04-26T20:58:08.736 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.734+0000 7f4536a00640 1 -- 172.21.15.28:0/2580185196 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) ==== 170+0+0 (secure 0 0 0) 0x7f453007b0a0 con 0x7f45400b74c0 2024-04-26T20:58:08.736 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' created 2024-04-26T20:58:08.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.738+0000 7f453fe00640 1 -- 172.21.15.28:0/2580185196 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f451c06c8f0 msgr2=0x7f451c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:08.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.738+0000 7f453fe00640 1 --2- 172.21.15.28:0/2580185196 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f451c06c8f0 0x7f451c06edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4540140a50 tx=0x7f45280079b0 comp rx=0 tx=0).stop 2024-04-26T20:58:08.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.738+0000 7f453fe00640 1 -- 172.21.15.28:0/2580185196 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45400b74c0 msgr2=0x7f454013cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:08.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.738+0000 7f453fe00640 1 --2- 172.21.15.28:0/2580185196 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45400b74c0 0x7f454013cf20 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4530034040 tx=0x7f4530004a40 comp rx=0 tx=0).stop 2024-04-26T20:58:08.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.740+0000 7f453fe00640 1 -- 172.21.15.28:0/2580185196 shutdown_connections 2024-04-26T20:58:08.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.740+0000 7f453fe00640 1 --2- 172.21.15.28:0/2580185196 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f451c06c8f0 0x7f451c06edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:08.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.740+0000 7f453fe00640 1 --2- 172.21.15.28:0/2580185196 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4540148a90 0x7f4540143680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:08.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.740+0000 7f453fe00640 1 --2- 172.21.15.28:0/2580185196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45400b9730 0x7f454013e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:08.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.740+0000 7f453fe00640 1 --2- 172.21.15.28:0/2580185196 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45400b74c0 0x7f454013cf20 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:08.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.740+0000 7f453fe00640 1 -- 172.21.15.28:0/2580185196 >> 172.21.15.28:0/2580185196 conn(0x7f454006c530 msgr2=0x7f45401603f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:08.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.740+0000 7f453fe00640 1 -- 172.21.15.28:0/2580185196 shutdown_connections 2024-04-26T20:58:08.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.741+0000 7f453fe00640 1 -- 172.21.15.28:0/2580185196 wait complete. 2024-04-26T20:58:08.782 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T20:58:08.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.961+0000 7ff27a000640 1 Processor -- start 2024-04-26T20:58:08.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.961+0000 7ff27a000640 1 -- start start 2024-04-26T20:58:08.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.963+0000 7ff27a000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff2740d1420 0x7ff27406e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:08.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.963+0000 7ff27a000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2740b3ea0 con 0x7ff2740b4f40 2024-04-26T20:58:08.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.963+0000 7ff27a000640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2740b4180 con 0x7ff27415efd0 2024-04-26T20:58:08.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.963+0000 7ff27a000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff2740b4460 con 0x7ff2740d1420 2024-04-26T20:58:08.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.963+0000 7ff273e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff2740d1420 0x7ff27406e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:08.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.963+0000 7ff273e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff2740d1420 0x7ff27406e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:32888/0 (socket says 172.21.15.28:32888) 2024-04-26T20:58:08.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.963+0000 7ff273e00640 1 -- 172.21.15.28:0/1054493307 learned_addr learned my addr 172.21.15.28:0/1054493307 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:08.964 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.963+0000 7ff279600640 1 --1- 172.21.15.28:0/1054493307 >> v1:172.21.15.28:6789/0 conn(0x7ff2740b4f40 0x7ff27406e7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36568/0 (socket says 172.21.15.28:36568) 2024-04-26T20:58:08.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.963+0000 7ff273e00640 1 -- 172.21.15.28:0/1054493307 >> v1:172.21.15.103:6789/0 conn(0x7ff27415efd0 legacy=0x7ff2740d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:08.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.964+0000 7ff273e00640 1 -- 172.21.15.28:0/1054493307 >> v1:172.21.15.28:6789/0 conn(0x7ff2740b4f40 legacy=0x7ff27406e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:08.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.964+0000 7ff273e00640 1 -- 172.21.15.28:0/1054493307 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2740b4740 con 0x7ff2740d1420 2024-04-26T20:58:08.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.965+0000 7ff273e00640 1 --2- 172.21.15.28:0/1054493307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2740d1420 0x7ff27406e270 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff260009870 tx=0x7ff26002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7029ba15c67bc29 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:08.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.965+0000 7ff273400640 1 -- 172.21.15.28:0/1054493307 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260037070 con 0x7ff2740d1420 2024-04-26T20:58:08.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.965+0000 7ff273400640 1 -- 172.21.15.28:0/1054493307 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff260032440 con 0x7ff2740d1420 2024-04-26T20:58:08.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.965+0000 7ff273400640 1 -- 172.21.15.28:0/1054493307 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260032810 con 0x7ff2740d1420 2024-04-26T20:58:08.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.966+0000 7ff27a000640 1 -- 172.21.15.28:0/1054493307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2740d1420 msgr2=0x7ff27406e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:08.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.966+0000 7ff27a000640 1 --2- 172.21.15.28:0/1054493307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2740d1420 0x7ff27406e270 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff260009870 tx=0x7ff26002a2a0 comp rx=0 tx=0).stop 2024-04-26T20:58:08.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.966+0000 7ff27a000640 1 -- 172.21.15.28:0/1054493307 shutdown_connections 2024-04-26T20:58:08.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.966+0000 7ff27a000640 1 --2- 172.21.15.28:0/1054493307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2740d1420 0x7ff27406e270 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:08.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.966+0000 7ff27a000640 1 -- 172.21.15.28:0/1054493307 >> 172.21.15.28:0/1054493307 conn(0x7ff27406c530 msgr2=0x7ff27406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:08.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.966+0000 7ff27a000640 1 -- 172.21.15.28:0/1054493307 shutdown_connections 2024-04-26T20:58:08.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.966+0000 7ff27a000640 1 -- 172.21.15.28:0/1054493307 wait complete. 2024-04-26T20:58:08.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.968+0000 7ff27a000640 1 Processor -- start 2024-04-26T20:58:08.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.968+0000 7ff27a000640 1 -- start start 2024-04-26T20:58:08.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.969+0000 7ff27a000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2740b4f40 0x7ff274152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:08.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.969+0000 7ff27a000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff2740d1420 0x7ff27415b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:08.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.969+0000 7ff27a000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff27415efd0 0x7ff27415be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:08.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.969+0000 7ff27a000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff2740b56c0 con 0x7ff2740b4f40 2024-04-26T20:58:08.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.969+0000 7ff27a000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff27415f3b0 con 0x7ff27415efd0 2024-04-26T20:58:08.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.969+0000 7ff27a000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff2740b5840 con 0x7ff2740d1420 2024-04-26T20:58:08.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.969+0000 7ff278c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2740b4f40 0x7ff274152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:08.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.969+0000 7ff278c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2740b4f40 0x7ff274152080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:32900/0 (socket says 172.21.15.28:32900) 2024-04-26T20:58:08.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.969+0000 7ff278c00640 1 -- 172.21.15.28:0/3075600026 learned_addr learned my addr 172.21.15.28:0/3075600026 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:08.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.970+0000 7ff273e00640 1 --2- 172.21.15.28:0/3075600026 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff2740d1420 0x7ff27415b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:08.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.970+0000 7ff279600640 1 --2- 172.21.15.28:0/3075600026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff27415efd0 0x7ff27415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:08.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.970+0000 7ff278c00640 1 -- 172.21.15.28:0/3075600026 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff2740d1420 msgr2=0x7ff27415b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:08.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.970+0000 7ff278c00640 1 --2- 172.21.15.28:0/3075600026 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff2740d1420 0x7ff27415b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:08.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.970+0000 7ff278c00640 1 -- 172.21.15.28:0/3075600026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff27415efd0 msgr2=0x7ff27415be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:08.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.970+0000 7ff278c00640 1 --2- 172.21.15.28:0/3075600026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff27415efd0 0x7ff27415be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:08.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.970+0000 7ff278c00640 1 -- 172.21.15.28:0/3075600026 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff274152690 con 0x7ff2740b4f40 2024-04-26T20:58:08.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.970+0000 7ff273e00640 1 --2- 172.21.15.28:0/3075600026 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff2740d1420 0x7ff27415b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:08.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.970+0000 7ff278c00640 1 --2- 172.21.15.28:0/3075600026 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2740b4f40 0x7ff274152080 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff26800b3f0 tx=0x7ff26800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:08.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.970+0000 7ff270c00640 1 -- 172.21.15.28:0/3075600026 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268015020 con 0x7ff2740b4f40 2024-04-26T20:58:08.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.970+0000 7ff270c00640 1 -- 172.21.15.28:0/3075600026 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff268011830 con 0x7ff2740b4f40 2024-04-26T20:58:08.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.970+0000 7ff270c00640 1 -- 172.21.15.28:0/3075600026 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff268011c20 con 0x7ff2740b4f40 2024-04-26T20:58:08.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.970+0000 7ff27a000640 1 -- 172.21.15.28:0/3075600026 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff274152970 con 0x7ff2740b4f40 2024-04-26T20:58:08.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.971+0000 7ff27a000640 1 -- 172.21.15.28:0/3075600026 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff274152f30 con 0x7ff2740b4f40 2024-04-26T20:58:08.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.972+0000 7ff27a000640 1 -- 172.21.15.28:0/3075600026 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff274069ac0 con 0x7ff2740b4f40 2024-04-26T20:58:08.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.973+0000 7ff270c00640 1 -- 172.21.15.28:0/3075600026 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff268018020 con 0x7ff2740b4f40 2024-04-26T20:58:08.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.974+0000 7ff270c00640 1 --2- 172.21.15.28:0/3075600026 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff25406c8f0 0x7ff25406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:08.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.974+0000 7ff270c00640 1 -- 172.21.15.28:0/3075600026 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5609+0+0 (secure 0 0 0) 0x7ff26808b830 con 0x7ff2740b4f40 2024-04-26T20:58:08.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.974+0000 7ff273e00640 1 --2- 172.21.15.28:0/3075600026 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff25406c8f0 0x7ff25406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:08.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.975+0000 7ff273e00640 1 --2- 172.21.15.28:0/3075600026 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff25406c8f0 0x7ff25406edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff260034000 tx=0x7ff26002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:08.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:08.981+0000 7ff270c00640 1 -- 172.21.15.28:0/3075600026 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff274069ac0 con 0x7ff2740b4f40 2024-04-26T20:58:09.225 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:09.224+0000 7ff27a000640 1 -- 172.21.15.28:0/3075600026 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff274076780 con 0x7ff2740b4f40 2024-04-26T20:58:09.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:09.792+0000 7ff270c00640 1 -- 172.21.15.28:0/3075600026 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7ff274076780 con 0x7ff2740b4f40 2024-04-26T20:58:09.796 INFO:teuthology.orchestra.run.smithi028.stderr:new fs with metadata pool 4 and data pool 5 2024-04-26T20:58:09.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:09.798+0000 7ff27a000640 1 -- 172.21.15.28:0/3075600026 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff25406c8f0 msgr2=0x7ff25406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:09.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:09.798+0000 7ff27a000640 1 --2- 172.21.15.28:0/3075600026 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff25406c8f0 0x7ff25406edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff260034000 tx=0x7ff26002e000 comp rx=0 tx=0).stop 2024-04-26T20:58:09.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:09.798+0000 7ff27a000640 1 -- 172.21.15.28:0/3075600026 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2740b4f40 msgr2=0x7ff274152080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:09.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:09.798+0000 7ff27a000640 1 --2- 172.21.15.28:0/3075600026 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2740b4f40 0x7ff274152080 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff26800b3f0 tx=0x7ff26800b8b0 comp rx=0 tx=0).stop 2024-04-26T20:58:09.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:09.799+0000 7ff27a000640 1 -- 172.21.15.28:0/3075600026 shutdown_connections 2024-04-26T20:58:09.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:09.799+0000 7ff27a000640 1 --2- 172.21.15.28:0/3075600026 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff25406c8f0 0x7ff25406edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:09.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:09.799+0000 7ff27a000640 1 --2- 172.21.15.28:0/3075600026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff27415efd0 0x7ff27415be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:09.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:09.799+0000 7ff27a000640 1 --2- 172.21.15.28:0/3075600026 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff2740d1420 0x7ff27415b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:09.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:09.799+0000 7ff27a000640 1 --2- 172.21.15.28:0/3075600026 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2740b4f40 0x7ff274152080 secure :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff26800b3f0 tx=0x7ff26800b8b0 comp rx=0 tx=0).stop 2024-04-26T20:58:09.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:09.799+0000 7ff27a000640 1 -- 172.21.15.28:0/3075600026 >> 172.21.15.28:0/3075600026 conn(0x7ff27406c530 msgr2=0x7ff2740b2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:09.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:09.800+0000 7ff27a000640 1 -- 172.21.15.28:0/3075600026 shutdown_connections 2024-04-26T20:58:09.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:09.800+0000 7ff27a000640 1 -- 172.21.15.28:0/3075600026 wait complete. 2024-04-26T20:58:09.830 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:58:10.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.003+0000 7f6ca7200640 1 Processor -- start 2024-04-26T20:58:10.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.004+0000 7f6ca7200640 1 -- start start 2024-04-26T20:58:10.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.005+0000 7f6ca7200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6ca00b6810 0x7f6ca015d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.005+0000 7f6ca7200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f6ca00740e0 0x7f6ca015daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.005+0000 7f6ca7200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ca00d0a30 con 0x7f6ca0146980 2024-04-26T20:58:10.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.005+0000 7f6ca7200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f6ca015e510 con 0x7f6ca00b6810 2024-04-26T20:58:10.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.005+0000 7f6ca7200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f6ca015e230 con 0x7f6ca00740e0 2024-04-26T20:58:10.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.006+0000 7f6c97400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6ca00b6810 0x7f6ca015d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.006+0000 7f6c97400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6ca00b6810 0x7f6ca015d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51852/0 (socket says 172.21.15.28:51852) 2024-04-26T20:58:10.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.006+0000 7f6c97400640 1 -- 172.21.15.28:0/510626554 learned_addr learned my addr 172.21.15.28:0/510626554 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:10.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.006+0000 7f6ca5200640 1 --2- 172.21.15.28:0/510626554 >> v2:172.21.15.103:3300/0 conn(0x7f6ca00740e0 0x7f6ca015daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.006+0000 7f6c97400640 1 -- 172.21.15.28:0/510626554 >> v2:172.21.15.103:3300/0 conn(0x7f6ca00740e0 msgr2=0x7f6ca015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:10.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.006+0000 7f6c97400640 1 --2- 172.21.15.28:0/510626554 >> v2:172.21.15.103:3300/0 conn(0x7f6ca00740e0 0x7f6ca015daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.006+0000 7f6c97400640 1 -- 172.21.15.28:0/510626554 >> v1:172.21.15.103:6789/0 conn(0x7f6ca0146980 legacy=0x7f6ca00b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:10.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.006+0000 7f6c97400640 1 -- 172.21.15.28:0/510626554 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca0073600 con 0x7f6ca00b6810 2024-04-26T20:58:10.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.006+0000 7f6ca5200640 1 --2- 172.21.15.28:0/510626554 >> v2:172.21.15.103:3300/0 conn(0x7f6ca00740e0 0x7f6ca015daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:10.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.007+0000 7f6c97400640 1 --2- 172.21.15.28:0/510626554 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ca00b6810 0x7f6ca015d5b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0069090 tx=0x7f6c8802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9829b565540a3ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.008+0000 7f6c9f400640 1 -- 172.21.15.28:0/510626554 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c88037070 con 0x7f6ca00b6810 2024-04-26T20:58:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.008+0000 7f6c9f400640 1 -- 172.21.15.28:0/510626554 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c88032440 con 0x7f6ca00b6810 2024-04-26T20:58:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.008+0000 7f6c9f400640 1 -- 172.21.15.28:0/510626554 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c88032810 con 0x7f6ca00b6810 2024-04-26T20:58:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.008+0000 7f6ca7200640 1 -- 172.21.15.28:0/510626554 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ca00b6810 msgr2=0x7f6ca015d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.008+0000 7f6ca7200640 1 --2- 172.21.15.28:0/510626554 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ca00b6810 0x7f6ca015d5b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0069090 tx=0x7f6c8802a1a0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.008+0000 7f6ca7200640 1 -- 172.21.15.28:0/510626554 shutdown_connections 2024-04-26T20:58:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.008+0000 7f6ca7200640 1 --2- 172.21.15.28:0/510626554 >> v2:172.21.15.103:3300/0 conn(0x7f6ca00740e0 0x7f6ca015daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.008+0000 7f6ca7200640 1 --2- 172.21.15.28:0/510626554 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ca00b6810 0x7f6ca015d5b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.008+0000 7f6ca7200640 1 -- 172.21.15.28:0/510626554 >> 172.21.15.28:0/510626554 conn(0x7f6ca006e220 msgr2=0x7f6ca006e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.008+0000 7f6ca7200640 1 -- 172.21.15.28:0/510626554 shutdown_connections 2024-04-26T20:58:10.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.009+0000 7f6ca7200640 1 -- 172.21.15.28:0/510626554 wait complete. 2024-04-26T20:58:10.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.010+0000 7f6ca7200640 1 Processor -- start 2024-04-26T20:58:10.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.010+0000 7f6ca7200640 1 -- start start 2024-04-26T20:58:10.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.011+0000 7f6ca7200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ca00740e0 0x7f6ca00afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.011+0000 7f6ca7200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ca00b6810 0x7f6ca013e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.011+0000 7f6ca7200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ca0146980 0x7f6ca00b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.011+0000 7f6ca7200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca0071ff0 con 0x7f6ca0146980 2024-04-26T20:58:10.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.011+0000 7f6ca7200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca0073bc0 con 0x7f6ca00740e0 2024-04-26T20:58:10.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.011+0000 7f6ca7200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca0072170 con 0x7f6ca00b6810 2024-04-26T20:58:10.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.012+0000 7f6ca5200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ca0146980 0x7f6ca00b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.012+0000 7f6ca5200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ca0146980 0x7f6ca00b3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51866/0 (socket says 172.21.15.28:51866) 2024-04-26T20:58:10.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.012+0000 7f6ca5200640 1 -- 172.21.15.28:0/3413840952 learned_addr learned my addr 172.21.15.28:0/3413840952 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:10.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.012+0000 7f6c9fe00640 1 --2- 172.21.15.28:0/3413840952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ca00740e0 0x7f6ca00afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.012+0000 7f6c97400640 1 --2- 172.21.15.28:0/3413840952 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ca00b6810 0x7f6ca013e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.012+0000 7f6ca5200640 1 -- 172.21.15.28:0/3413840952 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ca00b6810 msgr2=0x7f6ca013e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:10.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.012+0000 7f6ca5200640 1 --2- 172.21.15.28:0/3413840952 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ca00b6810 0x7f6ca013e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.012+0000 7f6ca5200640 1 -- 172.21.15.28:0/3413840952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ca00740e0 msgr2=0x7f6ca00afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:10.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.012+0000 7f6ca5200640 1 --2- 172.21.15.28:0/3413840952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ca00740e0 0x7f6ca00afd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.012+0000 7f6ca5200640 1 -- 172.21.15.28:0/3413840952 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca015a060 con 0x7f6ca0146980 2024-04-26T20:58:10.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.013+0000 7f6ca5200640 1 --2- 172.21.15.28:0/3413840952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ca0146980 0x7f6ca00b3ef0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800cf20 tx=0x7f6c9800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:10.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.013+0000 7f6c9cc00640 1 -- 172.21.15.28:0/3413840952 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9801a070 con 0x7f6ca0146980 2024-04-26T20:58:10.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.013+0000 7f6c9cc00640 1 -- 172.21.15.28:0/3413840952 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c9800d840 con 0x7f6ca0146980 2024-04-26T20:58:10.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.013+0000 7f6ca7200640 1 -- 172.21.15.28:0/3413840952 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca00b4b00 con 0x7f6ca0146980 2024-04-26T20:58:10.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.013+0000 7f6c9cc00640 1 -- 172.21.15.28:0/3413840952 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9800dc30 con 0x7f6ca0146980 2024-04-26T20:58:10.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.013+0000 7f6ca7200640 1 -- 172.21.15.28:0/3413840952 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca015a2d0 con 0x7f6ca0146980 2024-04-26T20:58:10.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.014+0000 7f6ca7200640 1 -- 172.21.15.28:0/3413840952 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca006a400 con 0x7f6ca0146980 2024-04-26T20:58:10.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.015+0000 7f6c9cc00640 1 -- 172.21.15.28:0/3413840952 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c9801d070 con 0x7f6ca0146980 2024-04-26T20:58:10.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.016+0000 7f6c9cc00640 1 --2- 172.21.15.28:0/3413840952 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6c7c06c8f0 0x7f6c7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.016+0000 7f6c9cc00640 1 -- 172.21.15.28:0/3413840952 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f6c9808c410 con 0x7f6ca0146980 2024-04-26T20:58:10.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.017+0000 7f6c9fe00640 1 --2- 172.21.15.28:0/3413840952 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6c7c06c8f0 0x7f6c7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.017+0000 7f6c9fe00640 1 --2- 172.21.15.28:0/3413840952 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6c7c06c8f0 0x7f6c7c06edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6c90006fd0 tx=0x7f6c90008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:10.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.021+0000 7f6c9cc00640 1 -- 172.21.15.28:0/3413840952 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6c9805a840 con 0x7f6ca0146980 2024-04-26T20:58:10.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.216+0000 7f6ca7200640 1 -- 172.21.15.28:0/3413840952 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ca015c760 con 0x7f6ca0146980 2024-04-26T20:58:10.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.219+0000 7f6c9cc00640 1 -- 172.21.15.28:0/3413840952 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16658 (secure 0 0 0) 0x7f6c9805a140 con 0x7f6ca0146980 2024-04-26T20:58:10.220 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:10.220 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":33,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:58:09.226423+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:58:07.143890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:58:08.193405+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:58:10.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.221+0000 7f6ca7200640 1 -- 172.21.15.28:0/3413840952 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6c7c06c8f0 msgr2=0x7f6c7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:10.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.221+0000 7f6ca7200640 1 --2- 172.21.15.28:0/3413840952 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6c7c06c8f0 0x7f6c7c06edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6c90006fd0 tx=0x7f6c90008040 comp rx=0 tx=0).stop 2024-04-26T20:58:10.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.221+0000 7f6ca7200640 1 -- 172.21.15.28:0/3413840952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ca0146980 msgr2=0x7f6ca00b3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:10.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.221+0000 7f6ca7200640 1 --2- 172.21.15.28:0/3413840952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ca0146980 0x7f6ca00b3ef0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800cf20 tx=0x7f6c9800a7d0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.221+0000 7f6ca7200640 1 -- 172.21.15.28:0/3413840952 shutdown_connections 2024-04-26T20:58:10.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.221+0000 7f6ca7200640 1 --2- 172.21.15.28:0/3413840952 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6c7c06c8f0 0x7f6c7c06edb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.221+0000 7f6ca7200640 1 --2- 172.21.15.28:0/3413840952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6ca0146980 0x7f6ca00b3ef0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.221+0000 7f6ca7200640 1 --2- 172.21.15.28:0/3413840952 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6ca00b6810 0x7f6ca013e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.221+0000 7f6ca7200640 1 --2- 172.21.15.28:0/3413840952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6ca00740e0 0x7f6ca00afd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.221+0000 7f6ca7200640 1 -- 172.21.15.28:0/3413840952 >> 172.21.15.28:0/3413840952 conn(0x7f6ca006e220 msgr2=0x7f6ca00b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:10.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.221+0000 7f6ca7200640 1 -- 172.21.15.28:0/3413840952 shutdown_connections 2024-04-26T20:58:10.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.222+0000 7f6ca7200640 1 -- 172.21.15.28:0/3413840952 wait complete. 2024-04-26T20:58:10.241 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:58:10.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.442+0000 7fb327e00640 1 Processor -- start 2024-04-26T20:58:10.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.442+0000 7fb327e00640 1 -- start start 2024-04-26T20:58:10.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.443+0000 7fb327e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fb320073990 0x7fb320073d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.444+0000 7fb327e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb32015c430 0x7fb3200b5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.444+0000 7fb327e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb3200b5790 0x7fb3200b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.444+0000 7fb327e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb3200b41b0 con 0x7fb32015c430 2024-04-26T20:58:10.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.444+0000 7fb327e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fb3200b4490 con 0x7fb320073990 2024-04-26T20:58:10.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.444+0000 7fb327e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb3200b4770 con 0x7fb3200b5790 2024-04-26T20:58:10.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.444+0000 7fb324a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb32015c430 0x7fb3200b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.444+0000 7fb324a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb32015c430 0x7fb3200b5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51878/0 (socket says 172.21.15.28:51878) 2024-04-26T20:58:10.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.444+0000 7fb324a00640 1 -- 172.21.15.28:0/2181551968 learned_addr learned my addr 172.21.15.28:0/2181551968 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:10.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.444+0000 7fb325400640 1 --2- 172.21.15.28:0/2181551968 >> v2:172.21.15.103:3300/0 conn(0x7fb320073990 0x7fb320073d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.444+0000 7fb325e00640 1 --2- 172.21.15.28:0/2181551968 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb3200b5790 0x7fb3200b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.445+0000 7fb324a00640 1 -- 172.21.15.28:0/2181551968 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb3200b5790 msgr2=0x7fb3200b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:10.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.445+0000 7fb324a00640 1 --2- 172.21.15.28:0/2181551968 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb3200b5790 0x7fb3200b5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.445+0000 7fb324a00640 1 -- 172.21.15.28:0/2181551968 >> v2:172.21.15.103:3300/0 conn(0x7fb320073990 msgr2=0x7fb320073d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:10.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.445+0000 7fb324a00640 1 --2- 172.21.15.28:0/2181551968 >> v2:172.21.15.103:3300/0 conn(0x7fb320073990 0x7fb320073d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.445+0000 7fb324a00640 1 -- 172.21.15.28:0/2181551968 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3200b4a50 con 0x7fb32015c430 2024-04-26T20:58:10.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.445+0000 7fb325e00640 1 --2- 172.21.15.28:0/2181551968 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb3200b5790 0x7fb3200b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:10.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.446+0000 7fb324a00640 1 --2- 172.21.15.28:0/2181551968 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb32015c430 0x7fb3200b5250 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb320074510 tx=0x7fb30c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed2b6d8fc7cb6a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:10.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.446+0000 7fb31fe00640 1 -- 172.21.15.28:0/2181551968 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c037070 con 0x7fb32015c430 2024-04-26T20:58:10.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.446+0000 7fb31fe00640 1 -- 172.21.15.28:0/2181551968 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb30c032440 con 0x7fb32015c430 2024-04-26T20:58:10.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.446+0000 7fb31fe00640 1 -- 172.21.15.28:0/2181551968 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c032810 con 0x7fb32015c430 2024-04-26T20:58:10.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.447+0000 7fb327e00640 1 -- 172.21.15.28:0/2181551968 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb32015c430 msgr2=0x7fb3200b5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:10.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.447+0000 7fb327e00640 1 --2- 172.21.15.28:0/2181551968 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb32015c430 0x7fb3200b5250 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb320074510 tx=0x7fb30c02a140 comp rx=0 tx=0).stop 2024-04-26T20:58:10.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.447+0000 7fb327e00640 1 -- 172.21.15.28:0/2181551968 shutdown_connections 2024-04-26T20:58:10.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.447+0000 7fb327e00640 1 --2- 172.21.15.28:0/2181551968 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb3200b5790 0x7fb3200b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.447+0000 7fb327e00640 1 --2- 172.21.15.28:0/2181551968 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb32015c430 0x7fb3200b5250 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.447+0000 7fb327e00640 1 --2- 172.21.15.28:0/2181551968 >> v2:172.21.15.103:3300/0 conn(0x7fb320073990 0x7fb320073d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.447+0000 7fb327e00640 1 -- 172.21.15.28:0/2181551968 >> 172.21.15.28:0/2181551968 conn(0x7fb32006e220 msgr2=0x7fb32006e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:10.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.447+0000 7fb327e00640 1 -- 172.21.15.28:0/2181551968 shutdown_connections 2024-04-26T20:58:10.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.447+0000 7fb327e00640 1 -- 172.21.15.28:0/2181551968 wait complete. 2024-04-26T20:58:10.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.449+0000 7fb327e00640 1 Processor -- start 2024-04-26T20:58:10.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.449+0000 7fb327e00640 1 -- start start 2024-04-26T20:58:10.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.450+0000 7fb327e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb320073990 0x7fb3200aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.450+0000 7fb327e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb3200b5790 0x7fb320154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.450+0000 7fb327e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb320139110 0x7fb3200af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.450+0000 7fb327e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb3200b6160 con 0x7fb320139110 2024-04-26T20:58:10.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.450+0000 7fb327e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb3200742b0 con 0x7fb320073990 2024-04-26T20:58:10.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.450+0000 7fb327e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb320072220 con 0x7fb3200b5790 2024-04-26T20:58:10.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.450+0000 7fb325e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb320139110 0x7fb3200af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.450+0000 7fb325e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb320139110 0x7fb3200af110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51892/0 (socket says 172.21.15.28:51892) 2024-04-26T20:58:10.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.450+0000 7fb325e00640 1 -- 172.21.15.28:0/808767174 learned_addr learned my addr 172.21.15.28:0/808767174 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:10.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.451+0000 7fb324a00640 1 --2- 172.21.15.28:0/808767174 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb3200b5790 0x7fb320154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.451+0000 7fb325400640 1 --2- 172.21.15.28:0/808767174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb320073990 0x7fb3200aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.451+0000 7fb325e00640 1 -- 172.21.15.28:0/808767174 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb3200b5790 msgr2=0x7fb320154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:10.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.451+0000 7fb325e00640 1 --2- 172.21.15.28:0/808767174 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb3200b5790 0x7fb320154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.451+0000 7fb325e00640 1 -- 172.21.15.28:0/808767174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb320073990 msgr2=0x7fb3200aa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:10.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.451+0000 7fb325e00640 1 --2- 172.21.15.28:0/808767174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb320073990 0x7fb3200aa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.451+0000 7fb325e00640 1 -- 172.21.15.28:0/808767174 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3200ae350 con 0x7fb320139110 2024-04-26T20:58:10.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.451+0000 7fb325400640 1 --2- 172.21.15.28:0/808767174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb320073990 0x7fb3200aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.451+0000 7fb324a00640 1 --2- 172.21.15.28:0/808767174 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb3200b5790 0x7fb320154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:58:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.451+0000 7fb325e00640 1 --2- 172.21.15.28:0/808767174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb320139110 0x7fb3200af110 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb31800cef0 tx=0x7fb31800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.451+0000 7fb31d600640 1 -- 172.21.15.28:0/808767174 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31801a070 con 0x7fb320139110 2024-04-26T20:58:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.451+0000 7fb31d600640 1 -- 172.21.15.28:0/808767174 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb31800d860 con 0x7fb320139110 2024-04-26T20:58:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.451+0000 7fb31d600640 1 -- 172.21.15.28:0/808767174 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31800dc50 con 0x7fb320139110 2024-04-26T20:58:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.451+0000 7fb327e00640 1 -- 172.21.15.28:0/808767174 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3200ae630 con 0x7fb320139110 2024-04-26T20:58:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.452+0000 7fb327e00640 1 -- 172.21.15.28:0/808767174 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3200aebf0 con 0x7fb320139110 2024-04-26T20:58:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.453+0000 7fb327e00640 1 -- 172.21.15.28:0/808767174 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb320138f10 con 0x7fb320139110 2024-04-26T20:58:10.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.454+0000 7fb31d600640 1 -- 172.21.15.28:0/808767174 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb31801d070 con 0x7fb320139110 2024-04-26T20:58:10.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.455+0000 7fb31d600640 1 --2- 172.21.15.28:0/808767174 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb30006c8f0 0x7fb30006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.455+0000 7fb31d600640 1 -- 172.21.15.28:0/808767174 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb31808c4c0 con 0x7fb320139110 2024-04-26T20:58:10.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.455+0000 7fb325400640 1 --2- 172.21.15.28:0/808767174 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb30006c8f0 0x7fb30006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.456+0000 7fb325400640 1 --2- 172.21.15.28:0/808767174 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb30006c8f0 0x7fb30006edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb314006fd0 tx=0x7fb314008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:10.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.460+0000 7fb31d600640 1 -- 172.21.15.28:0/808767174 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb320138f10 con 0x7fb320139110 2024-04-26T20:58:10.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.663+0000 7fb327e00640 1 -- 172.21.15.28:0/808767174 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb32006a400 con 0x7fb320139110 2024-04-26T20:58:10.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.666+0000 7fb31d600640 1 -- 172.21.15.28:0/808767174 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16658 (secure 0 0 0) 0x7fb31805a910 con 0x7fb320139110 2024-04-26T20:58:10.668 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:10.668 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":33,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:58:09.226423+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:58:07.143890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:58:08.193405+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:58:10.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.669+0000 7fb327e00640 1 -- 172.21.15.28:0/808767174 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb30006c8f0 msgr2=0x7fb30006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:10.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.669+0000 7fb327e00640 1 --2- 172.21.15.28:0/808767174 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb30006c8f0 0x7fb30006edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb314006fd0 tx=0x7fb314008040 comp rx=0 tx=0).stop 2024-04-26T20:58:10.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.669+0000 7fb327e00640 1 -- 172.21.15.28:0/808767174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb320139110 msgr2=0x7fb3200af110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:10.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.669+0000 7fb327e00640 1 --2- 172.21.15.28:0/808767174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb320139110 0x7fb3200af110 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb31800cef0 tx=0x7fb31800a710 comp rx=0 tx=0).stop 2024-04-26T20:58:10.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.669+0000 7fb327e00640 1 -- 172.21.15.28:0/808767174 shutdown_connections 2024-04-26T20:58:10.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.669+0000 7fb327e00640 1 --2- 172.21.15.28:0/808767174 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb30006c8f0 0x7fb30006edb0 secure :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb314006fd0 tx=0x7fb314008040 comp rx=0 tx=0).stop 2024-04-26T20:58:10.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.669+0000 7fb327e00640 1 --2- 172.21.15.28:0/808767174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb320139110 0x7fb3200af110 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.669+0000 7fb327e00640 1 --2- 172.21.15.28:0/808767174 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb3200b5790 0x7fb320154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.669+0000 7fb327e00640 1 --2- 172.21.15.28:0/808767174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb320073990 0x7fb3200aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.669+0000 7fb327e00640 1 -- 172.21.15.28:0/808767174 >> 172.21.15.28:0/808767174 conn(0x7fb32006e220 msgr2=0x7fb32006e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:10.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.669+0000 7fb327e00640 1 -- 172.21.15.28:0/808767174 shutdown_connections 2024-04-26T20:58:10.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.669+0000 7fb327e00640 1 -- 172.21.15.28:0/808767174 wait complete. 2024-04-26T20:58:10.689 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T20:58:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.890+0000 7f5417e00640 1 Processor -- start 2024-04-26T20:58:10.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.890+0000 7f5417e00640 1 -- start start 2024-04-26T20:58:10.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.892+0000 7f5417e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f541815f360 0x7f54180b7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.892+0000 7f5417e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f54180bac00 0x7f54180b8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.892+0000 7f5417e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54180ba350 con 0x7f54180d1de0 2024-04-26T20:58:10.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.892+0000 7f5417e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f54180b9ab0 con 0x7f541815f360 2024-04-26T20:58:10.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.892+0000 7f5417e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f54180b9d90 con 0x7f54180bac00 2024-04-26T20:58:10.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.892+0000 7f5416a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f54180d1de0 0x7f54180d21c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50094/0 (socket says 172.21.15.28:50094) 2024-04-26T20:58:10.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.892+0000 7f5416a00640 1 -- 172.21.15.28:0/1091551025 learned_addr learned my addr 172.21.15.28:0/1091551025 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:10.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.893+0000 7f540fe00640 1 --2- 172.21.15.28:0/1091551025 >> v2:172.21.15.103:3300/0 conn(0x7f541815f360 0x7f54180b7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.893+0000 7f5417400640 1 --2- 172.21.15.28:0/1091551025 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f54180bac00 0x7f54180b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.893+0000 7f5416000640 1 -- 172.21.15.28:0/1091551025 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2223064140 0 0) 0x7f54180ba350 con 0x7f54180d1de0 2024-04-26T20:58:10.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.893+0000 7f5416000640 1 -- 172.21.15.28:0/1091551025 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53fc003520 con 0x7f54180d1de0 2024-04-26T20:58:10.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.893+0000 7f5416000640 1 -- 172.21.15.28:0/1091551025 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 534787597 0 0) 0x7f53fc003520 con 0x7f54180d1de0 2024-04-26T20:58:10.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.893+0000 7f5416000640 1 -- 172.21.15.28:0/1091551025 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f54180bac00 msgr2=0x7f54180b8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:10.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.893+0000 7f5416000640 1 --2- 172.21.15.28:0/1091551025 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f54180bac00 0x7f54180b8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.893+0000 7f5416000640 1 -- 172.21.15.28:0/1091551025 >> v2:172.21.15.103:3300/0 conn(0x7f541815f360 msgr2=0x7f54180b7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:10.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.893+0000 7f5416000640 1 --2- 172.21.15.28:0/1091551025 >> v2:172.21.15.103:3300/0 conn(0x7f541815f360 0x7f54180b7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.893+0000 7f5416000640 1 -- 172.21.15.28:0/1091551025 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54180ba070 con 0x7f54180d1de0 2024-04-26T20:58:10.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.893+0000 7f5416000640 1 -- 172.21.15.28:0/1091551025 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f5408001670 con 0x7f54180d1de0 2024-04-26T20:58:10.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.893+0000 7f5416000640 1 -- 172.21.15.28:0/1091551025 >> v1:172.21.15.28:6789/0 conn(0x7f54180d1de0 legacy=0x7f54180d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:10.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.893+0000 7f5416000640 1 --2- 172.21.15.28:0/1091551025 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53fc0039b0 0x7f53fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.893+0000 7f5416000640 1 --2- 172.21.15.28:0/1091551025 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f54180bac00 0x7f53fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.893+0000 7f5416000640 1 --2- 172.21.15.28:0/1091551025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53fc004d40 0x7f53fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.893+0000 7f5416000640 1 -- 172.21.15.28:0/1091551025 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f54180b9ab0 con 0x7f53fc0039b0 2024-04-26T20:58:10.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.893+0000 7f5416000640 1 -- 172.21.15.28:0/1091551025 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f54180b9d90 con 0x7f53fc004d40 2024-04-26T20:58:10.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.893+0000 7f5416000640 1 -- 172.21.15.28:0/1091551025 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f53fc00b9c0 con 0x7f54180bac00 2024-04-26T20:58:10.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.894+0000 7f5416a00640 1 --2- 172.21.15.28:0/1091551025 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53fc0039b0 0x7f53fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.894+0000 7f5417400640 1 --2- 172.21.15.28:0/1091551025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53fc004d40 0x7f53fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.894+0000 7f5416a00640 1 -- 172.21.15.28:0/1091551025 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f54180bac00 msgr2=0x7f53fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:10.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.894+0000 7f5416a00640 1 --2- 172.21.15.28:0/1091551025 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f54180bac00 0x7f53fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.894+0000 7f5416a00640 1 -- 172.21.15.28:0/1091551025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53fc004d40 msgr2=0x7f53fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:10.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.894+0000 7f5416a00640 1 --2- 172.21.15.28:0/1091551025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53fc004d40 0x7f53fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.894+0000 7f5416a00640 1 -- 172.21.15.28:0/1091551025 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53fc003520 con 0x7f53fc0039b0 2024-04-26T20:58:10.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.894+0000 7f5417400640 1 --2- 172.21.15.28:0/1091551025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53fc004d40 0x7f53fc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:10.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.895+0000 7f5416a00640 1 --2- 172.21.15.28:0/1091551025 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53fc0039b0 0x7f53fc003dd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f540800c240 tx=0x7f53fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ef6af232f2ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:10.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.895+0000 7f5416000640 1 -- 172.21.15.28:0/1091551025 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5408007d50 con 0x7f53fc0039b0 2024-04-26T20:58:10.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.895+0000 7f5416000640 1 -- 172.21.15.28:0/1091551025 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f540802e070 con 0x7f53fc0039b0 2024-04-26T20:58:10.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.895+0000 7f5416000640 1 -- 172.21.15.28:0/1091551025 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5408005d30 con 0x7f53fc0039b0 2024-04-26T20:58:10.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.896+0000 7f5417e00640 1 -- 172.21.15.28:0/1091551025 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53fc0039b0 msgr2=0x7f53fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.896+0000 7f5417e00640 1 --2- 172.21.15.28:0/1091551025 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53fc0039b0 0x7f53fc003dd0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f540800c240 tx=0x7f53fc005830 comp rx=0 tx=0).stop 2024-04-26T20:58:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.896+0000 7f5417400640 1 -- 172.21.15.28:0/1091551025 reap_dead start 2024-04-26T20:58:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.896+0000 7f5417e00640 1 -- 172.21.15.28:0/1091551025 shutdown_connections 2024-04-26T20:58:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.896+0000 7f5417e00640 1 -- 172.21.15.28:0/1091551025 >> 172.21.15.28:0/1091551025 conn(0x7f541806c770 msgr2=0x7f541806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.896+0000 7f5417e00640 1 -- 172.21.15.28:0/1091551025 shutdown_connections 2024-04-26T20:58:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.896+0000 7f5417e00640 1 -- 172.21.15.28:0/1091551025 wait complete. 2024-04-26T20:58:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.898+0000 7f5417e00640 1 Processor -- start 2024-04-26T20:58:10.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.898+0000 7f5417e00640 1 -- start start 2024-04-26T20:58:10.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.899+0000 7f5417e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f53fc0039b0 0x7f54180cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.899+0000 7f5417e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54181f7fb0 0x7f54181616e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.899+0000 7f5417e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54180cc4a0 0x7f5418161c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.899+0000 7f5417e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f54180d4620 con 0x7f54180cc4a0 2024-04-26T20:58:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.899+0000 7f5417e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f54180d44a0 con 0x7f54181f7fb0 2024-04-26T20:58:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.899+0000 7f5417e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f54180d47a0 con 0x7f53fc0039b0 2024-04-26T20:58:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.899+0000 7f5417400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54180cc4a0 0x7f5418161c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.899+0000 7f5417400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54180cc4a0 0x7f5418161c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51920/0 (socket says 172.21.15.28:51920) 2024-04-26T20:58:10.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.899+0000 7f5417400640 1 -- 172.21.15.28:0/3539496549 learned_addr learned my addr 172.21.15.28:0/3539496549 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.900+0000 7f5416a00640 1 --2- 172.21.15.28:0/3539496549 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f53fc0039b0 0x7f54180cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.900+0000 7f540fe00640 1 --2- 172.21.15.28:0/3539496549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54181f7fb0 0x7f54181616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.900+0000 7f5417400640 1 -- 172.21.15.28:0/3539496549 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f53fc0039b0 msgr2=0x7f54180cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.900+0000 7f5417400640 1 --2- 172.21.15.28:0/3539496549 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f53fc0039b0 0x7f54180cbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.900+0000 7f5417400640 1 -- 172.21.15.28:0/3539496549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54181f7fb0 msgr2=0x7f54181616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.900+0000 7f5417400640 1 --2- 172.21.15.28:0/3539496549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54181f7fb0 0x7f54181616e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.900+0000 7f5417400640 1 -- 172.21.15.28:0/3539496549 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54180cca20 con 0x7f54180cc4a0 2024-04-26T20:58:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.900+0000 7f5417400640 1 --2- 172.21.15.28:0/3539496549 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54180cc4a0 0x7f5418161c20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5410009f30 tx=0x7f541000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.900+0000 7f540ea00640 1 -- 172.21.15.28:0/3539496549 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541001c070 con 0x7f54180cc4a0 2024-04-26T20:58:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.900+0000 7f540ea00640 1 -- 172.21.15.28:0/3539496549 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5410017400 con 0x7f54180cc4a0 2024-04-26T20:58:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.900+0000 7f540ea00640 1 -- 172.21.15.28:0/3539496549 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54100177d0 con 0x7f54180cc4a0 2024-04-26T20:58:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.901+0000 7f5417e00640 1 -- 172.21.15.28:0/3539496549 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54180ccd00 con 0x7f54180cc4a0 2024-04-26T20:58:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.901+0000 7f5417e00640 1 -- 172.21.15.28:0/3539496549 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54180cd2c0 con 0x7f54180cc4a0 2024-04-26T20:58:10.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.902+0000 7f5417e00640 1 -- 172.21.15.28:0/3539496549 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f541806a250 con 0x7f54180cc4a0 2024-04-26T20:58:10.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.903+0000 7f540ea00640 1 -- 172.21.15.28:0/3539496549 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f541001f070 con 0x7f54180cc4a0 2024-04-26T20:58:10.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.904+0000 7f540ea00640 1 --2- 172.21.15.28:0/3539496549 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f53f406c920 0x7f53f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:10.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.904+0000 7f540ea00640 1 -- 172.21.15.28:0/3539496549 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f541008bbe0 con 0x7f54180cc4a0 2024-04-26T20:58:10.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.905+0000 7f5416a00640 1 --2- 172.21.15.28:0/3539496549 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f53f406c920 0x7f53f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:10.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.905+0000 7f5416a00640 1 --2- 172.21.15.28:0/3539496549 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f53f406c920 0x7f53f406ede0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5408033000 tx=0x7f5408009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:10.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:10.909+0000 7f540ea00640 1 -- 172.21.15.28:0/3539496549 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5410012070 con 0x7f54180cc4a0 2024-04-26T20:58:11.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:11.145+0000 7f5417e00640 1 -- 172.21.15.28:0/3539496549 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f54180b3e90 con 0x7f54180cc4a0 2024-04-26T20:58:11.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:11.811+0000 7f540ea00640 1 -- 172.21.15.28:0/3539496549 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f5410059f70 con 0x7f54180cc4a0 2024-04-26T20:58:11.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:11.815+0000 7f5417e00640 1 -- 172.21.15.28:0/3539496549 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f53f406c920 msgr2=0x7f53f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:11.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:11.815+0000 7f5417e00640 1 --2- 172.21.15.28:0/3539496549 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f53f406c920 0x7f53f406ede0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5408033000 tx=0x7f5408009030 comp rx=0 tx=0).stop 2024-04-26T20:58:11.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:11.815+0000 7f5417e00640 1 -- 172.21.15.28:0/3539496549 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54180cc4a0 msgr2=0x7f5418161c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:11.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:11.815+0000 7f5417e00640 1 --2- 172.21.15.28:0/3539496549 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54180cc4a0 0x7f5418161c20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5410009f30 tx=0x7f541000f4a0 comp rx=0 tx=0).stop 2024-04-26T20:58:11.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:11.815+0000 7f5417e00640 1 -- 172.21.15.28:0/3539496549 shutdown_connections 2024-04-26T20:58:11.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:11.815+0000 7f5417e00640 1 --2- 172.21.15.28:0/3539496549 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f53f406c920 0x7f53f406ede0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:11.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:11.815+0000 7f5417e00640 1 --2- 172.21.15.28:0/3539496549 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f54180cc4a0 0x7f5418161c20 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:11.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:11.815+0000 7f5417e00640 1 --2- 172.21.15.28:0/3539496549 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54181f7fb0 0x7f54181616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:11.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:11.815+0000 7f5417e00640 1 --2- 172.21.15.28:0/3539496549 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f53fc0039b0 0x7f54180cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:11.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:11.815+0000 7f5417e00640 1 -- 172.21.15.28:0/3539496549 >> 172.21.15.28:0/3539496549 conn(0x7f541806c770 msgr2=0x7f54180b91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:11.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:11.815+0000 7f5417e00640 1 -- 172.21.15.28:0/3539496549 shutdown_connections 2024-04-26T20:58:11.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:11.815+0000 7f5417e00640 1 -- 172.21.15.28:0/3539496549 wait complete. 2024-04-26T20:58:11.839 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T20:58:11.839 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T20:58:12.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.054+0000 7f1753e00640 1 Processor -- start 2024-04-26T20:58:12.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.054+0000 7f1753e00640 1 -- start start 2024-04-26T20:58:12.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.056+0000 7f1753e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f175414ce40 0x7f175406f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:12.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.056+0000 7f1753e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1754166f50 0x7f17540d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:12.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.056+0000 7f1753e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17540721e0 con 0x7f17540d1250 2024-04-26T20:58:12.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.056+0000 7f1753e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f17540cd4e0 con 0x7f175414ce40 2024-04-26T20:58:12.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.056+0000 7f1753e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f175406fdd0 con 0x7f1754166f50 2024-04-26T20:58:12.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.057+0000 7f1753400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f17540d1250 0x7f175406df20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50104/0 (socket says 172.21.15.28:50104) 2024-04-26T20:58:12.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.057+0000 7f1753400640 1 -- 172.21.15.28:0/2908507136 learned_addr learned my addr 172.21.15.28:0/2908507136 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:12.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.057+0000 7f1752a00640 1 --2- 172.21.15.28:0/2908507136 >> v2:172.21.15.103:3300/0 conn(0x7f175414ce40 0x7f175406f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:12.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.057+0000 7f174a000640 1 --2- 172.21.15.28:0/2908507136 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1754166f50 0x7f17540d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:12.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.057+0000 7f1752000640 1 -- 172.21.15.28:0/2908507136 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3068859081 0 0) 0x7f17540721e0 con 0x7f17540d1250 2024-04-26T20:58:12.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.057+0000 7f1752000640 1 -- 172.21.15.28:0/2908507136 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1738003520 con 0x7f17540d1250 2024-04-26T20:58:12.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.058+0000 7f1752000640 1 -- 172.21.15.28:0/2908507136 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 919659883 0 0) 0x7f1738003520 con 0x7f17540d1250 2024-04-26T20:58:12.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.058+0000 7f1752000640 1 -- 172.21.15.28:0/2908507136 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1754166f50 msgr2=0x7f17540d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:12.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.058+0000 7f1752000640 1 --2- 172.21.15.28:0/2908507136 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1754166f50 0x7f17540d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:12.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.058+0000 7f1752000640 1 -- 172.21.15.28:0/2908507136 >> v2:172.21.15.103:3300/0 conn(0x7f175414ce40 msgr2=0x7f175406f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:12.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.058+0000 7f1752000640 1 --2- 172.21.15.28:0/2908507136 >> v2:172.21.15.103:3300/0 conn(0x7f175414ce40 0x7f175406f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:12.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.058+0000 7f1752000640 1 -- 172.21.15.28:0/2908507136 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f175414d220 con 0x7f17540d1250 2024-04-26T20:58:12.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.058+0000 7f1752000640 1 -- 172.21.15.28:0/2908507136 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f174c0019f0 con 0x7f17540d1250 2024-04-26T20:58:12.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.058+0000 7f1752000640 1 -- 172.21.15.28:0/2908507136 >> v1:172.21.15.28:6789/0 conn(0x7f17540d1250 legacy=0x7f175406df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:12.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.058+0000 7f1752000640 1 --2- 172.21.15.28:0/2908507136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17380039b0 0x7f1738003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:12.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.058+0000 7f1752000640 1 --2- 172.21.15.28:0/2908507136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1738004800 0x7f1738004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:12.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.058+0000 7f1752000640 1 --2- 172.21.15.28:0/2908507136 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1754166f50 0x7f17380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:12.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.058+0000 7f1752000640 1 -- 172.21.15.28:0/2908507136 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f175406fdd0 con 0x7f1738004800 2024-04-26T20:58:12.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.058+0000 7f1752000640 1 -- 172.21.15.28:0/2908507136 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f173800b9c0 con 0x7f17380039b0 2024-04-26T20:58:12.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.058+0000 7f1752000640 1 -- 172.21.15.28:0/2908507136 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f173800bb40 con 0x7f1754166f50 2024-04-26T20:58:12.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.058+0000 7f174a000640 1 --2- 172.21.15.28:0/2908507136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1738004800 0x7f1738004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:12.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.059+0000 7f1752a00640 1 --2- 172.21.15.28:0/2908507136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17380039b0 0x7f1738003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:12.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.059+0000 7f1753400640 1 --2- 172.21.15.28:0/2908507136 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1754166f50 0x7f17380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:12.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.059+0000 7f174a000640 1 -- 172.21.15.28:0/2908507136 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1754166f50 msgr2=0x7f17380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:12.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.059+0000 7f174a000640 1 --2- 172.21.15.28:0/2908507136 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1754166f50 0x7f17380051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:12.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.059+0000 7f174a000640 1 -- 172.21.15.28:0/2908507136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17380039b0 msgr2=0x7f1738003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:12.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.059+0000 7f174a000640 1 --2- 172.21.15.28:0/2908507136 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f17380039b0 0x7f1738003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:12.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.059+0000 7f174a000640 1 -- 172.21.15.28:0/2908507136 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1738003520 con 0x7f1738004800 2024-04-26T20:58:12.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.060+0000 7f174a000640 1 --2- 172.21.15.28:0/2908507136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1738004800 0x7f1738004c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f173c009ac0 tx=0x7f1738005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bdbfac6feecad87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:12.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.060+0000 7f1752000640 1 -- 172.21.15.28:0/2908507136 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c036070 con 0x7f1738004800 2024-04-26T20:58:12.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.060+0000 7f1752000640 1 -- 172.21.15.28:0/2908507136 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f173c031400 con 0x7f1738004800 2024-04-26T20:58:12.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.060+0000 7f1752000640 1 -- 172.21.15.28:0/2908507136 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173c009690 con 0x7f1738004800 2024-04-26T20:58:12.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.061+0000 7f1753e00640 1 -- 172.21.15.28:0/2908507136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1738004800 msgr2=0x7f1738004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:12.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.061+0000 7f1753e00640 1 --2- 172.21.15.28:0/2908507136 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1738004800 0x7f1738004c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f173c009ac0 tx=0x7f1738005830 comp rx=0 tx=0).stop 2024-04-26T20:58:12.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.061+0000 7f1753400640 1 -- 172.21.15.28:0/2908507136 reap_dead start 2024-04-26T20:58:12.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.061+0000 7f1753e00640 1 -- 172.21.15.28:0/2908507136 shutdown_connections 2024-04-26T20:58:12.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.061+0000 7f1753e00640 1 -- 172.21.15.28:0/2908507136 >> 172.21.15.28:0/2908507136 conn(0x7f175406c530 msgr2=0x7f175406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:12.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.061+0000 7f1753e00640 1 -- 172.21.15.28:0/2908507136 shutdown_connections 2024-04-26T20:58:12.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.061+0000 7f1753e00640 1 -- 172.21.15.28:0/2908507136 wait complete. 2024-04-26T20:58:12.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.063+0000 7f1753e00640 1 Processor -- start 2024-04-26T20:58:12.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.063+0000 7f1753e00640 1 -- start start 2024-04-26T20:58:12.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.064+0000 7f1753e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1738004800 0x7f1754141d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:12.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.064+0000 7f1753e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1754146aa0 0x7f1754146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:12.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.064+0000 7f1753e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1754164250 0x7f17541646d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:12.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.064+0000 7f1753e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1754167a80 con 0x7f1754164250 2024-04-26T20:58:12.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.064+0000 7f1753e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f17541677a0 con 0x7f1754146aa0 2024-04-26T20:58:12.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.064+0000 7f1753e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f1754167d60 con 0x7f1738004800 2024-04-26T20:58:12.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.064+0000 7f1753400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1754164250 0x7f17541646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:12.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.064+0000 7f1753400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1754164250 0x7f17541646d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51940/0 (socket says 172.21.15.28:51940) 2024-04-26T20:58:12.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.064+0000 7f1753400640 1 -- 172.21.15.28:0/3299757787 learned_addr learned my addr 172.21.15.28:0/3299757787 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:12.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.065+0000 7f1752a00640 1 --2- 172.21.15.28:0/3299757787 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1738004800 0x7f1754141d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:12.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.065+0000 7f174a000640 1 --2- 172.21.15.28:0/3299757787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1754146aa0 0x7f1754146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:12.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.065+0000 7f1753400640 1 -- 172.21.15.28:0/3299757787 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1738004800 msgr2=0x7f1754141d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:12.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.065+0000 7f1753400640 1 --2- 172.21.15.28:0/3299757787 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1738004800 0x7f1754141d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:12.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.065+0000 7f1753400640 1 -- 172.21.15.28:0/3299757787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1754146aa0 msgr2=0x7f1754146f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:12.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.065+0000 7f1753400640 1 --2- 172.21.15.28:0/3299757787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1754146aa0 0x7f1754146f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:12.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.065+0000 7f1753400640 1 -- 172.21.15.28:0/3299757787 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17540b1be0 con 0x7f1754164250 2024-04-26T20:58:12.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.065+0000 7f174a000640 1 --2- 172.21.15.28:0/3299757787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1754146aa0 0x7f1754146f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:12.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.065+0000 7f1753400640 1 --2- 172.21.15.28:0/3299757787 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1754164250 0x7f17541646d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f17540cd490 tx=0x7f174c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:12.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.065+0000 7f174be00640 1 -- 172.21.15.28:0/3299757787 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c004d60 con 0x7f1754164250 2024-04-26T20:58:12.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.065+0000 7f174be00640 1 -- 172.21.15.28:0/3299757787 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f174c00bb20 con 0x7f1754164250 2024-04-26T20:58:12.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.065+0000 7f1753e00640 1 -- 172.21.15.28:0/3299757787 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17540b1e50 con 0x7f1754164250 2024-04-26T20:58:12.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.065+0000 7f174be00640 1 -- 172.21.15.28:0/3299757787 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c007070 con 0x7f1754164250 2024-04-26T20:58:12.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.065+0000 7f1753e00640 1 -- 172.21.15.28:0/3299757787 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17540b2410 con 0x7f1754164250 2024-04-26T20:58:12.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.067+0000 7f1753e00640 1 -- 172.21.15.28:0/3299757787 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1754147460 con 0x7f1754164250 2024-04-26T20:58:12.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.068+0000 7f174be00640 1 -- 172.21.15.28:0/3299757787 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f174c01b020 con 0x7f1754164250 2024-04-26T20:58:12.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.068+0000 7f174be00640 1 --2- 172.21.15.28:0/3299757787 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f173006c8f0 0x7f173006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:12.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.069+0000 7f174be00640 1 -- 172.21.15.28:0/3299757787 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f174c08bd40 con 0x7f1754164250 2024-04-26T20:58:12.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.069+0000 7f1752a00640 1 --2- 172.21.15.28:0/3299757787 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f173006c8f0 0x7f173006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:12.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.070+0000 7f1752a00640 1 --2- 172.21.15.28:0/3299757787 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f173006c8f0 0x7f173006edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f17440099d0 tx=0x7f1744008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:12.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.073+0000 7f174be00640 1 -- 172.21.15.28:0/3299757787 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f174c05a170 con 0x7f1754164250 2024-04-26T20:58:12.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.317+0000 7f1753e00640 1 -- 172.21.15.28:0/3299757787 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f175406a250 con 0x7f1754164250 2024-04-26T20:58:12.649 INFO:tasks.ceph.mds.d.smithi103.stderr:2024-04-26T20:58:12.648+0000 7fc512600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:58:12.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.823+0000 7f174be00640 1 -- 172.21.15.28:0/3299757787 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f174c059a70 con 0x7f1754164250 2024-04-26T20:58:12.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.827+0000 7f1753e00640 1 -- 172.21.15.28:0/3299757787 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f173006c8f0 msgr2=0x7f173006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:12.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.827+0000 7f1753e00640 1 --2- 172.21.15.28:0/3299757787 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f173006c8f0 0x7f173006edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f17440099d0 tx=0x7f1744008040 comp rx=0 tx=0).stop 2024-04-26T20:58:12.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.827+0000 7f1753e00640 1 -- 172.21.15.28:0/3299757787 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1754164250 msgr2=0x7f17541646d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:12.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.827+0000 7f1753e00640 1 --2- 172.21.15.28:0/3299757787 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1754164250 0x7f17541646d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f17540cd490 tx=0x7f174c00b4a0 comp rx=0 tx=0).stop 2024-04-26T20:58:12.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.827+0000 7f1753e00640 1 -- 172.21.15.28:0/3299757787 shutdown_connections 2024-04-26T20:58:12.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.827+0000 7f1753e00640 1 --2- 172.21.15.28:0/3299757787 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f173006c8f0 0x7f173006edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:12.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.827+0000 7f1753e00640 1 --2- 172.21.15.28:0/3299757787 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1754164250 0x7f17541646d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:12.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.827+0000 7f1753e00640 1 --2- 172.21.15.28:0/3299757787 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1754146aa0 0x7f1754146f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:12.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.827+0000 7f1753e00640 1 --2- 172.21.15.28:0/3299757787 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1738004800 0x7f1754141d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:12.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.827+0000 7f1753e00640 1 -- 172.21.15.28:0/3299757787 >> 172.21.15.28:0/3299757787 conn(0x7f175406c530 msgr2=0x7f175414ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:12.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.828+0000 7f1753e00640 1 -- 172.21.15.28:0/3299757787 shutdown_connections 2024-04-26T20:58:12.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:12.828+0000 7f1753e00640 1 -- 172.21.15.28:0/3299757787 wait complete. 2024-04-26T20:58:12.856 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T20:58:12.856 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:58:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.068+0000 7f35a9e00640 1 Processor -- start 2024-04-26T20:58:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.068+0000 7f35a9e00640 1 -- start start 2024-04-26T20:58:13.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.070+0000 7f35a9e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f35a4146060 0x7f35a4071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:13.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.070+0000 7f35a9e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f35a41610c0 0x7f35a40cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:13.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.070+0000 7f35a9e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35a4146440 con 0x7f35a4146670 2024-04-26T20:58:13.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.070+0000 7f35a9e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f35a406f510 con 0x7f35a41610c0 2024-04-26T20:58:13.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.070+0000 7f35a9e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f35a406f010 con 0x7f35a4146060 2024-04-26T20:58:13.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.070+0000 7f35a9400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f35a41610c0 0x7f35a40cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:13.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.071+0000 7f35a9400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f35a41610c0 0x7f35a40cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51944/0 (socket says 172.21.15.28:51944) 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.071+0000 7f35a9400640 1 -- 172.21.15.28:0/1375541731 learned_addr learned my addr 172.21.15.28:0/1375541731 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.071+0000 7f35a8a00640 1 --2- 172.21.15.28:0/1375541731 >> v2:172.21.15.103:3300/0 conn(0x7f35a4146060 0x7f35a4071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.071+0000 7f35a9400640 1 -- 172.21.15.28:0/1375541731 >> v2:172.21.15.103:3300/0 conn(0x7f35a4146060 msgr2=0x7f35a4071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.071+0000 7f35a9400640 1 --2- 172.21.15.28:0/1375541731 >> v2:172.21.15.103:3300/0 conn(0x7f35a4146060 0x7f35a4071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.071+0000 7f35a9400640 1 -- 172.21.15.28:0/1375541731 >> v1:172.21.15.28:6789/0 conn(0x7f35a4146670 legacy=0x7f35a41609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.071+0000 7f35a9400640 1 -- 172.21.15.28:0/1375541731 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a41614d0 con 0x7f35a41610c0 2024-04-26T20:58:13.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.072+0000 7f35a9400640 1 --2- 172.21.15.28:0/1375541731 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35a41610c0 0x7f35a40cb0a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f35a406d870 tx=0x7f359c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc732b27a863642e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:13.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.073+0000 7f35a3e00640 1 -- 172.21.15.28:0/1375541731 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c036070 con 0x7f35a41610c0 2024-04-26T20:58:13.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.073+0000 7f35a3e00640 1 -- 172.21.15.28:0/1375541731 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f359c031600 con 0x7f35a41610c0 2024-04-26T20:58:13.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.073+0000 7f35a3e00640 1 -- 172.21.15.28:0/1375541731 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c0319d0 con 0x7f35a41610c0 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.073+0000 7f35a9e00640 1 -- 172.21.15.28:0/1375541731 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35a41610c0 msgr2=0x7f35a40cb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.073+0000 7f35a9e00640 1 --2- 172.21.15.28:0/1375541731 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35a41610c0 0x7f35a40cb0a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f35a406d870 tx=0x7f359c02a6a0 comp rx=0 tx=0).stop 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.073+0000 7f35a9e00640 1 -- 172.21.15.28:0/1375541731 shutdown_connections 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.073+0000 7f35a9e00640 1 --2- 172.21.15.28:0/1375541731 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35a41610c0 0x7f35a40cb0a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.073+0000 7f35a9e00640 1 --2- 172.21.15.28:0/1375541731 >> v2:172.21.15.103:3300/0 conn(0x7f35a4146060 0x7f35a4071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.073+0000 7f35a9e00640 1 -- 172.21.15.28:0/1375541731 >> 172.21.15.28:0/1375541731 conn(0x7f35a40d0a50 msgr2=0x7f35a40d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.073+0000 7f35a9e00640 1 -- 172.21.15.28:0/1375541731 shutdown_connections 2024-04-26T20:58:13.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.074+0000 7f35a9e00640 1 -- 172.21.15.28:0/1375541731 wait complete. 2024-04-26T20:58:13.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.075+0000 7f35a9e00640 1 Processor -- start 2024-04-26T20:58:13.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.075+0000 7f35a9e00640 1 -- start start 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.076+0000 7f35a9e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35a4146060 0x7f35a4142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.076+0000 7f35a9e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f35a4146670 0x7f35a4155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.076+0000 7f35a9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35a41610c0 0x7f35a413e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.076+0000 7f35a9e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f35a406ff60 con 0x7f35a41610c0 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.076+0000 7f35a9e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f35a406fc80 con 0x7f35a4146060 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.076+0000 7f35a9e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f35a40cb5e0 con 0x7f35a4146670 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.077+0000 7f35a9400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35a41610c0 0x7f35a413e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.077+0000 7f35a9400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35a41610c0 0x7f35a413e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51956/0 (socket says 172.21.15.28:51956) 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.077+0000 7f35a9400640 1 -- 172.21.15.28:0/668590670 learned_addr learned my addr 172.21.15.28:0/668590670 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:13.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.077+0000 7f35a8a00640 1 --2- 172.21.15.28:0/668590670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35a4146060 0x7f35a4142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:13.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.077+0000 7f359be00640 1 --2- 172.21.15.28:0/668590670 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f35a4146670 0x7f35a4155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.077+0000 7f35a9400640 1 -- 172.21.15.28:0/668590670 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f35a4146670 msgr2=0x7f35a4155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.077+0000 7f35a9400640 1 --2- 172.21.15.28:0/668590670 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f35a4146670 0x7f35a4155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.077+0000 7f35a9400640 1 -- 172.21.15.28:0/668590670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35a4146060 msgr2=0x7f35a4142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.077+0000 7f35a9400640 1 --2- 172.21.15.28:0/668590670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35a4146060 0x7f35a4142e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.077+0000 7f35a9400640 1 -- 172.21.15.28:0/668590670 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a413f170 con 0x7f35a41610c0 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.078+0000 7f35a9400640 1 --2- 172.21.15.28:0/668590670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35a41610c0 0x7f35a413e790 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f359c034040 tx=0x7f359c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.078+0000 7f35a1600640 1 -- 172.21.15.28:0/668590670 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c036050 con 0x7f35a41610c0 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.078+0000 7f35a1600640 1 -- 172.21.15.28:0/668590670 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f359c042070 con 0x7f35a41610c0 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.078+0000 7f35a1600640 1 -- 172.21.15.28:0/668590670 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c009510 con 0x7f35a41610c0 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.078+0000 7f35a9e00640 1 -- 172.21.15.28:0/668590670 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a40b3a60 con 0x7f35a41610c0 2024-04-26T20:58:13.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.078+0000 7f35a9e00640 1 -- 172.21.15.28:0/668590670 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a4155a50 con 0x7f35a41610c0 2024-04-26T20:58:13.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.079+0000 7f35a9e00640 1 -- 172.21.15.28:0/668590670 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a406a360 con 0x7f35a41610c0 2024-04-26T20:58:13.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.080+0000 7f35a1600640 1 -- 172.21.15.28:0/668590670 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f359c007bd0 con 0x7f35a41610c0 2024-04-26T20:58:13.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.081+0000 7f35a1600640 1 --2- 172.21.15.28:0/668590670 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f358006c920 0x7f358006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:13.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.081+0000 7f35a1600640 1 -- 172.21.15.28:0/668590670 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f359c0aca60 con 0x7f35a41610c0 2024-04-26T20:58:13.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.082+0000 7f35a8a00640 1 --2- 172.21.15.28:0/668590670 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f358006c920 0x7f358006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:13.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.082+0000 7f35a8a00640 1 --2- 172.21.15.28:0/668590670 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f358006c920 0x7f358006ede0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3594006fd0 tx=0x7f3594008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:13.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.086+0000 7f35a1600640 1 -- 172.21.15.28:0/668590670 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f359c07adf0 con 0x7f35a41610c0 2024-04-26T20:58:13.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.332+0000 7f35a9e00640 1 -- 172.21.15.28:0/668590670 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35a40cb0e0 con 0x7f35a41610c0 2024-04-26T20:58:13.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.334+0000 7f35a1600640 1 -- 172.21.15.28:0/668590670 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7f359c07a6f0 con 0x7f35a41610c0 2024-04-26T20:58:13.335 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:13.335 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/1364678224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1364678224},{"type":"v1","addr":"172.21.15.103:6834","nonce":1364678224}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4081926466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":4081926466},{"type":"v1","addr":"172.21.15.28:6837","nonce":4081926466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5175,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/1370252546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":1370252546},{"type":"v1","addr":"172.21.15.28:6835","nonce":1370252546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:58:09.226353+0000","modified":"2024-04-26T20:58:12.319110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5184":{"gid":5184,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6835/1781724684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":1781724684},{"type":"v1","addr":"172.21.15.103:6835","nonce":1781724684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5184,"qdb_cluster":[5184]},"id":2}]} 2024-04-26T20:58:13.335 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-04-26T20:58:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.336+0000 7f35a9e00640 1 -- 172.21.15.28:0/668590670 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f358006c920 msgr2=0x7f358006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.336+0000 7f35a9e00640 1 --2- 172.21.15.28:0/668590670 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f358006c920 0x7f358006ede0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3594006fd0 tx=0x7f3594008040 comp rx=0 tx=0).stop 2024-04-26T20:58:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.336+0000 7f35a9e00640 1 -- 172.21.15.28:0/668590670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35a41610c0 msgr2=0x7f35a413e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.336+0000 7f35a9e00640 1 --2- 172.21.15.28:0/668590670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35a41610c0 0x7f35a413e790 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f359c034040 tx=0x7f359c004a40 comp rx=0 tx=0).stop 2024-04-26T20:58:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.336+0000 7f35a9e00640 1 -- 172.21.15.28:0/668590670 shutdown_connections 2024-04-26T20:58:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.336+0000 7f35a9e00640 1 --2- 172.21.15.28:0/668590670 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f358006c920 0x7f358006ede0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.336+0000 7f35a9e00640 1 --2- 172.21.15.28:0/668590670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35a41610c0 0x7f35a413e790 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.336+0000 7f35a9e00640 1 --2- 172.21.15.28:0/668590670 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f35a4146670 0x7f35a4155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.336+0000 7f35a9e00640 1 --2- 172.21.15.28:0/668590670 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35a4146060 0x7f35a4142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.336+0000 7f35a9e00640 1 -- 172.21.15.28:0/668590670 >> 172.21.15.28:0/668590670 conn(0x7f35a40d0a50 msgr2=0x7f35a40b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.336+0000 7f35a9e00640 1 -- 172.21.15.28:0/668590670 shutdown_connections 2024-04-26T20:58:13.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.336+0000 7f35a9e00640 1 -- 172.21.15.28:0/668590670 wait complete. 2024-04-26T20:58:13.356 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:58:13.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.566+0000 7fe4fdc00640 1 Processor -- start 2024-04-26T20:58:13.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.567+0000 7fe4fdc00640 1 -- start start 2024-04-26T20:58:13.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.568+0000 7fe4fdc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe4f80adf40 0x7fe4f80ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:13.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.568+0000 7fe4fdc00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f80b49b0 0x7fe4f80b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:13.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.568+0000 7fe4fdc00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4f806e780 con 0x7fe4f8069ac0 2024-04-26T20:58:13.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.568+0000 7fe4fdc00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe4f806f390 con 0x7fe4f80adf40 2024-04-26T20:58:13.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.568+0000 7fe4fdc00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe4f806ea60 con 0x7fe4f80b49b0 2024-04-26T20:58:13.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.569+0000 7fe4efe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe4f80adf40 0x7fe4f80ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:13.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.569+0000 7fe4efe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe4f80adf40 0x7fe4f80ae3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51968/0 (socket says 172.21.15.28:51968) 2024-04-26T20:58:13.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.569+0000 7fe4efe00640 1 -- 172.21.15.28:0/1895445066 learned_addr learned my addr 172.21.15.28:0/1895445066 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:13.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.569+0000 7fe4efe00640 1 -- 172.21.15.28:0/1895445066 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f80b49b0 msgr2=0x7fe4f80b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:13.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.569+0000 7fe4efe00640 1 --2- 172.21.15.28:0/1895445066 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f80b49b0 0x7fe4f80b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:13.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.569+0000 7fe4efe00640 1 -- 172.21.15.28:0/1895445066 >> v1:172.21.15.103:6789/0 conn(0x7fe4f8069ac0 legacy=0x7fe4f8072980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:13.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.569+0000 7fe4efe00640 1 -- 172.21.15.28:0/1895445066 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f806ed40 con 0x7fe4f80adf40 2024-04-26T20:58:13.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.570+0000 7fe4efe00640 1 --2- 172.21.15.28:0/1895445066 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f80adf40 0x7fe4f80ae3a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000bb10 tx=0x7fe4e002c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd414aef1e3b070d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:13.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.571+0000 7fe4f6000640 1 -- 172.21.15.28:0/1895445066 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0036020 con 0x7fe4f80adf40 2024-04-26T20:58:13.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.571+0000 7fe4f6000640 1 -- 172.21.15.28:0/1895445066 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4e0032820 con 0x7fe4f80adf40 2024-04-26T20:58:13.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.571+0000 7fe4f6000640 1 -- 172.21.15.28:0/1895445066 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4e0032bf0 con 0x7fe4f80adf40 2024-04-26T20:58:13.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.571+0000 7fe4fdc00640 1 -- 172.21.15.28:0/1895445066 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f80adf40 msgr2=0x7fe4f80ae3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:13.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.571+0000 7fe4fdc00640 1 --2- 172.21.15.28:0/1895445066 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f80adf40 0x7fe4f80ae3a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe4e000bb10 tx=0x7fe4e002c920 comp rx=0 tx=0).stop 2024-04-26T20:58:13.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.571+0000 7fe4fdc00640 1 -- 172.21.15.28:0/1895445066 shutdown_connections 2024-04-26T20:58:13.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.572+0000 7fe4fdc00640 1 --2- 172.21.15.28:0/1895445066 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f80b49b0 0x7fe4f80b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:13.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.572+0000 7fe4fdc00640 1 --2- 172.21.15.28:0/1895445066 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f80adf40 0x7fe4f80ae3a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:13.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.572+0000 7fe4fdc00640 1 -- 172.21.15.28:0/1895445066 >> 172.21.15.28:0/1895445066 conn(0x7fe4f80b2c60 msgr2=0x7fe4f80b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:13.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.572+0000 7fe4fdc00640 1 -- 172.21.15.28:0/1895445066 shutdown_connections 2024-04-26T20:58:13.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.572+0000 7fe4fdc00640 1 -- 172.21.15.28:0/1895445066 wait complete. 2024-04-26T20:58:13.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.574+0000 7fe4fdc00640 1 Processor -- start 2024-04-26T20:58:13.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.574+0000 7fe4fdc00640 1 -- start start 2024-04-26T20:58:13.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.575+0000 7fe4fdc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4f8069ac0 0x7fe4f81f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:13.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.575+0000 7fe4fdc00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f80adf40 0x7fe4f809ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:13.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.575+0000 7fe4fdc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f80b49b0 0x7fe4f809bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:13.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.575+0000 7fe4fdc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f806e410 con 0x7fe4f80b49b0 2024-04-26T20:58:13.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.575+0000 7fe4fdc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe4f80ae8e0 con 0x7fe4f8069ac0 2024-04-26T20:58:13.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.575+0000 7fe4fdc00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe4f8073090 con 0x7fe4f80adf40 2024-04-26T20:58:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.575+0000 7fe4f7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f80b49b0 0x7fe4f809bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.575+0000 7fe4f7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f80b49b0 0x7fe4f809bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51970/0 (socket says 172.21.15.28:51970) 2024-04-26T20:58:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.575+0000 7fe4f7400640 1 -- 172.21.15.28:0/2666880066 learned_addr learned my addr 172.21.15.28:0/2666880066 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.576+0000 7fe4f6a00640 1 --2- 172.21.15.28:0/2666880066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4f8069ac0 0x7fe4f81f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.576+0000 7fe4efe00640 1 --2- 172.21.15.28:0/2666880066 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f80adf40 0x7fe4f809ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.576+0000 7fe4f7400640 1 -- 172.21.15.28:0/2666880066 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f80adf40 msgr2=0x7fe4f809ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.576+0000 7fe4f7400640 1 --2- 172.21.15.28:0/2666880066 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f80adf40 0x7fe4f809ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.576+0000 7fe4f7400640 1 -- 172.21.15.28:0/2666880066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4f8069ac0 msgr2=0x7fe4f81f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.576+0000 7fe4f7400640 1 --2- 172.21.15.28:0/2666880066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4f8069ac0 0x7fe4f81f5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.576+0000 7fe4f7400640 1 -- 172.21.15.28:0/2666880066 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4f8153d20 con 0x7fe4f80b49b0 2024-04-26T20:58:13.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.576+0000 7fe4f7400640 1 --2- 172.21.15.28:0/2666880066 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f80b49b0 0x7fe4f809bf90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000b9a0 tx=0x7fe4f000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:13.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.577+0000 7fe4eea00640 1 -- 172.21.15.28:0/2666880066 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0019070 con 0x7fe4f80b49b0 2024-04-26T20:58:13.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.577+0000 7fe4eea00640 1 -- 172.21.15.28:0/2666880066 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4f000cbc0 con 0x7fe4f80b49b0 2024-04-26T20:58:13.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.577+0000 7fe4eea00640 1 -- 172.21.15.28:0/2666880066 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0014440 con 0x7fe4f80b49b0 2024-04-26T20:58:13.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.577+0000 7fe4fdc00640 1 -- 172.21.15.28:0/2666880066 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4f8154000 con 0x7fe4f80b49b0 2024-04-26T20:58:13.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.577+0000 7fe4fdc00640 1 -- 172.21.15.28:0/2666880066 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4f81545c0 con 0x7fe4f80b49b0 2024-04-26T20:58:13.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.578+0000 7fe4fdc00640 1 -- 172.21.15.28:0/2666880066 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4f806a400 con 0x7fe4f80b49b0 2024-04-26T20:58:13.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.579+0000 7fe4eea00640 1 -- 172.21.15.28:0/2666880066 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4f001c070 con 0x7fe4f80b49b0 2024-04-26T20:58:13.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.580+0000 7fe4eea00640 1 --2- 172.21.15.28:0/2666880066 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe4d406c8f0 0x7fe4d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:13.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.581+0000 7fe4eea00640 1 -- 172.21.15.28:0/2666880066 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fe4f008b8e0 con 0x7fe4f80b49b0 2024-04-26T20:58:13.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.581+0000 7fe4f6a00640 1 --2- 172.21.15.28:0/2666880066 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe4d406c8f0 0x7fe4d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:13.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.581+0000 7fe4f6a00640 1 --2- 172.21.15.28:0/2666880066 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe4d406c8f0 0x7fe4d406edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8007fd0 tx=0x7fe4e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:13.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.585+0000 7fe4eea00640 1 -- 172.21.15.28:0/2666880066 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe4f0059c70 con 0x7fe4f80b49b0 2024-04-26T20:58:13.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.788+0000 7fe4fdc00640 1 -- 172.21.15.28:0/2666880066 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4f814df50 con 0x7fe4f80b49b0 2024-04-26T20:58:13.791 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.790+0000 7fe4eea00640 1 -- 172.21.15.28:0/2666880066 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16645 (secure 0 0 0) 0x7fe4f0059570 con 0x7fe4f80b49b0 2024-04-26T20:58:13.792 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:13.792 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":34,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:58:10.759775+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:58:07.143890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:58:08.193405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:58:13.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.793+0000 7fe4fdc00640 1 -- 172.21.15.28:0/2666880066 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe4d406c8f0 msgr2=0x7fe4d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:13.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.793+0000 7fe4fdc00640 1 --2- 172.21.15.28:0/2666880066 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe4d406c8f0 0x7fe4d406edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe4e8007fd0 tx=0x7fe4e8009040 comp rx=0 tx=0).stop 2024-04-26T20:58:13.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.793+0000 7fe4fdc00640 1 -- 172.21.15.28:0/2666880066 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f80b49b0 msgr2=0x7fe4f809bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:13.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.793+0000 7fe4fdc00640 1 --2- 172.21.15.28:0/2666880066 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f80b49b0 0x7fe4f809bf90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000b9a0 tx=0x7fe4f000be60 comp rx=0 tx=0).stop 2024-04-26T20:58:13.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.793+0000 7fe4fdc00640 1 -- 172.21.15.28:0/2666880066 shutdown_connections 2024-04-26T20:58:13.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.793+0000 7fe4fdc00640 1 --2- 172.21.15.28:0/2666880066 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe4d406c8f0 0x7fe4d406edb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:13.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.793+0000 7fe4fdc00640 1 --2- 172.21.15.28:0/2666880066 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4f80b49b0 0x7fe4f809bf90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:13.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.793+0000 7fe4fdc00640 1 --2- 172.21.15.28:0/2666880066 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe4f80adf40 0x7fe4f809ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:13.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.793+0000 7fe4fdc00640 1 --2- 172.21.15.28:0/2666880066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4f8069ac0 0x7fe4f81f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:13.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.793+0000 7fe4fdc00640 1 -- 172.21.15.28:0/2666880066 >> 172.21.15.28:0/2666880066 conn(0x7fe4f80b2c60 msgr2=0x7fe4f80b5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:13.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.794+0000 7fe4fdc00640 1 -- 172.21.15.28:0/2666880066 shutdown_connections 2024-04-26T20:58:13.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:13.794+0000 7fe4fdc00640 1 -- 172.21.15.28:0/2666880066 wait complete. 2024-04-26T20:58:13.814 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T20:58:13.814 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:58:14.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.025+0000 7fefb3200640 1 Processor -- start 2024-04-26T20:58:14.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.026+0000 7fefb3200640 1 -- start start 2024-04-26T20:58:14.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.027+0000 7fefb3200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb415f070 0x7fefb40d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:14.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.027+0000 7fefb3200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fefb40ba710 0x7fefb40d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:14.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.027+0000 7fefb3200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fefb40d4500 con 0x7fefb40bab70 2024-04-26T20:58:14.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.027+0000 7fefb3200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fefb415f450 con 0x7fefb40ba710 2024-04-26T20:58:14.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.027+0000 7fefb3200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fefb40d4a80 con 0x7fefb415f070 2024-04-26T20:58:14.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.028+0000 7fefb1400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fefb40ba710 0x7fefb40d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:14.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.028+0000 7fefb1400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fefb40ba710 0x7fefb40d1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51980/0 (socket says 172.21.15.28:51980) 2024-04-26T20:58:14.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.028+0000 7fefb1400640 1 -- 172.21.15.28:0/1214454449 learned_addr learned my addr 172.21.15.28:0/1214454449 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:14.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.028+0000 7fefb1e00640 1 --2- 172.21.15.28:0/1214454449 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb415f070 0x7fefb40d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:14.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.028+0000 7fefb1400640 1 -- 172.21.15.28:0/1214454449 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb415f070 msgr2=0x7fefb40d36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:14.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.028+0000 7fefb1400640 1 --2- 172.21.15.28:0/1214454449 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb415f070 0x7fefb40d36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.029+0000 7fefb1400640 1 -- 172.21.15.28:0/1214454449 >> v1:172.21.15.103:6789/0 conn(0x7fefb40bab70 legacy=0x7fefb40b7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:14.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.029+0000 7fefb1400640 1 -- 172.21.15.28:0/1214454449 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb40d01c0 con 0x7fefb40ba710 2024-04-26T20:58:14.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.030+0000 7fefb1400640 1 --2- 172.21.15.28:0/1214454449 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb40ba710 0x7fefb40d1800 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fefb415f630 tx=0x7fefa002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e804f1d3994c108 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:14.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.030+0000 7fefb0a00640 1 -- 172.21.15.28:0/1214454449 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0037070 con 0x7fefb40ba710 2024-04-26T20:58:14.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.030+0000 7fefb0a00640 1 -- 172.21.15.28:0/1214454449 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefa0032440 con 0x7fefb40ba710 2024-04-26T20:58:14.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.030+0000 7fefb0a00640 1 -- 172.21.15.28:0/1214454449 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0032810 con 0x7fefb40ba710 2024-04-26T20:58:14.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.031+0000 7fefb3200640 1 -- 172.21.15.28:0/1214454449 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb40ba710 msgr2=0x7fefb40d1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:14.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.031+0000 7fefb3200640 1 --2- 172.21.15.28:0/1214454449 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb40ba710 0x7fefb40d1800 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fefb415f630 tx=0x7fefa002a1c0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.031+0000 7fefb3200640 1 -- 172.21.15.28:0/1214454449 shutdown_connections 2024-04-26T20:58:14.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.031+0000 7fefb3200640 1 --2- 172.21.15.28:0/1214454449 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb40ba710 0x7fefb40d1800 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.031+0000 7fefb3200640 1 --2- 172.21.15.28:0/1214454449 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb415f070 0x7fefb40d36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.031+0000 7fefb3200640 1 -- 172.21.15.28:0/1214454449 >> 172.21.15.28:0/1214454449 conn(0x7fefb406c980 msgr2=0x7fefb4072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:14.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.031+0000 7fefb3200640 1 -- 172.21.15.28:0/1214454449 shutdown_connections 2024-04-26T20:58:14.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.031+0000 7fefb3200640 1 -- 172.21.15.28:0/1214454449 wait complete. 2024-04-26T20:58:14.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.033+0000 7fefb3200640 1 Processor -- start 2024-04-26T20:58:14.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.033+0000 7fefb3200640 1 -- start start 2024-04-26T20:58:14.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.034+0000 7fefb3200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb40ba710 0x7fefb40b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:14.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.034+0000 7fefb3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb40bab70 0x7fefb41f58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:14.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.034+0000 7fefb3200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb415f070 0x7fefb41e9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:14.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.034+0000 7fefb3200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fefb40d3bf0 con 0x7fefb40bab70 2024-04-26T20:58:14.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.034+0000 7fefb3200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fefb40d04a0 con 0x7fefb415f070 2024-04-26T20:58:14.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.034+0000 7fefb3200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fefb40d3d70 con 0x7fefb40ba710 2024-04-26T20:58:14.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.034+0000 7fefb1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb40bab70 0x7fefb41f58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:14.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.034+0000 7fefb1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb40bab70 0x7fefb41f58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51990/0 (socket says 172.21.15.28:51990) 2024-04-26T20:58:14.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.034+0000 7fefb1400640 1 -- 172.21.15.28:0/1967477293 learned_addr learned my addr 172.21.15.28:0/1967477293 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:14.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.035+0000 7fefb2800640 1 --2- 172.21.15.28:0/1967477293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb415f070 0x7fefb41e9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:14.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.035+0000 7fefb1400640 1 -- 172.21.15.28:0/1967477293 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb40ba710 msgr2=0x7fefb40b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:14.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.035+0000 7fefb1400640 1 --2- 172.21.15.28:0/1967477293 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb40ba710 0x7fefb40b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.035+0000 7fefb1400640 1 -- 172.21.15.28:0/1967477293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb415f070 msgr2=0x7fefb41e9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:14.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.035+0000 7fefb1400640 1 --2- 172.21.15.28:0/1967477293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb415f070 0x7fefb41e9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.035+0000 7fefb1400640 1 -- 172.21.15.28:0/1967477293 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb41f5df0 con 0x7fefb40bab70 2024-04-26T20:58:14.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.035+0000 7fefb2800640 1 --2- 172.21.15.28:0/1967477293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb415f070 0x7fefb41e9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:14.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.035+0000 7fefb1400640 1 --2- 172.21.15.28:0/1967477293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb40bab70 0x7fefb41f58b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009960 tx=0x7fefa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:14.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.035+0000 7fefaa000640 1 -- 172.21.15.28:0/1967477293 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0041020 con 0x7fefb40bab70 2024-04-26T20:58:14.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.035+0000 7fefaa000640 1 -- 172.21.15.28:0/1967477293 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefa0030780 con 0x7fefb40bab70 2024-04-26T20:58:14.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.035+0000 7fefaa000640 1 -- 172.21.15.28:0/1967477293 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa0037040 con 0x7fefb40bab70 2024-04-26T20:58:14.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.035+0000 7fefb3200640 1 -- 172.21.15.28:0/1967477293 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb40b6b60 con 0x7fefb40bab70 2024-04-26T20:58:14.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.036+0000 7fefb3200640 1 -- 172.21.15.28:0/1967477293 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb40b7120 con 0x7fefb40bab70 2024-04-26T20:58:14.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.037+0000 7fefb3200640 1 -- 172.21.15.28:0/1967477293 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefb406a360 con 0x7fefb40bab70 2024-04-26T20:58:14.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.038+0000 7fefaa000640 1 -- 172.21.15.28:0/1967477293 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fefa0044020 con 0x7fefb40bab70 2024-04-26T20:58:14.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.039+0000 7fefaa000640 1 --2- 172.21.15.28:0/1967477293 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fef9406c8f0 0x7fef9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:14.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.039+0000 7fefaa000640 1 -- 172.21.15.28:0/1967477293 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fefa00ac3c0 con 0x7fefb40bab70 2024-04-26T20:58:14.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.039+0000 7fefb1e00640 1 --2- 172.21.15.28:0/1967477293 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fef9406c8f0 0x7fef9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:14.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.040+0000 7fefb1e00640 1 --2- 172.21.15.28:0/1967477293 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fef9406c8f0 0x7fef9406edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fefa40070b0 tx=0x7fefa4007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:14.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.043+0000 7fefaa000640 1 -- 172.21.15.28:0/1967477293 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fefa004b030 con 0x7fefb40bab70 2024-04-26T20:58:14.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.238+0000 7fefb3200640 1 -- 172.21.15.28:0/1967477293 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fefb40d2d40 con 0x7fef9406c8f0 2024-04-26T20:58:14.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.259+0000 7fefaa000640 1 -- 172.21.15.28:0/1967477293 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245333 (secure 0 0 0) 0x7fefb40d2d40 con 0x7fef9406c8f0 2024-04-26T20:58:14.264 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:14.265 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T20:58:14.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.265+0000 7fefb3200640 1 -- 172.21.15.28:0/1967477293 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fef9406c8f0 msgr2=0x7fef9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:14.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.265+0000 7fefb3200640 1 --2- 172.21.15.28:0/1967477293 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fef9406c8f0 0x7fef9406edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fefa40070b0 tx=0x7fefa4007040 comp rx=0 tx=0).stop 2024-04-26T20:58:14.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.265+0000 7fefb3200640 1 -- 172.21.15.28:0/1967477293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb40bab70 msgr2=0x7fefb41f58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:14.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.265+0000 7fefb3200640 1 --2- 172.21.15.28:0/1967477293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb40bab70 0x7fefb41f58b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009960 tx=0x7fefa00046c0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.265+0000 7fefb3200640 1 -- 172.21.15.28:0/1967477293 shutdown_connections 2024-04-26T20:58:14.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.265+0000 7fefb3200640 1 --2- 172.21.15.28:0/1967477293 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fef9406c8f0 0x7fef9406edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.265+0000 7fefb3200640 1 --2- 172.21.15.28:0/1967477293 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fefb415f070 0x7fefb41e9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.265+0000 7fefb3200640 1 --2- 172.21.15.28:0/1967477293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefb40bab70 0x7fefb41f58b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.266+0000 7fefb3200640 1 --2- 172.21.15.28:0/1967477293 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fefb40ba710 0x7fefb40b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.266+0000 7fefb3200640 1 -- 172.21.15.28:0/1967477293 >> 172.21.15.28:0/1967477293 conn(0x7fefb406c980 msgr2=0x7fefb40ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:14.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.266+0000 7fefb3200640 1 -- 172.21.15.28:0/1967477293 shutdown_connections 2024-04-26T20:58:14.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.266+0000 7fefb3200640 1 -- 172.21.15.28:0/1967477293 wait complete. 2024-04-26T20:58:14.285 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2024-04-26T20:58:12.730976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":20,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236680,"kb_used_data":3880,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738040,"statfs":{"total":773094113280,"available":772851752960,"internally_reserved":0,"allocated":3973120,"data_stored":2236296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.001030"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.761519+0000","last_change":"2024-04-26T20:58:08.761519+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.761519+0000","last_undegraded":"2024-04-26T20:58:08.761519+0000","last_fullsized":"2024-04-26T20:58:08.761519+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.754332+0000","last_change":"2024-04-26T20:58:08.754332+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.754332+0000","last_undegraded":"2024-04-26T20:58:08.754332+0000","last_fullsized":"2024-04-26T20:58:08.754332+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.751351+0000","last_change":"2024-04-26T20:58:08.751351+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.751351+0000","last_undegraded":"2024-04-26T20:58:08.751351+0000","last_fullsized":"2024-04-26T20:58:08.751351+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.761242+0000","last_change":"2024-04-26T20:58:08.761242+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.761242+0000","last_undegraded":"2024-04-26T20:58:08.761242+0000","last_fullsized":"2024-04-26T20:58:08.761242+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.754347+0000","last_change":"2024-04-26T20:58:08.754347+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.754347+0000","last_undegraded":"2024-04-26T20:58:08.754347+0000","last_fullsized":"2024-04-26T20:58:08.754347+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.755379+0000","last_change":"2024-04-26T20:58:08.755379+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.755379+0000","last_undegraded":"2024-04-26T20:58:08.755379+0000","last_fullsized":"2024-04-26T20:58:08.755379+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.761864+0000","last_change":"2024-04-26T20:58:08.761864+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.761864+0000","last_undegraded":"2024-04-26T20:58:08.761864+0000","last_fullsized":"2024-04-26T20:58:08.761864+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.762067+0000","last_change":"2024-04-26T20:58:08.762067+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.762067+0000","last_undegraded":"2024-04-26T20:58:08.762067+0000","last_fullsized":"2024-04-26T20:58:08.762067+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.751872+0000","last_change":"2024-04-26T20:58:08.751872+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.751872+0000","last_undegraded":"2024-04-26T20:58:08.751872+0000","last_fullsized":"2024-04-26T20:58:08.751872+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.759713+0000","last_change":"2024-04-26T20:58:08.759713+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.759713+0000","last_undegraded":"2024-04-26T20:58:08.759713+0000","last_fullsized":"2024-04-26T20:58:08.759713+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.760481+0000","last_change":"2024-04-26T20:58:08.760481+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.760481+0000","last_undegraded":"2024-04-26T20:58:08.760481+0000","last_fullsized":"2024-04-26T20:58:08.760481+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.750606+0000","last_change":"2024-04-26T20:58:08.750606+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.750606+0000","last_undegraded":"2024-04-26T20:58:08.750606+0000","last_fullsized":"2024-04-26T20:58:08.750606+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.753700+0000","last_change":"2024-04-26T20:58:08.753700+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.753700+0000","last_undegraded":"2024-04-26T20:58:08.753700+0000","last_fullsized":"2024-04-26T20:58:08.753700+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.752569+0000","last_change":"2024-04-26T20:58:08.752569+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.752569+0000","last_undegraded":"2024-04-26T20:58:08.752569+0000","last_fullsized":"2024-04-26T20:58:08.752569+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.751010+0000","last_change":"2024-04-26T20:58:08.751010+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.751010+0000","last_undegraded":"2024-04-26T20:58:08.751010+0000","last_fullsized":"2024-04-26T20:58:08.751010+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.757307+0000","last_change":"2024-04-26T20:58:08.757307+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.757307+0000","last_undegraded":"2024-04-26T20:58:08.757307+0000","last_fullsized":"2024-04-26T20:58:08.757307+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.758877+0000","last_change":"2024-04-26T20:58:08.758877+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.758877+0000","last_undegraded":"2024-04-26T20:58:08.758877+0000","last_fullsized":"2024-04-26T20:58:08.758877+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.753775+0000","last_change":"2024-04-26T20:58:08.753775+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.753775+0000","last_undegraded":"2024-04-26T20:58:08.753775+0000","last_fullsized":"2024-04-26T20:58:08.753775+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.751256+0000","last_change":"2024-04-26T20:58:08.751256+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.751256+0000","last_undegraded":"2024-04-26T20:58:08.751256+0000","last_fullsized":"2024-04-26T20:58:08.751256+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.751684+0000","last_change":"2024-04-26T20:58:08.751684+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.751684+0000","last_undegraded":"2024-04-26T20:58:08.751684+0000","last_fullsized":"2024-04-26T20:58:08.751684+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.752708+0000","last_change":"2024-04-26T20:58:08.752708+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.752708+0000","last_undegraded":"2024-04-26T20:58:08.752708+0000","last_fullsized":"2024-04-26T20:58:08.752708+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.749012+0000","last_change":"2024-04-26T20:58:08.749012+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.749012+0000","last_undegraded":"2024-04-26T20:58:08.749012+0000","last_fullsized":"2024-04-26T20:58:08.749012+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.749188+0000","last_change":"2024-04-26T20:58:08.749188+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.749188+0000","last_undegraded":"2024-04-26T20:58:08.749188+0000","last_fullsized":"2024-04-26T20:58:08.749188+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.746417+0000","last_change":"2024-04-26T20:58:08.746417+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.746417+0000","last_undegraded":"2024-04-26T20:58:08.746417+0000","last_fullsized":"2024-04-26T20:58:08.746417+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.756779+0000","last_change":"2024-04-26T20:58:08.756779+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.756779+0000","last_undegraded":"2024-04-26T20:58:08.756779+0000","last_fullsized":"2024-04-26T20:58:08.756779+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.749565+0000","last_change":"2024-04-26T20:58:08.749565+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.749565+0000","last_undegraded":"2024-04-26T20:58:08.749565+0000","last_fullsized":"2024-04-26T20:58:08.749565+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.756623+0000","last_change":"2024-04-26T20:58:08.756623+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.756623+0000","last_undegraded":"2024-04-26T20:58:08.756623+0000","last_fullsized":"2024-04-26T20:58:08.756623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.747793+0000","last_change":"2024-04-26T20:58:08.747793+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.747793+0000","last_undegraded":"2024-04-26T20:58:08.747793+0000","last_fullsized":"2024-04-26T20:58:08.747793+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.750966+0000","last_change":"2024-04-26T20:58:08.750966+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.750966+0000","last_undegraded":"2024-04-26T20:58:08.750966+0000","last_fullsized":"2024-04-26T20:58:08.750966+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.759280+0000","last_change":"2024-04-26T20:58:08.759280+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.759280+0000","last_undegraded":"2024-04-26T20:58:08.759280+0000","last_fullsized":"2024-04-26T20:58:08.759280+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.765424+0000","last_change":"2024-04-26T20:58:08.765424+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.765424+0000","last_undegraded":"2024-04-26T20:58:08.765424+0000","last_fullsized":"2024-04-26T20:58:08.765424+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.761588+0000","last_change":"2024-04-26T20:58:08.761588+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.761588+0000","last_undegraded":"2024-04-26T20:58:08.761588+0000","last_fullsized":"2024-04-26T20:58:08.761588+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.756116+0000","last_change":"2024-04-26T20:58:08.756116+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.756116+0000","last_undegraded":"2024-04-26T20:58:08.756116+0000","last_fullsized":"2024-04-26T20:58:08.756116+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:58:07.747173+0000","last_change":"2024-04-26T20:58:07.747173+0000","last_active":"2024-04-26T20:58:07.715649+0000","last_peered":"2024-04-26T20:58:07.715649+0000","last_clean":"2024-04-26T20:58:07.715649+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:07.747173+0000","last_undegraded":"2024-04-26T20:58:07.747173+0000","last_fullsized":"2024-04-26T20:58:07.747173+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.760631+0000","last_change":"2024-04-26T20:58:08.760631+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.760631+0000","last_undegraded":"2024-04-26T20:58:08.760631+0000","last_fullsized":"2024-04-26T20:58:08.760631+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-26T20:58:08.758374+0000","last_change":"2024-04-26T20:58:08.758374+0000","last_active":"2024-04-26T20:58:07.715649+0000","last_peered":"2024-04-26T20:58:07.715649+0000","last_clean":"2024-04-26T20:58:07.715649+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.758374+0000","last_undegraded":"2024-04-26T20:58:08.758374+0000","last_fullsized":"2024-04-26T20:58:08.758374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.750212+0000","last_change":"2024-04-26T20:58:08.750212+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.750212+0000","last_undegraded":"2024-04-26T20:58:08.750212+0000","last_fullsized":"2024-04-26T20:58:08.750212+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T20:58:08.767382+0000","last_change":"2024-04-26T20:58:08.764290+0000","last_active":"2024-04-26T20:58:08.767382+0000","last_peered":"2024-04-26T20:58:08.767382+0000","last_clean":"2024-04-26T20:58:08.767382+0000","last_became_active":"2024-04-26T20:58:08.763983+0000","last_became_peered":"2024-04-26T20:58:08.763983+0000","last_unstale":"2024-04-26T20:58:08.767382+0000","last_undegraded":"2024-04-26T20:58:08.767382+0000","last_fullsized":"2024-04-26T20:58:08.767382+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:03.515785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.750758+0000","last_change":"2024-04-26T20:58:08.750758+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.750758+0000","last_undegraded":"2024-04-26T20:58:08.750758+0000","last_fullsized":"2024-04-26T20:58:08.750758+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-26T20:58:08.751169+0000","last_change":"2024-04-26T20:58:08.751169+0000","last_active":"2024-04-26T20:58:07.715649+0000","last_peered":"2024-04-26T20:58:07.715649+0000","last_clean":"2024-04-26T20:58:07.715649+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.751169+0000","last_undegraded":"2024-04-26T20:58:08.751169+0000","last_fullsized":"2024-04-26T20:58:08.751169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.757102+0000","last_change":"2024-04-26T20:58:08.757102+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.757102+0000","last_undegraded":"2024-04-26T20:58:08.757102+0000","last_fullsized":"2024-04-26T20:58:08.757102+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T20:58:08.766060+0000","last_change":"2024-04-26T20:58:08.766060+0000","last_active":"2024-04-26T20:58:08.766060+0000","last_peered":"2024-04-26T20:58:08.766060+0000","last_clean":"2024-04-26T20:58:08.766060+0000","last_became_active":"2024-04-26T20:58:08.765679+0000","last_became_peered":"2024-04-26T20:58:08.765679+0000","last_unstale":"2024-04-26T20:58:08.766060+0000","last_undegraded":"2024-04-26T20:58:08.766060+0000","last_fullsized":"2024-04-26T20:58:08.766060+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:45:23.703375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.746137+0000","last_change":"2024-04-26T20:58:08.746137+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.746137+0000","last_undegraded":"2024-04-26T20:58:08.746137+0000","last_fullsized":"2024-04-26T20:58:08.746137+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T20:58:08.764378+0000","last_change":"2024-04-26T20:58:08.764378+0000","last_active":"2024-04-26T20:58:08.764378+0000","last_peered":"2024-04-26T20:58:08.764378+0000","last_clean":"2024-04-26T20:58:08.764378+0000","last_became_active":"2024-04-26T20:58:08.764073+0000","last_became_peered":"2024-04-26T20:58:08.764073+0000","last_unstale":"2024-04-26T20:58:08.764378+0000","last_undegraded":"2024-04-26T20:58:08.764378+0000","last_fullsized":"2024-04-26T20:58:08.764378+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:14:25.754235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.746329+0000","last_change":"2024-04-26T20:58:08.746329+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.746329+0000","last_undegraded":"2024-04-26T20:58:08.746329+0000","last_fullsized":"2024-04-26T20:58:08.746329+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T20:58:08.766464+0000","last_change":"2024-04-26T20:58:08.761574+0000","last_active":"2024-04-26T20:58:08.766464+0000","last_peered":"2024-04-26T20:58:08.766464+0000","last_clean":"2024-04-26T20:58:08.766464+0000","last_became_active":"2024-04-26T20:58:08.760990+0000","last_became_peered":"2024-04-26T20:58:08.760990+0000","last_unstale":"2024-04-26T20:58:08.766464+0000","last_undegraded":"2024-04-26T20:58:08.766464+0000","last_fullsized":"2024-04-26T20:58:08.766464+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:32.017697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.752731+0000","last_change":"2024-04-26T20:58:08.752731+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.752731+0000","last_undegraded":"2024-04-26T20:58:08.752731+0000","last_fullsized":"2024-04-26T20:58:08.752731+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T20:58:08.768307+0000","last_change":"2024-04-26T20:58:08.768307+0000","last_active":"2024-04-26T20:58:08.768307+0000","last_peered":"2024-04-26T20:58:08.768307+0000","last_clean":"2024-04-26T20:58:08.768307+0000","last_became_active":"2024-04-26T20:58:08.767185+0000","last_became_peered":"2024-04-26T20:58:08.767185+0000","last_unstale":"2024-04-26T20:58:08.768307+0000","last_undegraded":"2024-04-26T20:58:08.768307+0000","last_fullsized":"2024-04-26T20:58:08.768307+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:08:49.213941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.752707+0000","last_change":"2024-04-26T20:58:08.752707+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.752707+0000","last_undegraded":"2024-04-26T20:58:08.752707+0000","last_fullsized":"2024-04-26T20:58:08.752707+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-26T20:58:08.740033+0000","last_change":"2024-04-26T20:58:08.740033+0000","last_active":"2024-04-26T20:58:07.715649+0000","last_peered":"2024-04-26T20:58:07.715649+0000","last_clean":"2024-04-26T20:58:07.715649+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.740033+0000","last_undegraded":"2024-04-26T20:58:08.740033+0000","last_fullsized":"2024-04-26T20:58:08.740033+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.749474+0000","last_change":"2024-04-26T20:58:08.749474+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.749474+0000","last_undegraded":"2024-04-26T20:58:08.749474+0000","last_fullsized":"2024-04-26T20:58:08.749474+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-26T20:58:08.752246+0000","last_change":"2024-04-26T20:58:08.752246+0000","last_active":"2024-04-26T20:58:07.715649+0000","last_peered":"2024-04-26T20:58:07.715649+0000","last_clean":"2024-04-26T20:58:07.715649+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.752246+0000","last_undegraded":"2024-04-26T20:58:08.752246+0000","last_fullsized":"2024-04-26T20:58:08.752246+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.749335+0000","last_change":"2024-04-26T20:58:08.749335+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.749335+0000","last_undegraded":"2024-04-26T20:58:08.749335+0000","last_fullsized":"2024-04-26T20:58:08.749335+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-26T20:58:08.749962+0000","last_change":"2024-04-26T20:58:08.749962+0000","last_active":"2024-04-26T20:58:07.715649+0000","last_peered":"2024-04-26T20:58:07.715649+0000","last_clean":"2024-04-26T20:58:07.715649+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.749962+0000","last_undegraded":"2024-04-26T20:58:08.749962+0000","last_fullsized":"2024-04-26T20:58:08.749962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.753551+0000","last_change":"2024-04-26T20:58:08.753551+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.753551+0000","last_undegraded":"2024-04-26T20:58:08.753551+0000","last_fullsized":"2024-04-26T20:58:08.753551+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":6,"reported_epoch":32,"state":"active","last_fresh":"2024-04-26T20:58:08.764302+0000","last_change":"2024-04-26T20:58:08.764227+0000","last_active":"2024-04-26T20:58:08.764302+0000","last_peered":"2024-04-26T20:58:08.764302+0000","last_clean":"2024-04-26T20:58:07.715649+0000","last_became_active":"2024-04-26T20:58:08.764227+0000","last_became_peered":"2024-04-26T20:58:08.764227+0000","last_unstale":"2024-04-26T20:58:08.764302+0000","last_undegraded":"2024-04-26T20:58:08.764302+0000","last_fullsized":"2024-04-26T20:58:08.764302+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":["3","1"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.751307+0000","last_change":"2024-04-26T20:58:08.751307+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.751307+0000","last_undegraded":"2024-04-26T20:58:08.751307+0000","last_fullsized":"2024-04-26T20:58:08.751307+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T20:58:08.771486+0000","last_change":"2024-04-26T20:58:08.771486+0000","last_active":"2024-04-26T20:58:08.771486+0000","last_peered":"2024-04-26T20:58:08.771486+0000","last_clean":"2024-04-26T20:58:08.771486+0000","last_became_active":"2024-04-26T20:58:08.771023+0000","last_became_peered":"2024-04-26T20:58:08.771023+0000","last_unstale":"2024-04-26T20:58:08.771486+0000","last_undegraded":"2024-04-26T20:58:08.771486+0000","last_fullsized":"2024-04-26T20:58:08.771486+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:25.364273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.746889+0000","last_change":"2024-04-26T20:58:08.746889+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.746889+0000","last_undegraded":"2024-04-26T20:58:08.746889+0000","last_fullsized":"2024-04-26T20:58:08.746889+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-26T20:58:08.751734+0000","last_change":"2024-04-26T20:58:08.751734+0000","last_active":"2024-04-26T20:58:07.715649+0000","last_peered":"2024-04-26T20:58:07.715649+0000","last_clean":"2024-04-26T20:58:07.715649+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.751734+0000","last_undegraded":"2024-04-26T20:58:08.751734+0000","last_fullsized":"2024-04-26T20:58:08.751734+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":["1","6"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.761124+0000","last_change":"2024-04-26T20:58:08.761124+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.761124+0000","last_undegraded":"2024-04-26T20:58:08.761124+0000","last_fullsized":"2024-04-26T20:58:08.761124+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-26T20:58:08.739724+0000","last_change":"2024-04-26T20:58:08.739724+0000","last_active":"2024-04-26T20:58:07.715649+0000","last_peered":"2024-04-26T20:58:07.715649+0000","last_clean":"2024-04-26T20:58:07.715649+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.739724+0000","last_undegraded":"2024-04-26T20:58:08.739724+0000","last_fullsized":"2024-04-26T20:58:08.739724+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.764801+0000","last_change":"2024-04-26T20:58:08.764801+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.764801+0000","last_undegraded":"2024-04-26T20:58:08.764801+0000","last_fullsized":"2024-04-26T20:58:08.764801+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T20:58:08.762882+0000","last_change":"2024-04-26T20:58:08.757290+0000","last_active":"2024-04-26T20:58:08.762882+0000","last_peered":"2024-04-26T20:58:08.762882+0000","last_clean":"2024-04-26T20:58:08.762882+0000","last_became_active":"2024-04-26T20:58:08.756980+0000","last_became_peered":"2024-04-26T20:58:08.756980+0000","last_unstale":"2024-04-26T20:58:08.762882+0000","last_undegraded":"2024-04-26T20:58:08.762882+0000","last_fullsized":"2024-04-26T20:58:08.762882+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:24.623434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.751720+0000","last_change":"2024-04-26T20:58:08.751720+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.751720+0000","last_undegraded":"2024-04-26T20:58:08.751720+0000","last_fullsized":"2024-04-26T20:58:08.751720+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-26T20:58:08.747902+0000","last_change":"2024-04-26T20:58:08.747902+0000","last_active":"2024-04-26T20:58:07.715649+0000","last_peered":"2024-04-26T20:58:07.715649+0000","last_clean":"2024-04-26T20:58:07.715649+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.747902+0000","last_undegraded":"2024-04-26T20:58:08.747902+0000","last_fullsized":"2024-04-26T20:58:08.747902+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.752300+0000","last_change":"2024-04-26T20:58:08.752300+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.752300+0000","last_undegraded":"2024-04-26T20:58:08.752300+0000","last_fullsized":"2024-04-26T20:58:08.752300+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T20:58:08.770184+0000","last_change":"2024-04-26T20:58:08.770184+0000","last_active":"2024-04-26T20:58:08.770184+0000","last_peered":"2024-04-26T20:58:08.770184+0000","last_clean":"2024-04-26T20:58:08.770184+0000","last_became_active":"2024-04-26T20:58:08.769472+0000","last_became_peered":"2024-04-26T20:58:08.769472+0000","last_unstale":"2024-04-26T20:58:08.770184+0000","last_undegraded":"2024-04-26T20:58:08.770184+0000","last_fullsized":"2024-04-26T20:58:08.770184+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:57.829155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.753003+0000","last_change":"2024-04-26T20:58:08.753003+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.753003+0000","last_undegraded":"2024-04-26T20:58:08.753003+0000","last_fullsized":"2024-04-26T20:58:08.753003+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T20:58:08.765303+0000","last_change":"2024-04-26T20:58:08.765303+0000","last_active":"2024-04-26T20:58:08.765303+0000","last_peered":"2024-04-26T20:58:08.765303+0000","last_clean":"2024-04-26T20:58:08.765303+0000","last_became_active":"2024-04-26T20:58:08.764957+0000","last_became_peered":"2024-04-26T20:58:08.764957+0000","last_unstale":"2024-04-26T20:58:08.765303+0000","last_undegraded":"2024-04-26T20:58:08.765303+0000","last_fullsized":"2024-04-26T20:58:08.765303+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:48.313805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.754489+0000","last_change":"2024-04-26T20:58:08.754489+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.754489+0000","last_undegraded":"2024-04-26T20:58:08.754489+0000","last_fullsized":"2024-04-26T20:58:08.754489+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T20:58:08.769798+0000","last_change":"2024-04-26T20:58:08.769798+0000","last_active":"2024-04-26T20:58:08.769798+0000","last_peered":"2024-04-26T20:58:08.769798+0000","last_clean":"2024-04-26T20:58:08.769798+0000","last_became_active":"2024-04-26T20:58:08.769441+0000","last_became_peered":"2024-04-26T20:58:08.769441+0000","last_unstale":"2024-04-26T20:58:08.769798+0000","last_undegraded":"2024-04-26T20:58:08.769798+0000","last_fullsized":"2024-04-26T20:58:08.769798+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:51:15.739374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.756314+0000","last_change":"2024-04-26T20:58:08.756314+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.756314+0000","last_undegraded":"2024-04-26T20:58:08.756314+0000","last_fullsized":"2024-04-26T20:58:08.756314+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-26T20:58:08.750288+0000","last_change":"2024-04-26T20:58:08.750288+0000","last_active":"2024-04-26T20:58:07.715649+0000","last_peered":"2024-04-26T20:58:07.715649+0000","last_clean":"2024-04-26T20:58:07.715649+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.750288+0000","last_undegraded":"2024-04-26T20:58:08.750288+0000","last_fullsized":"2024-04-26T20:58:08.750288+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.753055+0000","last_change":"2024-04-26T20:58:08.753055+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.753055+0000","last_undegraded":"2024-04-26T20:58:08.753055+0000","last_fullsized":"2024-04-26T20:58:08.753055+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-26T20:58:08.767710+0000","last_change":"2024-04-26T20:58:08.767710+0000","last_active":"2024-04-26T20:58:07.715649+0000","last_peered":"2024-04-26T20:58:07.715649+0000","last_clean":"2024-04-26T20:58:07.715649+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.767710+0000","last_undegraded":"2024-04-26T20:58:08.767710+0000","last_fullsized":"2024-04-26T20:58:08.767710+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":["6","1"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.755269+0000","last_change":"2024-04-26T20:58:08.755269+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.755269+0000","last_undegraded":"2024-04-26T20:58:08.755269+0000","last_fullsized":"2024-04-26T20:58:08.755269+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T20:58:08.767164+0000","last_change":"2024-04-26T20:58:08.767164+0000","last_active":"2024-04-26T20:58:08.767164+0000","last_peered":"2024-04-26T20:58:08.767164+0000","last_clean":"2024-04-26T20:58:08.767164+0000","last_became_active":"2024-04-26T20:58:08.766710+0000","last_became_peered":"2024-04-26T20:58:08.766710+0000","last_unstale":"2024-04-26T20:58:08.767164+0000","last_undegraded":"2024-04-26T20:58:08.767164+0000","last_fullsized":"2024-04-26T20:58:08.767164+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:08.127278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.762411+0000","last_change":"2024-04-26T20:58:08.762411+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.762411+0000","last_undegraded":"2024-04-26T20:58:08.762411+0000","last_fullsized":"2024-04-26T20:58:08.762411+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-26T20:58:08.745614+0000","last_change":"2024-04-26T20:58:08.745614+0000","last_active":"2024-04-26T20:58:07.715649+0000","last_peered":"2024-04-26T20:58:07.715649+0000","last_clean":"2024-04-26T20:58:07.715649+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.745614+0000","last_undegraded":"2024-04-26T20:58:08.745614+0000","last_fullsized":"2024-04-26T20:58:08.745614+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.752830+0000","last_change":"2024-04-26T20:58:08.752830+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.752830+0000","last_undegraded":"2024-04-26T20:58:08.752830+0000","last_fullsized":"2024-04-26T20:58:08.752830+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-26T20:58:08.747940+0000","last_change":"2024-04-26T20:58:08.747940+0000","last_active":"2024-04-26T20:58:07.715649+0000","last_peered":"2024-04-26T20:58:07.715649+0000","last_clean":"2024-04-26T20:58:07.715649+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.747940+0000","last_undegraded":"2024-04-26T20:58:08.747940+0000","last_fullsized":"2024-04-26T20:58:08.747940+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.749694+0000","last_change":"2024-04-26T20:58:08.749694+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.749694+0000","last_undegraded":"2024-04-26T20:58:08.749694+0000","last_fullsized":"2024-04-26T20:58:08.749694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-26T20:58:08.751793+0000","last_change":"2024-04-26T20:58:08.751793+0000","last_active":"2024-04-26T20:58:07.715649+0000","last_peered":"2024-04-26T20:58:07.715649+0000","last_clean":"2024-04-26T20:58:07.715649+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.751793+0000","last_undegraded":"2024-04-26T20:58:08.751793+0000","last_fullsized":"2024-04-26T20:58:08.751793+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.748437+0000","last_change":"2024-04-26T20:58:08.748437+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.748437+0000","last_undegraded":"2024-04-26T20:58:08.748437+0000","last_fullsized":"2024-04-26T20:58:08.748437+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":8,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T20:58:08.763704+0000","last_change":"2024-04-26T20:58:08.763704+0000","last_active":"2024-04-26T20:58:08.763704+0000","last_peered":"2024-04-26T20:58:08.763704+0000","last_clean":"2024-04-26T20:58:08.763704+0000","last_became_active":"2024-04-26T20:58:08.763378+0000","last_became_peered":"2024-04-26T20:58:08.763378+0000","last_unstale":"2024-04-26T20:58:08.763704+0000","last_undegraded":"2024-04-26T20:58:08.763704+0000","last_fullsized":"2024-04-26T20:58:08.763704+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:59.389988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.762985+0000","last_change":"2024-04-26T20:58:08.762985+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.762985+0000","last_undegraded":"2024-04-26T20:58:08.762985+0000","last_fullsized":"2024-04-26T20:58:08.762985+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-26T20:58:08.749782+0000","last_change":"2024-04-26T20:58:08.749782+0000","last_active":"2024-04-26T20:58:07.715649+0000","last_peered":"2024-04-26T20:58:07.715649+0000","last_clean":"2024-04-26T20:58:07.715649+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.749782+0000","last_undegraded":"2024-04-26T20:58:08.749782+0000","last_fullsized":"2024-04-26T20:58:08.749782+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.752090+0000","last_change":"2024-04-26T20:58:08.752090+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.752090+0000","last_undegraded":"2024-04-26T20:58:08.752090+0000","last_fullsized":"2024-04-26T20:58:08.752090+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-26T20:58:08.750012+0000","last_change":"2024-04-26T20:58:08.750012+0000","last_active":"2024-04-26T20:58:07.715649+0000","last_peered":"2024-04-26T20:58:07.715649+0000","last_clean":"2024-04-26T20:58:07.715649+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.750012+0000","last_undegraded":"2024-04-26T20:58:08.750012+0000","last_fullsized":"2024-04-26T20:58:08.750012+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":["1","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.746722+0000","last_change":"2024-04-26T20:58:08.746722+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.746722+0000","last_undegraded":"2024-04-26T20:58:08.746722+0000","last_fullsized":"2024-04-26T20:58:08.746722+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T20:58:08.771396+0000","last_change":"2024-04-26T20:58:08.762291+0000","last_active":"2024-04-26T20:58:08.771396+0000","last_peered":"2024-04-26T20:58:08.771396+0000","last_clean":"2024-04-26T20:58:08.771396+0000","last_became_active":"2024-04-26T20:58:08.761994+0000","last_became_peered":"2024-04-26T20:58:08.761994+0000","last_unstale":"2024-04-26T20:58:08.771396+0000","last_undegraded":"2024-04-26T20:58:08.771396+0000","last_fullsized":"2024-04-26T20:58:08.771396+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:56:51.025506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.747364+0000","last_change":"2024-04-26T20:58:08.747364+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.747364+0000","last_undegraded":"2024-04-26T20:58:08.747364+0000","last_fullsized":"2024-04-26T20:58:08.747364+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":3,"reported_epoch":32,"state":"creating+activating","last_fresh":"2024-04-26T20:58:08.741112+0000","last_change":"2024-04-26T20:58:08.741112+0000","last_active":"2024-04-26T20:58:07.715649+0000","last_peered":"2024-04-26T20:58:07.715649+0000","last_clean":"2024-04-26T20:58:07.715649+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.741112+0000","last_undegraded":"2024-04-26T20:58:08.741112+0000","last_fullsized":"2024-04-26T20:58:08.741112+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-04-26T20:58:08.756904+0000","last_change":"2024-04-26T20:58:08.756904+0000","last_active":"2024-04-26T20:58:08.728179+0000","last_peered":"2024-04-26T20:58:08.728179+0000","last_clean":"2024-04-26T20:58:08.728179+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:08.756904+0000","last_undegraded":"2024-04-26T20:58:08.756904+0000","last_fullsized":"2024-04-26T20:58:08.756904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-26T20:58:07.746204+0000","last_change":"2024-04-26T20:58:07.746204+0000","last_active":"2024-04-26T20:58:07.715649+0000","last_peered":"2024-04-26T20:58:07.715649+0000","last_clean":"2024-04-26T20:58:07.715649+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:07.746204+0000","last_undegraded":"2024-04-26T20:58:07.746204+0000","last_fullsized":"2024-04-26T20:58:07.746204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":191,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T20:58:08.763513+0000","last_change":"2024-04-26T20:56:46.848603+0000","last_active":"2024-04-26T20:58:08.763513+0000","last_peered":"2024-04-26T20:58:08.763513+0000","last_clean":"2024-04-26T20:58:08.763513+0000","last_became_active":"2024-04-26T20:56:46.846091+0000","last_became_peered":"2024-04-26T20:56:46.846091+0000","last_unstale":"2024-04-26T20:58:08.763513+0000","last_undegraded":"2024-04-26T20:58:08.763513+0000","last_fullsized":"2024-04-26T20:58:08.763513+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_clean_scrub_stamp":"2024-04-26T20:56:45.594041+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:00.232387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":722313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":131945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":348160,"data_stored":131945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":348160,"data_stored":131945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31520,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340320,"statfs":{"total":96636764160,"available":96604487680,"internally_reserved":0,"allocated":348160,"data_stored":131945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":942080,"data_stored":722313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:58:14.289 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:58:14.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.539+0000 7f4fa5000640 1 Processor -- start 2024-04-26T20:58:14.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.539+0000 7f4fa5000640 1 -- start start 2024-04-26T20:58:14.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.541+0000 7f4fa5000640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f4fa0146060 0x7f4fa0071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:14.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.541+0000 7f4fa5000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4fa0146670 0x7f4fa01609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:14.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.541+0000 7f4fa5000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4fa0160ef0 0x7f4fa006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:14.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.541+0000 7f4fa5000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f4fa006f510 con 0x7f4fa0146670 2024-04-26T20:58:14.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.541+0000 7f4fa5000640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f4fa0146440 con 0x7f4fa0146060 2024-04-26T20:58:14.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.541+0000 7f4fa5000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f4fa006f010 con 0x7f4fa0160ef0 2024-04-26T20:58:14.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.541+0000 7f4f97e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4fa0146670 0x7f4fa01609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:14.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.541+0000 7f4f97e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4fa0146670 0x7f4fa01609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52002/0 (socket says 172.21.15.28:52002) 2024-04-26T20:58:14.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.541+0000 7f4f97e00640 1 -- 172.21.15.28:0/1489643238 learned_addr learned my addr 172.21.15.28:0/1489643238 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:14.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.542+0000 7f4f9f400640 1 --2- 172.21.15.28:0/1489643238 >> v2:172.21.15.103:3300/0 conn(0x7f4fa0146060 0x7f4fa0071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:14.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.542+0000 7f4f9fe00640 1 --2- 172.21.15.28:0/1489643238 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4fa0160ef0 0x7f4fa006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:14.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.542+0000 7f4f97e00640 1 -- 172.21.15.28:0/1489643238 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4fa0160ef0 msgr2=0x7f4fa006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:14.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.542+0000 7f4f97e00640 1 --2- 172.21.15.28:0/1489643238 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4fa0160ef0 0x7f4fa006de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.542+0000 7f4f97e00640 1 -- 172.21.15.28:0/1489643238 >> v2:172.21.15.103:3300/0 conn(0x7f4fa0146060 msgr2=0x7f4fa0071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:14.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.542+0000 7f4f97e00640 1 --2- 172.21.15.28:0/1489643238 >> v2:172.21.15.103:3300/0 conn(0x7f4fa0146060 0x7f4fa0071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.542+0000 7f4f97e00640 1 -- 172.21.15.28:0/1489643238 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa01617a0 con 0x7f4fa0146670 2024-04-26T20:58:14.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.542+0000 7f4f9fe00640 1 --2- 172.21.15.28:0/1489643238 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4fa0160ef0 0x7f4fa006de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:14.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.543+0000 7f4f97e00640 1 --2- 172.21.15.28:0/1489643238 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fa0146670 0x7f4fa01609b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4fa006d870 tx=0x7f4f8802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c64fb81c1e6537c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:14.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.543+0000 7f4f9ea00640 1 -- 172.21.15.28:0/1489643238 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f88037070 con 0x7f4fa0146670 2024-04-26T20:58:14.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.543+0000 7f4f9ea00640 1 -- 172.21.15.28:0/1489643238 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f88032440 con 0x7f4fa0146670 2024-04-26T20:58:14.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.543+0000 7f4f9ea00640 1 -- 172.21.15.28:0/1489643238 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f88032810 con 0x7f4fa0146670 2024-04-26T20:58:14.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.544+0000 7f4fa5000640 1 -- 172.21.15.28:0/1489643238 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fa0146670 msgr2=0x7f4fa01609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:14.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.544+0000 7f4fa5000640 1 --2- 172.21.15.28:0/1489643238 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fa0146670 0x7f4fa01609b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4fa006d870 tx=0x7f4f8802a140 comp rx=0 tx=0).stop 2024-04-26T20:58:14.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.544+0000 7f4fa5000640 1 -- 172.21.15.28:0/1489643238 shutdown_connections 2024-04-26T20:58:14.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.544+0000 7f4fa5000640 1 --2- 172.21.15.28:0/1489643238 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4fa0160ef0 0x7f4fa006de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.544+0000 7f4fa5000640 1 --2- 172.21.15.28:0/1489643238 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fa0146670 0x7f4fa01609b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.544+0000 7f4fa5000640 1 --2- 172.21.15.28:0/1489643238 >> v2:172.21.15.103:3300/0 conn(0x7f4fa0146060 0x7f4fa0071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.544+0000 7f4fa5000640 1 -- 172.21.15.28:0/1489643238 >> 172.21.15.28:0/1489643238 conn(0x7f4fa00d0a50 msgr2=0x7f4fa00d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:14.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.544+0000 7f4fa5000640 1 -- 172.21.15.28:0/1489643238 shutdown_connections 2024-04-26T20:58:14.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.544+0000 7f4fa5000640 1 -- 172.21.15.28:0/1489643238 wait complete. 2024-04-26T20:58:14.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.546+0000 7f4fa5000640 1 Processor -- start 2024-04-26T20:58:14.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.546+0000 7f4fa5000640 1 -- start start 2024-04-26T20:58:14.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.547+0000 7f4fa5000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fa0146060 0x7f4fa00ab030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:14.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.547+0000 7f4fa5000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4fa0146670 0x7f4fa00c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:14.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.547+0000 7f4fa5000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fa0160ef0 0x7f4fa00c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:14.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.547+0000 7f4fa5000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa0161340 con 0x7f4fa0146060 2024-04-26T20:58:14.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.547+0000 7f4fa5000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa006ff60 con 0x7f4fa0160ef0 2024-04-26T20:58:14.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.547+0000 7f4fa5000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f4fa01614c0 con 0x7f4fa0146670 2024-04-26T20:58:14.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.548+0000 7f4f9f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fa0146060 0x7f4fa00ab030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:14.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.548+0000 7f4f9f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fa0146060 0x7f4fa00ab030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52016/0 (socket says 172.21.15.28:52016) 2024-04-26T20:58:14.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.548+0000 7f4f9f400640 1 -- 172.21.15.28:0/1154973833 learned_addr learned my addr 172.21.15.28:0/1154973833 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:14.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.548+0000 7f4f9fe00640 1 --2- 172.21.15.28:0/1154973833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fa0160ef0 0x7f4fa00c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:14.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.548+0000 7f4f97e00640 1 --2- 172.21.15.28:0/1154973833 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4fa0146670 0x7f4fa00c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:14.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.548+0000 7f4f9f400640 1 -- 172.21.15.28:0/1154973833 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4fa0146670 msgr2=0x7f4fa00c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:14.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.548+0000 7f4f9f400640 1 --2- 172.21.15.28:0/1154973833 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4fa0146670 0x7f4fa00c5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.548+0000 7f4f9f400640 1 -- 172.21.15.28:0/1154973833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fa0160ef0 msgr2=0x7f4fa00c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:14.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.548+0000 7f4f9f400640 1 --2- 172.21.15.28:0/1154973833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fa0160ef0 0x7f4fa00c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.548+0000 7f4f9f400640 1 -- 172.21.15.28:0/1154973833 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa00c6570 con 0x7f4fa0146060 2024-04-26T20:58:14.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.548+0000 7f4f9fe00640 1 --2- 172.21.15.28:0/1154973833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fa0160ef0 0x7f4fa00c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:58:14.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.548+0000 7f4f9f400640 1 --2- 172.21.15.28:0/1154973833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fa0146060 0x7f4fa00ab030 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000d800 tx=0x7f4f9000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:14.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.549+0000 7f4f97400640 1 -- 172.21.15.28:0/1154973833 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f90018030 con 0x7f4fa0146060 2024-04-26T20:58:14.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.549+0000 7f4f97400640 1 -- 172.21.15.28:0/1154973833 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f900096a0 con 0x7f4fa0146060 2024-04-26T20:58:14.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.549+0000 7f4f97400640 1 -- 172.21.15.28:0/1154973833 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f90009870 con 0x7f4fa0146060 2024-04-26T20:58:14.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.549+0000 7f4fa5000640 1 -- 172.21.15.28:0/1154973833 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa00c6850 con 0x7f4fa0146060 2024-04-26T20:58:14.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.549+0000 7f4fa5000640 1 -- 172.21.15.28:0/1154973833 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa00c6e10 con 0x7f4fa0146060 2024-04-26T20:58:14.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.550+0000 7f4fa5000640 1 -- 172.21.15.28:0/1154973833 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fa006a360 con 0x7f4fa0146060 2024-04-26T20:58:14.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.551+0000 7f4f97400640 1 -- 172.21.15.28:0/1154973833 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f9001b070 con 0x7f4fa0146060 2024-04-26T20:58:14.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.552+0000 7f4f97400640 1 --2- 172.21.15.28:0/1154973833 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4f7c06c8f0 0x7f4f7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:14.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.552+0000 7f4f97400640 1 -- 172.21.15.28:0/1154973833 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f4f9008be80 con 0x7f4fa0146060 2024-04-26T20:58:14.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.552+0000 7f4f97e00640 1 --2- 172.21.15.28:0/1154973833 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4f7c06c8f0 0x7f4f7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:14.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.553+0000 7f4f97e00640 1 --2- 172.21.15.28:0/1154973833 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4f7c06c8f0 0x7f4f7c06edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4f88034000 tx=0x7f4f8802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:14.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.556+0000 7f4f97400640 1 -- 172.21.15.28:0/1154973833 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4f9005a210 con 0x7f4fa0146060 2024-04-26T20:58:14.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.749+0000 7f4fa5000640 1 -- 172.21.15.28:0/1154973833 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4fa00cb2f0 con 0x7f4f7c06c8f0 2024-04-26T20:58:14.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.768+0000 7f4f97400640 1 -- 172.21.15.28:0/1154973833 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251951 (secure 0 0 0) 0x7f4fa00cb2f0 con 0x7f4f7c06c8f0 2024-04-26T20:58:14.772 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:14.774 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T20:58:14.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.773+0000 7f4fa5000640 1 -- 172.21.15.28:0/1154973833 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4f7c06c8f0 msgr2=0x7f4f7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:14.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.773+0000 7f4fa5000640 1 --2- 172.21.15.28:0/1154973833 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4f7c06c8f0 0x7f4f7c06edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4f88034000 tx=0x7f4f8802e000 comp rx=0 tx=0).stop 2024-04-26T20:58:14.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.773+0000 7f4fa5000640 1 -- 172.21.15.28:0/1154973833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fa0146060 msgr2=0x7f4fa00ab030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:14.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.773+0000 7f4fa5000640 1 --2- 172.21.15.28:0/1154973833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fa0146060 0x7f4fa00ab030 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000d800 tx=0x7f4f9000dcc0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.773+0000 7f4fa5000640 1 -- 172.21.15.28:0/1154973833 shutdown_connections 2024-04-26T20:58:14.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.773+0000 7f4fa5000640 1 --2- 172.21.15.28:0/1154973833 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4f7c06c8f0 0x7f4f7c06edb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.773+0000 7f4fa5000640 1 --2- 172.21.15.28:0/1154973833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4fa0160ef0 0x7f4fa00c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.773+0000 7f4fa5000640 1 --2- 172.21.15.28:0/1154973833 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4fa0146670 0x7f4fa00c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.773+0000 7f4fa5000640 1 --2- 172.21.15.28:0/1154973833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4fa0146060 0x7f4fa00ab030 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:14.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.773+0000 7f4fa5000640 1 -- 172.21.15.28:0/1154973833 >> 172.21.15.28:0/1154973833 conn(0x7f4fa00d0a50 msgr2=0x7f4fa00b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:14.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.773+0000 7f4fa5000640 1 -- 172.21.15.28:0/1154973833 shutdown_connections 2024-04-26T20:58:14.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:14.773+0000 7f4fa5000640 1 -- 172.21.15.28:0/1154973833 wait complete. 2024-04-26T20:58:14.792 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-04-26T20:58:14.731588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237072,"kb_used_data":4232,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737648,"statfs":{"total":773094113280,"available":772851351552,"internally_reserved":0,"allocated":4333568,"data_stored":2395852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001629"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.769597+0000","last_change":"2024-04-26T20:58:09.796133+0000","last_active":"2024-04-26T20:58:10.769597+0000","last_peered":"2024-04-26T20:58:10.769597+0000","last_clean":"2024-04-26T20:58:10.769597+0000","last_became_active":"2024-04-26T20:58:09.795791+0000","last_became_peered":"2024-04-26T20:58:09.795791+0000","last_unstale":"2024-04-26T20:58:10.769597+0000","last_undegraded":"2024-04-26T20:58:10.769597+0000","last_fullsized":"2024-04-26T20:58:10.769597+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:17:07.322131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.765985+0000","last_change":"2024-04-26T20:58:09.800762+0000","last_active":"2024-04-26T20:58:10.765985+0000","last_peered":"2024-04-26T20:58:10.765985+0000","last_clean":"2024-04-26T20:58:10.765985+0000","last_became_active":"2024-04-26T20:58:09.800478+0000","last_became_peered":"2024-04-26T20:58:09.800478+0000","last_unstale":"2024-04-26T20:58:10.765985+0000","last_undegraded":"2024-04-26T20:58:10.765985+0000","last_fullsized":"2024-04-26T20:58:10.765985+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:14.684384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.069558+0000","last_change":"2024-04-26T20:58:09.801633+0000","last_active":"2024-04-26T20:58:11.069558+0000","last_peered":"2024-04-26T20:58:11.069558+0000","last_clean":"2024-04-26T20:58:11.069558+0000","last_became_active":"2024-04-26T20:58:09.801363+0000","last_became_peered":"2024-04-26T20:58:09.801363+0000","last_unstale":"2024-04-26T20:58:11.069558+0000","last_undegraded":"2024-04-26T20:58:11.069558+0000","last_fullsized":"2024-04-26T20:58:11.069558+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:58.708037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.823014+0000","last_change":"2024-04-26T20:58:09.809432+0000","last_active":"2024-04-26T20:58:10.823014+0000","last_peered":"2024-04-26T20:58:10.823014+0000","last_clean":"2024-04-26T20:58:10.823014+0000","last_became_active":"2024-04-26T20:58:09.808664+0000","last_became_peered":"2024-04-26T20:58:09.808664+0000","last_unstale":"2024-04-26T20:58:10.823014+0000","last_undegraded":"2024-04-26T20:58:10.823014+0000","last_fullsized":"2024-04-26T20:58:10.823014+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:41:19.722253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.064843+0000","last_change":"2024-04-26T20:58:09.806552+0000","last_active":"2024-04-26T20:58:11.064843+0000","last_peered":"2024-04-26T20:58:11.064843+0000","last_clean":"2024-04-26T20:58:11.064843+0000","last_became_active":"2024-04-26T20:58:09.805083+0000","last_became_peered":"2024-04-26T20:58:09.805083+0000","last_unstale":"2024-04-26T20:58:11.064843+0000","last_undegraded":"2024-04-26T20:58:11.064843+0000","last_fullsized":"2024-04-26T20:58:11.064843+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:45:26.363652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.764764+0000","last_change":"2024-04-26T20:58:09.804559+0000","last_active":"2024-04-26T20:58:10.764764+0000","last_peered":"2024-04-26T20:58:10.764764+0000","last_clean":"2024-04-26T20:58:10.764764+0000","last_became_active":"2024-04-26T20:58:09.803963+0000","last_became_peered":"2024-04-26T20:58:09.803963+0000","last_unstale":"2024-04-26T20:58:10.764764+0000","last_undegraded":"2024-04-26T20:58:10.764764+0000","last_fullsized":"2024-04-26T20:58:10.764764+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:58.708037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.771564+0000","last_change":"2024-04-26T20:58:09.814097+0000","last_active":"2024-04-26T20:58:10.771564+0000","last_peered":"2024-04-26T20:58:10.771564+0000","last_clean":"2024-04-26T20:58:10.771564+0000","last_became_active":"2024-04-26T20:58:09.813427+0000","last_became_peered":"2024-04-26T20:58:09.813427+0000","last_unstale":"2024-04-26T20:58:10.771564+0000","last_undegraded":"2024-04-26T20:58:10.771564+0000","last_fullsized":"2024-04-26T20:58:10.771564+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:41.072543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.770790+0000","last_change":"2024-04-26T20:58:09.809435+0000","last_active":"2024-04-26T20:58:10.770790+0000","last_peered":"2024-04-26T20:58:10.770790+0000","last_clean":"2024-04-26T20:58:10.770790+0000","last_became_active":"2024-04-26T20:58:09.808191+0000","last_became_peered":"2024-04-26T20:58:09.808191+0000","last_unstale":"2024-04-26T20:58:10.770790+0000","last_undegraded":"2024-04-26T20:58:10.770790+0000","last_fullsized":"2024-04-26T20:58:10.770790+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:04:02.282206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.069017+0000","last_change":"2024-04-26T20:58:09.800035+0000","last_active":"2024-04-26T20:58:11.069017+0000","last_peered":"2024-04-26T20:58:11.069017+0000","last_clean":"2024-04-26T20:58:11.069017+0000","last_became_active":"2024-04-26T20:58:09.799734+0000","last_became_peered":"2024-04-26T20:58:09.799734+0000","last_unstale":"2024-04-26T20:58:11.069017+0000","last_undegraded":"2024-04-26T20:58:11.069017+0000","last_fullsized":"2024-04-26T20:58:11.069017+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:23.309838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.823800+0000","last_change":"2024-04-26T20:58:09.803133+0000","last_active":"2024-04-26T20:58:10.823800+0000","last_peered":"2024-04-26T20:58:10.823800+0000","last_clean":"2024-04-26T20:58:10.823800+0000","last_became_active":"2024-04-26T20:58:09.802595+0000","last_became_peered":"2024-04-26T20:58:09.802595+0000","last_unstale":"2024-04-26T20:58:10.823800+0000","last_undegraded":"2024-04-26T20:58:10.823800+0000","last_fullsized":"2024-04-26T20:58:10.823800+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:55.629400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.771592+0000","last_change":"2024-04-26T20:58:09.812479+0000","last_active":"2024-04-26T20:58:10.771592+0000","last_peered":"2024-04-26T20:58:10.771592+0000","last_clean":"2024-04-26T20:58:10.771592+0000","last_became_active":"2024-04-26T20:58:09.812107+0000","last_became_peered":"2024-04-26T20:58:09.812107+0000","last_unstale":"2024-04-26T20:58:10.771592+0000","last_undegraded":"2024-04-26T20:58:10.771592+0000","last_fullsized":"2024-04-26T20:58:10.771592+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:59.526836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.068921+0000","last_change":"2024-04-26T20:58:09.805530+0000","last_active":"2024-04-26T20:58:11.068921+0000","last_peered":"2024-04-26T20:58:11.068921+0000","last_clean":"2024-04-26T20:58:11.068921+0000","last_became_active":"2024-04-26T20:58:09.805193+0000","last_became_peered":"2024-04-26T20:58:09.805193+0000","last_unstale":"2024-04-26T20:58:11.068921+0000","last_undegraded":"2024-04-26T20:58:11.068921+0000","last_fullsized":"2024-04-26T20:58:11.068921+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:34:56.854941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.064811+0000","last_change":"2024-04-26T20:58:09.799222+0000","last_active":"2024-04-26T20:58:11.064811+0000","last_peered":"2024-04-26T20:58:11.064811+0000","last_clean":"2024-04-26T20:58:11.064811+0000","last_became_active":"2024-04-26T20:58:09.798434+0000","last_became_peered":"2024-04-26T20:58:09.798434+0000","last_unstale":"2024-04-26T20:58:11.064811+0000","last_undegraded":"2024-04-26T20:58:11.064811+0000","last_fullsized":"2024-04-26T20:58:11.064811+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:15.134320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.764992+0000","last_change":"2024-04-26T20:58:09.795847+0000","last_active":"2024-04-26T20:58:10.764992+0000","last_peered":"2024-04-26T20:58:10.764992+0000","last_clean":"2024-04-26T20:58:10.764992+0000","last_became_active":"2024-04-26T20:58:09.795391+0000","last_became_peered":"2024-04-26T20:58:09.795391+0000","last_unstale":"2024-04-26T20:58:10.764992+0000","last_undegraded":"2024-04-26T20:58:10.764992+0000","last_fullsized":"2024-04-26T20:58:10.764992+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:05.917326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.067970+0000","last_change":"2024-04-26T20:58:09.794471+0000","last_active":"2024-04-26T20:58:11.067970+0000","last_peered":"2024-04-26T20:58:11.067970+0000","last_clean":"2024-04-26T20:58:11.067970+0000","last_became_active":"2024-04-26T20:58:09.793912+0000","last_became_peered":"2024-04-26T20:58:09.793912+0000","last_unstale":"2024-04-26T20:58:11.067970+0000","last_undegraded":"2024-04-26T20:58:11.067970+0000","last_fullsized":"2024-04-26T20:58:11.067970+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:49.071822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.772300+0000","last_change":"2024-04-26T20:58:09.810587+0000","last_active":"2024-04-26T20:58:10.772300+0000","last_peered":"2024-04-26T20:58:10.772300+0000","last_clean":"2024-04-26T20:58:10.772300+0000","last_became_active":"2024-04-26T20:58:09.810323+0000","last_became_peered":"2024-04-26T20:58:09.810323+0000","last_unstale":"2024-04-26T20:58:10.772300+0000","last_undegraded":"2024-04-26T20:58:10.772300+0000","last_fullsized":"2024-04-26T20:58:10.772300+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:02:58.708037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.772311+0000","last_change":"2024-04-26T20:58:09.811768+0000","last_active":"2024-04-26T20:58:10.772311+0000","last_peered":"2024-04-26T20:58:10.772311+0000","last_clean":"2024-04-26T20:58:10.772311+0000","last_became_active":"2024-04-26T20:58:09.811483+0000","last_became_peered":"2024-04-26T20:58:09.811483+0000","last_unstale":"2024-04-26T20:58:10.772311+0000","last_undegraded":"2024-04-26T20:58:10.772311+0000","last_fullsized":"2024-04-26T20:58:10.772311+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:05.917326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.766037+0000","last_change":"2024-04-26T20:58:09.802729+0000","last_active":"2024-04-26T20:58:10.766037+0000","last_peered":"2024-04-26T20:58:10.766037+0000","last_clean":"2024-04-26T20:58:10.766037+0000","last_became_active":"2024-04-26T20:58:09.802448+0000","last_became_peered":"2024-04-26T20:58:09.802448+0000","last_unstale":"2024-04-26T20:58:10.766037+0000","last_undegraded":"2024-04-26T20:58:10.766037+0000","last_fullsized":"2024-04-26T20:58:10.766037+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:55.245490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.765432+0000","last_change":"2024-04-26T20:58:09.812244+0000","last_active":"2024-04-26T20:58:10.765432+0000","last_peered":"2024-04-26T20:58:10.765432+0000","last_clean":"2024-04-26T20:58:10.765432+0000","last_became_active":"2024-04-26T20:58:09.811074+0000","last_became_peered":"2024-04-26T20:58:09.811074+0000","last_unstale":"2024-04-26T20:58:10.765432+0000","last_undegraded":"2024-04-26T20:58:10.765432+0000","last_fullsized":"2024-04-26T20:58:10.765432+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:55:59.445354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.064474+0000","last_change":"2024-04-26T20:58:09.798345+0000","last_active":"2024-04-26T20:58:11.064474+0000","last_peered":"2024-04-26T20:58:11.064474+0000","last_clean":"2024-04-26T20:58:11.064474+0000","last_became_active":"2024-04-26T20:58:09.798034+0000","last_became_peered":"2024-04-26T20:58:09.798034+0000","last_unstale":"2024-04-26T20:58:11.064474+0000","last_undegraded":"2024-04-26T20:58:11.064474+0000","last_fullsized":"2024-04-26T20:58:11.064474+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:44.635326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.066903+0000","last_change":"2024-04-26T20:58:09.782748+0000","last_active":"2024-04-26T20:58:11.066903+0000","last_peered":"2024-04-26T20:58:11.066903+0000","last_clean":"2024-04-26T20:58:11.066903+0000","last_became_active":"2024-04-26T20:58:09.781927+0000","last_became_peered":"2024-04-26T20:58:09.781927+0000","last_unstale":"2024-04-26T20:58:11.066903+0000","last_undegraded":"2024-04-26T20:58:11.066903+0000","last_fullsized":"2024-04-26T20:58:11.066903+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:24.012274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.067385+0000","last_change":"2024-04-26T20:58:09.790427+0000","last_active":"2024-04-26T20:58:11.067385+0000","last_peered":"2024-04-26T20:58:11.067385+0000","last_clean":"2024-04-26T20:58:11.067385+0000","last_became_active":"2024-04-26T20:58:09.789620+0000","last_became_peered":"2024-04-26T20:58:09.789620+0000","last_unstale":"2024-04-26T20:58:11.067385+0000","last_undegraded":"2024-04-26T20:58:11.067385+0000","last_fullsized":"2024-04-26T20:58:11.067385+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:39.977180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.816397+0000","last_change":"2024-04-26T20:58:09.793634+0000","last_active":"2024-04-26T20:58:10.816397+0000","last_peered":"2024-04-26T20:58:10.816397+0000","last_clean":"2024-04-26T20:58:10.816397+0000","last_became_active":"2024-04-26T20:58:09.787292+0000","last_became_peered":"2024-04-26T20:58:09.787292+0000","last_unstale":"2024-04-26T20:58:10.816397+0000","last_undegraded":"2024-04-26T20:58:10.816397+0000","last_fullsized":"2024-04-26T20:58:10.816397+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:53.718524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.067361+0000","last_change":"2024-04-26T20:58:09.809531+0000","last_active":"2024-04-26T20:58:11.067361+0000","last_peered":"2024-04-26T20:58:11.067361+0000","last_clean":"2024-04-26T20:58:11.067361+0000","last_became_active":"2024-04-26T20:58:09.809125+0000","last_became_peered":"2024-04-26T20:58:09.809125+0000","last_unstale":"2024-04-26T20:58:11.067361+0000","last_undegraded":"2024-04-26T20:58:11.067361+0000","last_fullsized":"2024-04-26T20:58:11.067361+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:01.425446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.823819+0000","last_change":"2024-04-26T20:58:09.814090+0000","last_active":"2024-04-26T20:58:10.823819+0000","last_peered":"2024-04-26T20:58:10.823819+0000","last_clean":"2024-04-26T20:58:10.823819+0000","last_became_active":"2024-04-26T20:58:09.812308+0000","last_became_peered":"2024-04-26T20:58:09.812308+0000","last_unstale":"2024-04-26T20:58:10.823819+0000","last_undegraded":"2024-04-26T20:58:10.823819+0000","last_fullsized":"2024-04-26T20:58:10.823819+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:34:56.854941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.766188+0000","last_change":"2024-04-26T20:58:09.810622+0000","last_active":"2024-04-26T20:58:10.766188+0000","last_peered":"2024-04-26T20:58:10.766188+0000","last_clean":"2024-04-26T20:58:10.766188+0000","last_became_active":"2024-04-26T20:58:09.810304+0000","last_became_peered":"2024-04-26T20:58:09.810304+0000","last_unstale":"2024-04-26T20:58:10.766188+0000","last_undegraded":"2024-04-26T20:58:10.766188+0000","last_fullsized":"2024-04-26T20:58:10.766188+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:26.218141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.823722+0000","last_change":"2024-04-26T20:58:09.804580+0000","last_active":"2024-04-26T20:58:10.823722+0000","last_peered":"2024-04-26T20:58:10.823722+0000","last_clean":"2024-04-26T20:58:10.823722+0000","last_became_active":"2024-04-26T20:58:09.804005+0000","last_became_peered":"2024-04-26T20:58:09.804005+0000","last_unstale":"2024-04-26T20:58:10.823722+0000","last_undegraded":"2024-04-26T20:58:10.823722+0000","last_fullsized":"2024-04-26T20:58:10.823722+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:49.071822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.069112+0000","last_change":"2024-04-26T20:58:09.796445+0000","last_active":"2024-04-26T20:58:11.069112+0000","last_peered":"2024-04-26T20:58:11.069112+0000","last_clean":"2024-04-26T20:58:11.069112+0000","last_became_active":"2024-04-26T20:58:09.796056+0000","last_became_peered":"2024-04-26T20:58:09.796056+0000","last_unstale":"2024-04-26T20:58:11.069112+0000","last_undegraded":"2024-04-26T20:58:11.069112+0000","last_fullsized":"2024-04-26T20:58:11.069112+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:41.072543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.067899+0000","last_change":"2024-04-26T20:58:09.805967+0000","last_active":"2024-04-26T20:58:11.067899+0000","last_peered":"2024-04-26T20:58:11.067899+0000","last_clean":"2024-04-26T20:58:11.067899+0000","last_became_active":"2024-04-26T20:58:09.805636+0000","last_became_peered":"2024-04-26T20:58:09.805636+0000","last_unstale":"2024-04-26T20:58:11.067899+0000","last_undegraded":"2024-04-26T20:58:11.067899+0000","last_fullsized":"2024-04-26T20:58:11.067899+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:11.757090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.771234+0000","last_change":"2024-04-26T20:58:09.811890+0000","last_active":"2024-04-26T20:58:10.771234+0000","last_peered":"2024-04-26T20:58:10.771234+0000","last_clean":"2024-04-26T20:58:10.771234+0000","last_became_active":"2024-04-26T20:58:09.811578+0000","last_became_peered":"2024-04-26T20:58:09.811578+0000","last_unstale":"2024-04-26T20:58:10.771234+0000","last_undegraded":"2024-04-26T20:58:10.771234+0000","last_fullsized":"2024-04-26T20:58:10.771234+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:01.425446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.769002+0000","last_change":"2024-04-26T20:58:09.801719+0000","last_active":"2024-04-26T20:58:10.769002+0000","last_peered":"2024-04-26T20:58:10.769002+0000","last_clean":"2024-04-26T20:58:10.769002+0000","last_became_active":"2024-04-26T20:58:09.801477+0000","last_became_peered":"2024-04-26T20:58:09.801477+0000","last_unstale":"2024-04-26T20:58:10.769002+0000","last_undegraded":"2024-04-26T20:58:10.769002+0000","last_fullsized":"2024-04-26T20:58:10.769002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:24.012274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.770957+0000","last_change":"2024-04-26T20:58:09.807655+0000","last_active":"2024-04-26T20:58:10.770957+0000","last_peered":"2024-04-26T20:58:10.770957+0000","last_clean":"2024-04-26T20:58:10.770957+0000","last_became_active":"2024-04-26T20:58:09.806141+0000","last_became_peered":"2024-04-26T20:58:09.806141+0000","last_unstale":"2024-04-26T20:58:10.770957+0000","last_undegraded":"2024-04-26T20:58:10.770957+0000","last_fullsized":"2024-04-26T20:58:10.770957+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:34:56.854941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.817123+0000","last_change":"2024-04-26T20:58:09.805764+0000","last_active":"2024-04-26T20:58:10.817123+0000","last_peered":"2024-04-26T20:58:10.817123+0000","last_clean":"2024-04-26T20:58:10.817123+0000","last_became_active":"2024-04-26T20:58:09.805413+0000","last_became_peered":"2024-04-26T20:58:09.805413+0000","last_unstale":"2024-04-26T20:58:10.817123+0000","last_undegraded":"2024-04-26T20:58:10.817123+0000","last_fullsized":"2024-04-26T20:58:10.817123+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:11.757090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.770267+0000","last_change":"2024-04-26T20:58:08.777415+0000","last_active":"2024-04-26T20:58:10.770267+0000","last_peered":"2024-04-26T20:58:10.770267+0000","last_clean":"2024-04-26T20:58:10.770267+0000","last_became_active":"2024-04-26T20:58:08.777127+0000","last_became_peered":"2024-04-26T20:58:08.777127+0000","last_unstale":"2024-04-26T20:58:10.770267+0000","last_undegraded":"2024-04-26T20:58:10.770267+0000","last_fullsized":"2024-04-26T20:58:10.770267+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:32.017697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.770023+0000","last_change":"2024-04-26T20:58:09.795420+0000","last_active":"2024-04-26T20:58:10.770023+0000","last_peered":"2024-04-26T20:58:10.770023+0000","last_clean":"2024-04-26T20:58:10.770023+0000","last_became_active":"2024-04-26T20:58:09.795057+0000","last_became_peered":"2024-04-26T20:58:09.795057+0000","last_unstale":"2024-04-26T20:58:10.770023+0000","last_undegraded":"2024-04-26T20:58:10.770023+0000","last_fullsized":"2024-04-26T20:58:10.770023+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:11.757090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.817099+0000","last_change":"2024-04-26T20:58:08.774633+0000","last_active":"2024-04-26T20:58:10.817099+0000","last_peered":"2024-04-26T20:58:10.817099+0000","last_clean":"2024-04-26T20:58:10.817099+0000","last_became_active":"2024-04-26T20:58:08.774302+0000","last_became_peered":"2024-04-26T20:58:08.774302+0000","last_unstale":"2024-04-26T20:58:10.817099+0000","last_undegraded":"2024-04-26T20:58:10.817099+0000","last_fullsized":"2024-04-26T20:58:10.817099+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:24.623434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.067864+0000","last_change":"2024-04-26T20:58:09.795203+0000","last_active":"2024-04-26T20:58:11.067864+0000","last_peered":"2024-04-26T20:58:11.067864+0000","last_clean":"2024-04-26T20:58:11.067864+0000","last_became_active":"2024-04-26T20:58:09.794918+0000","last_became_peered":"2024-04-26T20:58:09.794918+0000","last_unstale":"2024-04-26T20:58:11.067864+0000","last_undegraded":"2024-04-26T20:58:11.067864+0000","last_fullsized":"2024-04-26T20:58:11.067864+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:17:07.322131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.765235+0000","last_change":"2024-04-26T20:58:08.764290+0000","last_active":"2024-04-26T20:58:10.765235+0000","last_peered":"2024-04-26T20:58:10.765235+0000","last_clean":"2024-04-26T20:58:10.765235+0000","last_became_active":"2024-04-26T20:58:08.763983+0000","last_became_peered":"2024-04-26T20:58:08.763983+0000","last_unstale":"2024-04-26T20:58:10.765235+0000","last_undegraded":"2024-04-26T20:58:10.765235+0000","last_fullsized":"2024-04-26T20:58:10.765235+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:04:22.297307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.823887+0000","last_change":"2024-04-26T20:58:09.804321+0000","last_active":"2024-04-26T20:58:10.823887+0000","last_peered":"2024-04-26T20:58:10.823887+0000","last_clean":"2024-04-26T20:58:10.823887+0000","last_became_active":"2024-04-26T20:58:09.803672+0000","last_became_peered":"2024-04-26T20:58:09.803672+0000","last_unstale":"2024-04-26T20:58:10.823887+0000","last_undegraded":"2024-04-26T20:58:10.823887+0000","last_fullsized":"2024-04-26T20:58:10.823887+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:41.072543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.064584+0000","last_change":"2024-04-26T20:58:08.769123+0000","last_active":"2024-04-26T20:58:11.064584+0000","last_peered":"2024-04-26T20:58:11.064584+0000","last_clean":"2024-04-26T20:58:11.064584+0000","last_became_active":"2024-04-26T20:58:08.767514+0000","last_became_peered":"2024-04-26T20:58:08.767514+0000","last_unstale":"2024-04-26T20:58:11.064584+0000","last_undegraded":"2024-04-26T20:58:11.064584+0000","last_fullsized":"2024-04-26T20:58:11.064584+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:48.313805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.771353+0000","last_change":"2024-04-26T20:58:09.811780+0000","last_active":"2024-04-26T20:58:10.771353+0000","last_peered":"2024-04-26T20:58:10.771353+0000","last_clean":"2024-04-26T20:58:10.771353+0000","last_became_active":"2024-04-26T20:58:09.811472+0000","last_became_peered":"2024-04-26T20:58:09.811472+0000","last_unstale":"2024-04-26T20:58:10.771353+0000","last_undegraded":"2024-04-26T20:58:10.771353+0000","last_fullsized":"2024-04-26T20:58:10.771353+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:18:49.071822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.067575+0000","last_change":"2024-04-26T20:58:08.766060+0000","last_active":"2024-04-26T20:58:11.067575+0000","last_peered":"2024-04-26T20:58:11.067575+0000","last_clean":"2024-04-26T20:58:11.067575+0000","last_became_active":"2024-04-26T20:58:08.765679+0000","last_became_peered":"2024-04-26T20:58:08.765679+0000","last_unstale":"2024-04-26T20:58:11.067575+0000","last_undegraded":"2024-04-26T20:58:11.067575+0000","last_fullsized":"2024-04-26T20:58:11.067575+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:59.716390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.068574+0000","last_change":"2024-04-26T20:58:09.793254+0000","last_active":"2024-04-26T20:58:11.068574+0000","last_peered":"2024-04-26T20:58:11.068574+0000","last_clean":"2024-04-26T20:58:11.068574+0000","last_became_active":"2024-04-26T20:58:09.792742+0000","last_became_peered":"2024-04-26T20:58:09.792742+0000","last_unstale":"2024-04-26T20:58:11.068574+0000","last_undegraded":"2024-04-26T20:58:11.068574+0000","last_fullsized":"2024-04-26T20:58:11.068574+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:55.629400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.067630+0000","last_change":"2024-04-26T20:58:08.764378+0000","last_active":"2024-04-26T20:58:11.067630+0000","last_peered":"2024-04-26T20:58:11.067630+0000","last_clean":"2024-04-26T20:58:11.067630+0000","last_became_active":"2024-04-26T20:58:08.764073+0000","last_became_peered":"2024-04-26T20:58:08.764073+0000","last_unstale":"2024-04-26T20:58:11.067630+0000","last_undegraded":"2024-04-26T20:58:11.067630+0000","last_fullsized":"2024-04-26T20:58:11.067630+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:31:37.807551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.066918+0000","last_change":"2024-04-26T20:58:09.787328+0000","last_active":"2024-04-26T20:58:11.066918+0000","last_peered":"2024-04-26T20:58:11.066918+0000","last_clean":"2024-04-26T20:58:11.066918+0000","last_became_active":"2024-04-26T20:58:09.786994+0000","last_became_peered":"2024-04-26T20:58:09.786994+0000","last_unstale":"2024-04-26T20:58:11.066918+0000","last_undegraded":"2024-04-26T20:58:11.066918+0000","last_fullsized":"2024-04-26T20:58:11.066918+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:53.718524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.764653+0000","last_change":"2024-04-26T20:58:08.761574+0000","last_active":"2024-04-26T20:58:10.764653+0000","last_peered":"2024-04-26T20:58:10.764653+0000","last_clean":"2024-04-26T20:58:10.764653+0000","last_became_active":"2024-04-26T20:58:08.760990+0000","last_became_peered":"2024-04-26T20:58:08.760990+0000","last_unstale":"2024-04-26T20:58:10.764653+0000","last_undegraded":"2024-04-26T20:58:10.764653+0000","last_fullsized":"2024-04-26T20:58:10.764653+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:17:06.309601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.822608+0000","last_change":"2024-04-26T20:58:09.799766+0000","last_active":"2024-04-26T20:58:10.822608+0000","last_peered":"2024-04-26T20:58:10.822608+0000","last_clean":"2024-04-26T20:58:10.822608+0000","last_became_active":"2024-04-26T20:58:09.799429+0000","last_became_peered":"2024-04-26T20:58:09.799429+0000","last_unstale":"2024-04-26T20:58:10.822608+0000","last_undegraded":"2024-04-26T20:58:10.822608+0000","last_fullsized":"2024-04-26T20:58:10.822608+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:31.333563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.066800+0000","last_change":"2024-04-26T20:58:08.768307+0000","last_active":"2024-04-26T20:58:11.066800+0000","last_peered":"2024-04-26T20:58:11.066800+0000","last_clean":"2024-04-26T20:58:11.066800+0000","last_became_active":"2024-04-26T20:58:08.767185+0000","last_became_peered":"2024-04-26T20:58:08.767185+0000","last_unstale":"2024-04-26T20:58:11.066800+0000","last_undegraded":"2024-04-26T20:58:11.066800+0000","last_fullsized":"2024-04-26T20:58:11.066800+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:54.505265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.069332+0000","last_change":"2024-04-26T20:58:09.796994+0000","last_active":"2024-04-26T20:58:11.069332+0000","last_peered":"2024-04-26T20:58:11.069332+0000","last_clean":"2024-04-26T20:58:11.069332+0000","last_became_active":"2024-04-26T20:58:09.796625+0000","last_became_peered":"2024-04-26T20:58:09.796625+0000","last_unstale":"2024-04-26T20:58:11.069332+0000","last_undegraded":"2024-04-26T20:58:11.069332+0000","last_fullsized":"2024-04-26T20:58:11.069332+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:59.526836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.765549+0000","last_change":"2024-04-26T20:58:08.774718+0000","last_active":"2024-04-26T20:58:10.765549+0000","last_peered":"2024-04-26T20:58:10.765549+0000","last_clean":"2024-04-26T20:58:10.765549+0000","last_became_active":"2024-04-26T20:58:08.774165+0000","last_became_peered":"2024-04-26T20:58:08.774165+0000","last_unstale":"2024-04-26T20:58:10.765549+0000","last_undegraded":"2024-04-26T20:58:10.765549+0000","last_fullsized":"2024-04-26T20:58:10.765549+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:21:11.156189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.765676+0000","last_change":"2024-04-26T20:58:09.811618+0000","last_active":"2024-04-26T20:58:10.765676+0000","last_peered":"2024-04-26T20:58:10.765676+0000","last_clean":"2024-04-26T20:58:10.765676+0000","last_became_active":"2024-04-26T20:58:09.811286+0000","last_became_peered":"2024-04-26T20:58:09.811286+0000","last_unstale":"2024-04-26T20:58:10.765676+0000","last_undegraded":"2024-04-26T20:58:10.765676+0000","last_fullsized":"2024-04-26T20:58:10.765676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:12.288967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.817442+0000","last_change":"2024-04-26T20:58:08.771820+0000","last_active":"2024-04-26T20:58:10.817442+0000","last_peered":"2024-04-26T20:58:10.817442+0000","last_clean":"2024-04-26T20:58:10.817442+0000","last_became_active":"2024-04-26T20:58:08.771511+0000","last_became_peered":"2024-04-26T20:58:08.771511+0000","last_unstale":"2024-04-26T20:58:10.817442+0000","last_undegraded":"2024-04-26T20:58:10.817442+0000","last_fullsized":"2024-04-26T20:58:10.817442+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:03.515785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.765341+0000","last_change":"2024-04-26T20:58:09.800094+0000","last_active":"2024-04-26T20:58:10.765341+0000","last_peered":"2024-04-26T20:58:10.765341+0000","last_clean":"2024-04-26T20:58:10.765341+0000","last_became_active":"2024-04-26T20:58:09.799812+0000","last_became_peered":"2024-04-26T20:58:09.799812+0000","last_unstale":"2024-04-26T20:58:10.765341+0000","last_undegraded":"2024-04-26T20:58:10.765341+0000","last_fullsized":"2024-04-26T20:58:10.765341+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:55.381181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.069696+0000","last_change":"2024-04-26T20:58:08.771996+0000","last_active":"2024-04-26T20:58:11.069696+0000","last_peered":"2024-04-26T20:58:11.069696+0000","last_clean":"2024-04-26T20:58:11.069696+0000","last_became_active":"2024-04-26T20:58:08.771701+0000","last_became_peered":"2024-04-26T20:58:08.771701+0000","last_unstale":"2024-04-26T20:58:11.069696+0000","last_undegraded":"2024-04-26T20:58:11.069696+0000","last_fullsized":"2024-04-26T20:58:11.069696+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:22.999743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.069367+0000","last_change":"2024-04-26T20:58:09.811773+0000","last_active":"2024-04-26T20:58:11.069367+0000","last_peered":"2024-04-26T20:58:11.069367+0000","last_clean":"2024-04-26T20:58:11.069367+0000","last_became_active":"2024-04-26T20:58:09.811496+0000","last_became_peered":"2024-04-26T20:58:09.811496+0000","last_unstale":"2024-04-26T20:58:11.069367+0000","last_undegraded":"2024-04-26T20:58:11.069367+0000","last_fullsized":"2024-04-26T20:58:11.069367+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:03.261062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":20,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.765529+0000","last_change":"2024-04-26T20:58:08.769765+0000","last_active":"2024-04-26T20:58:10.765529+0000","last_peered":"2024-04-26T20:58:10.765529+0000","last_clean":"2024-04-26T20:58:10.765529+0000","last_became_active":"2024-04-26T20:58:08.764227+0000","last_became_peered":"2024-04-26T20:58:08.764227+0000","last_unstale":"2024-04-26T20:58:10.765529+0000","last_undegraded":"2024-04-26T20:58:10.765529+0000","last_fullsized":"2024-04-26T20:58:10.765529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:57:10.744560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.822628+0000","last_change":"2024-04-26T20:58:09.800363+0000","last_active":"2024-04-26T20:58:10.822628+0000","last_peered":"2024-04-26T20:58:10.822628+0000","last_clean":"2024-04-26T20:58:10.822628+0000","last_became_active":"2024-04-26T20:58:09.800074+0000","last_became_peered":"2024-04-26T20:58:09.800074+0000","last_unstale":"2024-04-26T20:58:10.822628+0000","last_undegraded":"2024-04-26T20:58:10.822628+0000","last_fullsized":"2024-04-26T20:58:10.822628+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:04:02.282206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.067916+0000","last_change":"2024-04-26T20:58:08.771486+0000","last_active":"2024-04-26T20:58:11.067916+0000","last_peered":"2024-04-26T20:58:11.067916+0000","last_clean":"2024-04-26T20:58:11.067916+0000","last_became_active":"2024-04-26T20:58:08.771023+0000","last_became_peered":"2024-04-26T20:58:08.771023+0000","last_unstale":"2024-04-26T20:58:11.067916+0000","last_undegraded":"2024-04-26T20:58:11.067916+0000","last_fullsized":"2024-04-26T20:58:11.067916+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:14.121790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.765385+0000","last_change":"2024-04-26T20:58:09.797625+0000","last_active":"2024-04-26T20:58:10.765385+0000","last_peered":"2024-04-26T20:58:10.765385+0000","last_clean":"2024-04-26T20:58:10.765385+0000","last_became_active":"2024-04-26T20:58:09.797338+0000","last_became_peered":"2024-04-26T20:58:09.797338+0000","last_unstale":"2024-04-26T20:58:10.765385+0000","last_undegraded":"2024-04-26T20:58:10.765385+0000","last_fullsized":"2024-04-26T20:58:10.765385+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:02:50.382356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.822499+0000","last_change":"2024-04-26T20:58:08.768527+0000","last_active":"2024-04-26T20:58:10.822499+0000","last_peered":"2024-04-26T20:58:10.822499+0000","last_clean":"2024-04-26T20:58:10.822499+0000","last_became_active":"2024-04-26T20:58:08.768198+0000","last_became_peered":"2024-04-26T20:58:08.768198+0000","last_unstale":"2024-04-26T20:58:10.822499+0000","last_undegraded":"2024-04-26T20:58:10.822499+0000","last_fullsized":"2024-04-26T20:58:10.822499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:35:50.189548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.771758+0000","last_change":"2024-04-26T20:58:09.810042+0000","last_active":"2024-04-26T20:58:10.771758+0000","last_peered":"2024-04-26T20:58:10.771758+0000","last_clean":"2024-04-26T20:58:10.771758+0000","last_became_active":"2024-04-26T20:58:09.809750+0000","last_became_peered":"2024-04-26T20:58:09.809750+0000","last_unstale":"2024-04-26T20:58:10.771758+0000","last_undegraded":"2024-04-26T20:58:10.771758+0000","last_fullsized":"2024-04-26T20:58:10.771758+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:03.261062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.764586+0000","last_change":"2024-04-26T20:58:08.773752+0000","last_active":"2024-04-26T20:58:10.764586+0000","last_peered":"2024-04-26T20:58:10.764586+0000","last_clean":"2024-04-26T20:58:10.764586+0000","last_became_active":"2024-04-26T20:58:08.773468+0000","last_became_peered":"2024-04-26T20:58:08.773468+0000","last_unstale":"2024-04-26T20:58:10.764586+0000","last_undegraded":"2024-04-26T20:58:10.764586+0000","last_fullsized":"2024-04-26T20:58:10.764586+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:22.999743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.769419+0000","last_change":"2024-04-26T20:58:09.799654+0000","last_active":"2024-04-26T20:58:10.769419+0000","last_peered":"2024-04-26T20:58:10.769419+0000","last_clean":"2024-04-26T20:58:10.769419+0000","last_became_active":"2024-04-26T20:58:09.799357+0000","last_became_peered":"2024-04-26T20:58:09.799357+0000","last_unstale":"2024-04-26T20:58:10.769419+0000","last_undegraded":"2024-04-26T20:58:10.769419+0000","last_fullsized":"2024-04-26T20:58:10.769419+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:39.977180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":22,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.764637+0000","last_change":"2024-04-26T20:58:08.757290+0000","last_active":"2024-04-26T20:58:10.764637+0000","last_peered":"2024-04-26T20:58:10.764637+0000","last_clean":"2024-04-26T20:58:10.764637+0000","last_became_active":"2024-04-26T20:58:08.756980+0000","last_became_peered":"2024-04-26T20:58:08.756980+0000","last_unstale":"2024-04-26T20:58:10.764637+0000","last_undegraded":"2024-04-26T20:58:10.764637+0000","last_fullsized":"2024-04-26T20:58:10.764637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:38.964650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.068922+0000","last_change":"2024-04-26T20:58:09.800445+0000","last_active":"2024-04-26T20:58:11.068922+0000","last_peered":"2024-04-26T20:58:11.068922+0000","last_clean":"2024-04-26T20:58:11.068922+0000","last_became_active":"2024-04-26T20:58:09.800159+0000","last_became_peered":"2024-04-26T20:58:09.800159+0000","last_unstale":"2024-04-26T20:58:11.068922+0000","last_undegraded":"2024-04-26T20:58:11.068922+0000","last_fullsized":"2024-04-26T20:58:11.068922+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:45:20.575336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.824013+0000","last_change":"2024-04-26T20:58:08.771724+0000","last_active":"2024-04-26T20:58:10.824013+0000","last_peered":"2024-04-26T20:58:10.824013+0000","last_clean":"2024-04-26T20:58:10.824013+0000","last_became_active":"2024-04-26T20:58:08.771333+0000","last_became_peered":"2024-04-26T20:58:08.771333+0000","last_unstale":"2024-04-26T20:58:10.824013+0000","last_undegraded":"2024-04-26T20:58:10.824013+0000","last_fullsized":"2024-04-26T20:58:10.824013+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:21.644823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.069712+0000","last_change":"2024-04-26T20:58:09.803136+0000","last_active":"2024-04-26T20:58:11.069712+0000","last_peered":"2024-04-26T20:58:11.069712+0000","last_clean":"2024-04-26T20:58:11.069712+0000","last_became_active":"2024-04-26T20:58:09.802852+0000","last_became_peered":"2024-04-26T20:58:09.802852+0000","last_unstale":"2024-04-26T20:58:11.069712+0000","last_undegraded":"2024-04-26T20:58:11.069712+0000","last_fullsized":"2024-04-26T20:58:11.069712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:55.381181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.069171+0000","last_change":"2024-04-26T20:58:08.770184+0000","last_active":"2024-04-26T20:58:11.069171+0000","last_peered":"2024-04-26T20:58:11.069171+0000","last_clean":"2024-04-26T20:58:11.069171+0000","last_became_active":"2024-04-26T20:58:08.769472+0000","last_became_peered":"2024-04-26T20:58:08.769472+0000","last_unstale":"2024-04-26T20:58:11.069171+0000","last_undegraded":"2024-04-26T20:58:11.069171+0000","last_fullsized":"2024-04-26T20:58:11.069171+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:11.613084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.823194+0000","last_change":"2024-04-26T20:58:09.808942+0000","last_active":"2024-04-26T20:58:10.823194+0000","last_peered":"2024-04-26T20:58:10.823194+0000","last_clean":"2024-04-26T20:58:10.823194+0000","last_became_active":"2024-04-26T20:58:09.808239+0000","last_became_peered":"2024-04-26T20:58:09.808239+0000","last_unstale":"2024-04-26T20:58:10.823194+0000","last_undegraded":"2024-04-26T20:58:10.823194+0000","last_fullsized":"2024-04-26T20:58:10.823194+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:01.425446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.068456+0000","last_change":"2024-04-26T20:58:08.765303+0000","last_active":"2024-04-26T20:58:11.068456+0000","last_peered":"2024-04-26T20:58:11.068456+0000","last_clean":"2024-04-26T20:58:11.068456+0000","last_became_active":"2024-04-26T20:58:08.764957+0000","last_became_peered":"2024-04-26T20:58:08.764957+0000","last_unstale":"2024-04-26T20:58:11.068456+0000","last_undegraded":"2024-04-26T20:58:11.068456+0000","last_fullsized":"2024-04-26T20:58:11.068456+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:03.145144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.065097+0000","last_change":"2024-04-26T20:58:09.794652+0000","last_active":"2024-04-26T20:58:11.065097+0000","last_peered":"2024-04-26T20:58:11.065097+0000","last_clean":"2024-04-26T20:58:11.065097+0000","last_became_active":"2024-04-26T20:58:09.794282+0000","last_became_peered":"2024-04-26T20:58:09.794282+0000","last_unstale":"2024-04-26T20:58:11.065097+0000","last_undegraded":"2024-04-26T20:58:11.065097+0000","last_fullsized":"2024-04-26T20:58:11.065097+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:15.118668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.067767+0000","last_change":"2024-04-26T20:58:08.769798+0000","last_active":"2024-04-26T20:58:11.067767+0000","last_peered":"2024-04-26T20:58:11.067767+0000","last_clean":"2024-04-26T20:58:11.067767+0000","last_became_active":"2024-04-26T20:58:08.769441+0000","last_became_peered":"2024-04-26T20:58:08.769441+0000","last_unstale":"2024-04-26T20:58:11.067767+0000","last_undegraded":"2024-04-26T20:58:11.067767+0000","last_fullsized":"2024-04-26T20:58:11.067767+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:03:43.622795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.064404+0000","last_change":"2024-04-26T20:58:09.792654+0000","last_active":"2024-04-26T20:58:11.064404+0000","last_peered":"2024-04-26T20:58:11.064404+0000","last_clean":"2024-04-26T20:58:11.064404+0000","last_became_active":"2024-04-26T20:58:09.792160+0000","last_became_peered":"2024-04-26T20:58:09.792160+0000","last_unstale":"2024-04-26T20:58:11.064404+0000","last_undegraded":"2024-04-26T20:58:11.064404+0000","last_fullsized":"2024-04-26T20:58:11.064404+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:02:04.157674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.816780+0000","last_change":"2024-04-26T20:58:08.770250+0000","last_active":"2024-04-26T20:58:10.816780+0000","last_peered":"2024-04-26T20:58:10.816780+0000","last_clean":"2024-04-26T20:58:10.816780+0000","last_became_active":"2024-04-26T20:58:08.769501+0000","last_became_peered":"2024-04-26T20:58:08.769501+0000","last_unstale":"2024-04-26T20:58:10.816780+0000","last_undegraded":"2024-04-26T20:58:10.816780+0000","last_fullsized":"2024-04-26T20:58:10.816780+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:01.243677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.065264+0000","last_change":"2024-04-26T20:58:09.793619+0000","last_active":"2024-04-26T20:58:11.065264+0000","last_peered":"2024-04-26T20:58:11.065264+0000","last_clean":"2024-04-26T20:58:11.065264+0000","last_became_active":"2024-04-26T20:58:09.793280+0000","last_became_peered":"2024-04-26T20:58:09.793280+0000","last_unstale":"2024-04-26T20:58:11.065264+0000","last_undegraded":"2024-04-26T20:58:11.065264+0000","last_fullsized":"2024-04-26T20:58:11.065264+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:21:00.848298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.769529+0000","last_change":"2024-04-26T20:58:08.777837+0000","last_active":"2024-04-26T20:58:10.769529+0000","last_peered":"2024-04-26T20:58:10.769529+0000","last_clean":"2024-04-26T20:58:10.769529+0000","last_became_active":"2024-04-26T20:58:08.777554+0000","last_became_peered":"2024-04-26T20:58:08.777554+0000","last_unstale":"2024-04-26T20:58:10.769529+0000","last_undegraded":"2024-04-26T20:58:10.769529+0000","last_fullsized":"2024-04-26T20:58:10.769529+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:01.243677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.064581+0000","last_change":"2024-04-26T20:58:09.795500+0000","last_active":"2024-04-26T20:58:11.064581+0000","last_peered":"2024-04-26T20:58:11.064581+0000","last_clean":"2024-04-26T20:58:11.064581+0000","last_became_active":"2024-04-26T20:58:09.792718+0000","last_became_peered":"2024-04-26T20:58:09.792718+0000","last_unstale":"2024-04-26T20:58:11.064581+0000","last_undegraded":"2024-04-26T20:58:11.064581+0000","last_fullsized":"2024-04-26T20:58:11.064581+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:55:00.728920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.066750+0000","last_change":"2024-04-26T20:58:08.767164+0000","last_active":"2024-04-26T20:58:11.066750+0000","last_peered":"2024-04-26T20:58:11.066750+0000","last_clean":"2024-04-26T20:58:11.066750+0000","last_became_active":"2024-04-26T20:58:08.766710+0000","last_became_peered":"2024-04-26T20:58:08.766710+0000","last_unstale":"2024-04-26T20:58:11.066750+0000","last_undegraded":"2024-04-26T20:58:11.066750+0000","last_fullsized":"2024-04-26T20:58:11.066750+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:45:25.351122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.771124+0000","last_change":"2024-04-26T20:58:09.813422+0000","last_active":"2024-04-26T20:58:10.771124+0000","last_peered":"2024-04-26T20:58:10.771124+0000","last_clean":"2024-04-26T20:58:10.771124+0000","last_became_active":"2024-04-26T20:58:09.813141+0000","last_became_peered":"2024-04-26T20:58:09.813141+0000","last_unstale":"2024-04-26T20:58:10.771124+0000","last_undegraded":"2024-04-26T20:58:10.771124+0000","last_fullsized":"2024-04-26T20:58:10.771124+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:41:19.722253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.067189+0000","last_change":"2024-04-26T20:58:08.773457+0000","last_active":"2024-04-26T20:58:11.067189+0000","last_peered":"2024-04-26T20:58:11.067189+0000","last_clean":"2024-04-26T20:58:11.067189+0000","last_became_active":"2024-04-26T20:58:08.773181+0000","last_became_peered":"2024-04-26T20:58:08.773181+0000","last_unstale":"2024-04-26T20:58:11.067189+0000","last_undegraded":"2024-04-26T20:58:11.067189+0000","last_fullsized":"2024-04-26T20:58:11.067189+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:49:23.334735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.064594+0000","last_change":"2024-04-26T20:58:09.806766+0000","last_active":"2024-04-26T20:58:11.064594+0000","last_peered":"2024-04-26T20:58:11.064594+0000","last_clean":"2024-04-26T20:58:11.064594+0000","last_became_active":"2024-04-26T20:58:09.805805+0000","last_became_peered":"2024-04-26T20:58:09.805805+0000","last_unstale":"2024-04-26T20:58:11.064594+0000","last_undegraded":"2024-04-26T20:58:11.064594+0000","last_fullsized":"2024-04-26T20:58:11.064594+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:15:12.625615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.824128+0000","last_change":"2024-04-26T20:58:08.774017+0000","last_active":"2024-04-26T20:58:10.824128+0000","last_peered":"2024-04-26T20:58:10.824128+0000","last_clean":"2024-04-26T20:58:10.824128+0000","last_became_active":"2024-04-26T20:58:08.773655+0000","last_became_peered":"2024-04-26T20:58:08.773655+0000","last_unstale":"2024-04-26T20:58:10.824128+0000","last_undegraded":"2024-04-26T20:58:10.824128+0000","last_fullsized":"2024-04-26T20:58:10.824128+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:22.999743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.069633+0000","last_change":"2024-04-26T20:58:09.800918+0000","last_active":"2024-04-26T20:58:11.069633+0000","last_peered":"2024-04-26T20:58:11.069633+0000","last_clean":"2024-04-26T20:58:11.069633+0000","last_became_active":"2024-04-26T20:58:09.800403+0000","last_became_peered":"2024-04-26T20:58:09.800403+0000","last_unstale":"2024-04-26T20:58:11.069633+0000","last_undegraded":"2024-04-26T20:58:11.069633+0000","last_fullsized":"2024-04-26T20:58:11.069633+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:05.917326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.817562+0000","last_change":"2024-04-26T20:58:08.775770+0000","last_active":"2024-04-26T20:58:10.817562+0000","last_peered":"2024-04-26T20:58:10.817562+0000","last_clean":"2024-04-26T20:58:10.817562+0000","last_became_active":"2024-04-26T20:58:08.775469+0000","last_became_peered":"2024-04-26T20:58:08.775469+0000","last_unstale":"2024-04-26T20:58:10.817562+0000","last_undegraded":"2024-04-26T20:58:10.817562+0000","last_fullsized":"2024-04-26T20:58:10.817562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:21.644823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.816647+0000","last_change":"2024-04-26T20:58:09.798021+0000","last_active":"2024-04-26T20:58:10.816647+0000","last_peered":"2024-04-26T20:58:10.816647+0000","last_clean":"2024-04-26T20:58:10.816647+0000","last_became_active":"2024-04-26T20:58:09.795951+0000","last_became_peered":"2024-04-26T20:58:09.795951+0000","last_unstale":"2024-04-26T20:58:10.816647+0000","last_undegraded":"2024-04-26T20:58:10.816647+0000","last_fullsized":"2024-04-26T20:58:10.816647+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:39.977180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.823301+0000","last_change":"2024-04-26T20:58:08.763704+0000","last_active":"2024-04-26T20:58:10.823301+0000","last_peered":"2024-04-26T20:58:10.823301+0000","last_clean":"2024-04-26T20:58:10.823301+0000","last_became_active":"2024-04-26T20:58:08.763378+0000","last_became_peered":"2024-04-26T20:58:08.763378+0000","last_unstale":"2024-04-26T20:58:10.823301+0000","last_undegraded":"2024-04-26T20:58:10.823301+0000","last_fullsized":"2024-04-26T20:58:10.823301+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:49:23.334735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.769079+0000","last_change":"2024-04-26T20:58:09.787792+0000","last_active":"2024-04-26T20:58:10.769079+0000","last_peered":"2024-04-26T20:58:10.769079+0000","last_clean":"2024-04-26T20:58:10.769079+0000","last_became_active":"2024-04-26T20:58:09.785714+0000","last_became_peered":"2024-04-26T20:58:09.785714+0000","last_unstale":"2024-04-26T20:58:10.769079+0000","last_undegraded":"2024-04-26T20:58:10.769079+0000","last_fullsized":"2024-04-26T20:58:10.769079+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:53.718524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.816963+0000","last_change":"2024-04-26T20:58:08.764863+0000","last_active":"2024-04-26T20:58:10.816963+0000","last_peered":"2024-04-26T20:58:10.816963+0000","last_clean":"2024-04-26T20:58:10.816963+0000","last_became_active":"2024-04-26T20:58:08.764569+0000","last_became_peered":"2024-04-26T20:58:08.764569+0000","last_unstale":"2024-04-26T20:58:10.816963+0000","last_undegraded":"2024-04-26T20:58:10.816963+0000","last_fullsized":"2024-04-26T20:58:10.816963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:32.017697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.816722+0000","last_change":"2024-04-26T20:58:09.788712+0000","last_active":"2024-04-26T20:58:10.816722+0000","last_peered":"2024-04-26T20:58:10.816722+0000","last_clean":"2024-04-26T20:58:10.816722+0000","last_became_active":"2024-04-26T20:58:09.788110+0000","last_became_peered":"2024-04-26T20:58:09.788110+0000","last_unstale":"2024-04-26T20:58:10.816722+0000","last_undegraded":"2024-04-26T20:58:10.816722+0000","last_fullsized":"2024-04-26T20:58:10.816722+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:17:07.322131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.822770+0000","last_change":"2024-04-26T20:58:08.770571+0000","last_active":"2024-04-26T20:58:10.822770+0000","last_peered":"2024-04-26T20:58:10.822770+0000","last_clean":"2024-04-26T20:58:10.822770+0000","last_became_active":"2024-04-26T20:58:08.769976+0000","last_became_peered":"2024-04-26T20:58:08.769976+0000","last_unstale":"2024-04-26T20:58:10.822770+0000","last_undegraded":"2024-04-26T20:58:10.822770+0000","last_fullsized":"2024-04-26T20:58:10.822770+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:24:35.518250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.764565+0000","last_change":"2024-04-26T20:58:09.803675+0000","last_active":"2024-04-26T20:58:10.764565+0000","last_peered":"2024-04-26T20:58:10.764565+0000","last_clean":"2024-04-26T20:58:10.764565+0000","last_became_active":"2024-04-26T20:58:09.803292+0000","last_became_peered":"2024-04-26T20:58:09.803292+0000","last_unstale":"2024-04-26T20:58:10.764565+0000","last_undegraded":"2024-04-26T20:58:10.764565+0000","last_fullsized":"2024-04-26T20:58:10.764565+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:03.261062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.067010+0000","last_change":"2024-04-26T20:58:08.762291+0000","last_active":"2024-04-26T20:58:11.067010+0000","last_peered":"2024-04-26T20:58:11.067010+0000","last_clean":"2024-04-26T20:58:11.067010+0000","last_became_active":"2024-04-26T20:58:08.761994+0000","last_became_peered":"2024-04-26T20:58:08.761994+0000","last_unstale":"2024-04-26T20:58:11.067010+0000","last_undegraded":"2024-04-26T20:58:11.067010+0000","last_fullsized":"2024-04-26T20:58:11.067010+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:14.106138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:11.066699+0000","last_change":"2024-04-26T20:58:09.791918+0000","last_active":"2024-04-26T20:58:11.066699+0000","last_peered":"2024-04-26T20:58:11.066699+0000","last_clean":"2024-04-26T20:58:11.066699+0000","last_became_active":"2024-04-26T20:58:09.790161+0000","last_became_peered":"2024-04-26T20:58:09.790161+0000","last_unstale":"2024-04-26T20:58:11.066699+0000","last_undegraded":"2024-04-26T20:58:11.066699+0000","last_fullsized":"2024-04-26T20:58:11.066699+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:49:24.347265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.764537+0000","last_change":"2024-04-26T20:58:08.771736+0000","last_active":"2024-04-26T20:58:10.764537+0000","last_peered":"2024-04-26T20:58:10.764537+0000","last_clean":"2024-04-26T20:58:10.764537+0000","last_became_active":"2024-04-26T20:58:08.771278+0000","last_became_peered":"2024-04-26T20:58:08.771278+0000","last_unstale":"2024-04-26T20:58:10.764537+0000","last_undegraded":"2024-04-26T20:58:10.764537+0000","last_fullsized":"2024-04-26T20:58:10.764537+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:40:52.705994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.771442+0000","last_change":"2024-04-26T20:58:09.809802+0000","last_active":"2024-04-26T20:58:10.771442+0000","last_peered":"2024-04-26T20:58:10.771442+0000","last_clean":"2024-04-26T20:58:10.771442+0000","last_became_active":"2024-04-26T20:58:09.809479+0000","last_became_peered":"2024-04-26T20:58:09.809479+0000","last_unstale":"2024-04-26T20:58:10.771442+0000","last_undegraded":"2024-04-26T20:58:10.771442+0000","last_fullsized":"2024-04-26T20:58:10.771442+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:08.728179+0000","last_clean_scrub_stamp":"2024-04-26T20:58:08.728179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:46:55.629400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.770033+0000","last_change":"2024-04-26T20:58:08.777072+0000","last_active":"2024-04-26T20:58:10.770033+0000","last_peered":"2024-04-26T20:58:10.770033+0000","last_clean":"2024-04-26T20:58:10.770033+0000","last_became_active":"2024-04-26T20:58:08.776746+0000","last_became_peered":"2024-04-26T20:58:08.776746+0000","last_unstale":"2024-04-26T20:58:10.770033+0000","last_undegraded":"2024-04-26T20:58:10.770033+0000","last_fullsized":"2024-04-26T20:58:10.770033+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:07.715649+0000","last_clean_scrub_stamp":"2024-04-26T20:58:07.715649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:24.623434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":195,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-04-26T20:58:10.772508+0000","last_change":"2024-04-26T20:56:46.848603+0000","last_active":"2024-04-26T20:58:10.772508+0000","last_peered":"2024-04-26T20:58:10.772508+0000","last_clean":"2024-04-26T20:58:10.772508+0000","last_became_active":"2024-04-26T20:56:46.846091+0000","last_became_peered":"2024-04-26T20:56:46.846091+0000","last_unstale":"2024-04-26T20:58:10.772508+0000","last_undegraded":"2024-04-26T20:58:10.772508+0000","last_fullsized":"2024-04-26T20:58:10.772508+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_clean_scrub_stamp":"2024-04-26T20:56:45.594041+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:00.232387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":983040,"data_stored":741745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":151903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":151959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":151391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":401408,"data_stored":152921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":389120,"data_stored":151357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31580,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340260,"statfs":{"total":96636764160,"available":96604426240,"internally_reserved":0,"allocated":401408,"data_stored":152921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":978944,"data_stored":741655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:58:14.795 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T20:58:14.795 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T20:58:15.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.050+0000 7f3f04a00640 1 Processor -- start 2024-04-26T20:58:15.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.051+0000 7f3f04a00640 1 -- start start 2024-04-26T20:58:15.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.052+0000 7f3f04a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f00148a90 0x7f3f000d1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:15.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.052+0000 7f3f04a00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3f000b9730 0x7f3f000b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:15.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.052+0000 7f3f04a00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f000d2010 con 0x7f3f000b72f0 2024-04-26T20:58:15.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.052+0000 7f3f04a00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f3f000b7760 con 0x7f3f000b9730 2024-04-26T20:58:15.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.052+0000 7f3f04a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f3f000b8b40 con 0x7f3f00148a90 2024-04-26T20:58:15.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.053+0000 7f3effe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3f000b72f0 0x7f3f00162d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50118/0 (socket says 172.21.15.28:50118) 2024-04-26T20:58:15.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.053+0000 7f3effe00640 1 -- 172.21.15.28:0/3711204028 learned_addr learned my addr 172.21.15.28:0/3711204028 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:15.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.053+0000 7f3efea00640 1 --2- 172.21.15.28:0/3711204028 >> v2:172.21.15.103:3300/0 conn(0x7f3f000b9730 0x7f3f000b6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:15.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.053+0000 7f3eff400640 1 --2- 172.21.15.28:0/3711204028 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f00148a90 0x7f3f000d1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:15.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.053+0000 7f3efe000640 1 -- 172.21.15.28:0/3711204028 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2876714980 0 0) 0x7f3f000d2010 con 0x7f3f000b72f0 2024-04-26T20:58:15.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.054+0000 7f3efe000640 1 -- 172.21.15.28:0/3711204028 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ee4003520 con 0x7f3f000b72f0 2024-04-26T20:58:15.056 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.054+0000 7f3efe000640 1 -- 172.21.15.28:0/3711204028 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2978026877 0 0) 0x7f3ee4003520 con 0x7f3f000b72f0 2024-04-26T20:58:15.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.054+0000 7f3efe000640 1 -- 172.21.15.28:0/3711204028 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f00148a90 msgr2=0x7f3f000d1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:15.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.054+0000 7f3efe000640 1 --2- 172.21.15.28:0/3711204028 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f00148a90 0x7f3f000d1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.054+0000 7f3efe000640 1 -- 172.21.15.28:0/3711204028 >> v2:172.21.15.103:3300/0 conn(0x7f3f000b9730 msgr2=0x7f3f000b6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:15.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.054+0000 7f3efe000640 1 --2- 172.21.15.28:0/3711204028 >> v2:172.21.15.103:3300/0 conn(0x7f3f000b9730 0x7f3f000b6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.054+0000 7f3efe000640 1 -- 172.21.15.28:0/3711204028 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f000d22f0 con 0x7f3f000b72f0 2024-04-26T20:58:15.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.054+0000 7f3efe000640 1 -- 172.21.15.28:0/3711204028 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f3ef8001670 con 0x7f3f000b72f0 2024-04-26T20:58:15.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.054+0000 7f3efe000640 1 -- 172.21.15.28:0/3711204028 >> v1:172.21.15.28:6789/0 conn(0x7f3f000b72f0 legacy=0x7f3f00162d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:15.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.054+0000 7f3efe000640 1 --2- 172.21.15.28:0/3711204028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ee40039e0 0x7f3ee4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:15.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.054+0000 7f3efe000640 1 --2- 172.21.15.28:0/3711204028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ee4004800 0x7f3ee4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:15.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.054+0000 7f3efe000640 1 --2- 172.21.15.28:0/3711204028 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f00148a90 0x7f3ee40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:15.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.054+0000 7f3efe000640 1 -- 172.21.15.28:0/3711204028 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3f000b7760 con 0x7f3ee40039e0 2024-04-26T20:58:15.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.054+0000 7f3efe000640 1 -- 172.21.15.28:0/3711204028 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3f000b8b40 con 0x7f3ee4004800 2024-04-26T20:58:15.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.054+0000 7f3efe000640 1 -- 172.21.15.28:0/3711204028 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f3ee400b9c0 con 0x7f3f00148a90 2024-04-26T20:58:15.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.054+0000 7f3eff400640 1 --2- 172.21.15.28:0/3711204028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ee40039e0 0x7f3ee4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:15.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.055+0000 7f3eff400640 1 -- 172.21.15.28:0/3711204028 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f00148a90 msgr2=0x7f3ee40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:15.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.055+0000 7f3eff400640 1 --2- 172.21.15.28:0/3711204028 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f00148a90 0x7f3ee40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.055+0000 7f3eff400640 1 -- 172.21.15.28:0/3711204028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ee4004800 msgr2=0x7f3ee4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:15.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.055+0000 7f3efea00640 1 --2- 172.21.15.28:0/3711204028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ee4004800 0x7f3ee4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:15.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.055+0000 7f3eff400640 1 --2- 172.21.15.28:0/3711204028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ee4004800 0x7f3ee4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.055+0000 7f3eff400640 1 -- 172.21.15.28:0/3711204028 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ee4003520 con 0x7f3ee40039e0 2024-04-26T20:58:15.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.055+0000 7f3efea00640 1 --2- 172.21.15.28:0/3711204028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ee4004800 0x7f3ee4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:15.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.056+0000 7f3eff400640 1 --2- 172.21.15.28:0/3711204028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ee40039e0 0x7f3ee4003e00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009fc0 tx=0x7f3ee4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646846c0625a62bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:15.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.056+0000 7f3efe000640 1 -- 172.21.15.28:0/3711204028 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0009c10 con 0x7f3ee40039e0 2024-04-26T20:58:15.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.056+0000 7f3efe000640 1 -- 172.21.15.28:0/3711204028 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef0036070 con 0x7f3ee40039e0 2024-04-26T20:58:15.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.056+0000 7f3efe000640 1 -- 172.21.15.28:0/3711204028 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef002d070 con 0x7f3ee40039e0 2024-04-26T20:58:15.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.057+0000 7f3f04a00640 1 -- 172.21.15.28:0/3711204028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ee40039e0 msgr2=0x7f3ee4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:15.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.057+0000 7f3f04a00640 1 --2- 172.21.15.28:0/3711204028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ee40039e0 0x7f3ee4003e00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009fc0 tx=0x7f3ee4005830 comp rx=0 tx=0).stop 2024-04-26T20:58:15.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.057+0000 7f3effe00640 1 -- 172.21.15.28:0/3711204028 reap_dead start 2024-04-26T20:58:15.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.057+0000 7f3f04a00640 1 -- 172.21.15.28:0/3711204028 shutdown_connections 2024-04-26T20:58:15.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.057+0000 7f3f04a00640 1 -- 172.21.15.28:0/3711204028 >> 172.21.15.28:0/3711204028 conn(0x7f3f0006c530 msgr2=0x7f3f0006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:15.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.057+0000 7f3f04a00640 1 -- 172.21.15.28:0/3711204028 shutdown_connections 2024-04-26T20:58:15.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.057+0000 7f3f04a00640 1 -- 172.21.15.28:0/3711204028 wait complete. 2024-04-26T20:58:15.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.059+0000 7f3f04a00640 1 Processor -- start 2024-04-26T20:58:15.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.059+0000 7f3f04a00640 1 -- start start 2024-04-26T20:58:15.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.060+0000 7f3f04a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ee40039e0 0x7f3f0013fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:15.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.060+0000 7f3f04a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f00160580 0x7f3f000b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:15.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.060+0000 7f3f04a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f000b5cd0 0x7f3f00150640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:15.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.060+0000 7f3f04a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3f000d2b90 con 0x7f3ee40039e0 2024-04-26T20:58:15.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.060+0000 7f3f04a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3f000d28b0 con 0x7f3f00160580 2024-04-26T20:58:15.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.060+0000 7f3f04a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f3f00148e70 con 0x7f3f000b5cd0 2024-04-26T20:58:15.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.060+0000 7f3eff400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ee40039e0 0x7f3f0013fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:15.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.060+0000 7f3eff400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ee40039e0 0x7f3f0013fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52038/0 (socket says 172.21.15.28:52038) 2024-04-26T20:58:15.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.060+0000 7f3eff400640 1 -- 172.21.15.28:0/3715083584 learned_addr learned my addr 172.21.15.28:0/3715083584 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:15.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.061+0000 7f3eff400640 1 -- 172.21.15.28:0/3715083584 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f000b5cd0 msgr2=0x7f3f00150640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:15.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.061+0000 7f3eff400640 1 --2- 172.21.15.28:0/3715083584 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f000b5cd0 0x7f3f00150640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.061+0000 7f3eff400640 1 -- 172.21.15.28:0/3715083584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f00160580 msgr2=0x7f3f000b5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:15.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.061+0000 7f3efea00640 1 --2- 172.21.15.28:0/3715083584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f00160580 0x7f3f000b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:15.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.061+0000 7f3eff400640 1 --2- 172.21.15.28:0/3715083584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f00160580 0x7f3f000b5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.061+0000 7f3eff400640 1 -- 172.21.15.28:0/3715083584 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f001513b0 con 0x7f3ee40039e0 2024-04-26T20:58:15.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.061+0000 7f3efea00640 1 --2- 172.21.15.28:0/3715083584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f00160580 0x7f3f000b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:15.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.061+0000 7f3eff400640 1 --2- 172.21.15.28:0/3715083584 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ee40039e0 0x7f3f0013fa30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3ef002af10 tx=0x7f3ef0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:15.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.061+0000 7f3ef7400640 1 -- 172.21.15.28:0/3715083584 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0009c10 con 0x7f3ee40039e0 2024-04-26T20:58:15.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.061+0000 7f3ef7400640 1 -- 172.21.15.28:0/3715083584 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef0003bb0 con 0x7f3ee40039e0 2024-04-26T20:58:15.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.061+0000 7f3ef7400640 1 -- 172.21.15.28:0/3715083584 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0004020 con 0x7f3ee40039e0 2024-04-26T20:58:15.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.061+0000 7f3f04a00640 1 -- 172.21.15.28:0/3715083584 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f0015faa0 con 0x7f3ee40039e0 2024-04-26T20:58:15.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.061+0000 7f3f04a00640 1 -- 172.21.15.28:0/3715083584 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f00160060 con 0x7f3ee40039e0 2024-04-26T20:58:15.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.063+0000 7f3f04a00640 1 -- 172.21.15.28:0/3715083584 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f000b6120 con 0x7f3ee40039e0 2024-04-26T20:58:15.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.063+0000 7f3ef7400640 1 -- 172.21.15.28:0/3715083584 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ef0005d10 con 0x7f3ee40039e0 2024-04-26T20:58:15.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.064+0000 7f3ef7400640 1 --2- 172.21.15.28:0/3715083584 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3edc06c8f0 0x7f3edc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:15.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.064+0000 7f3ef7400640 1 -- 172.21.15.28:0/3715083584 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f3ef00ad0a0 con 0x7f3ee40039e0 2024-04-26T20:58:15.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.065+0000 7f3efea00640 1 --2- 172.21.15.28:0/3715083584 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3edc06c8f0 0x7f3edc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:15.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.065+0000 7f3efea00640 1 --2- 172.21.15.28:0/3715083584 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3edc06c8f0 0x7f3edc06edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3f00160980 tx=0x7f3ee8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:15.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.069+0000 7f3ef7400640 1 -- 172.21.15.28:0/3715083584 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3ef007b4c0 con 0x7f3ee40039e0 2024-04-26T20:58:15.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.355+0000 7f3f04a00640 1 -- 172.21.15.28:0/3715083584 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f3f0006a2c0 con 0x7f3ee40039e0 2024-04-26T20:58:15.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.360+0000 7f3ef7400640 1 -- 172.21.15.28:0/3715083584 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7f3ef007adc0 con 0x7f3ee40039e0 2024-04-26T20:58:15.361 INFO:teuthology.orchestra.run.smithi028.stdout:[client.0] 2024-04-26T20:58:15.361 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDVFCxm/rQRLhAAGsnaSU0YseRGCeDvxvvp/A== 2024-04-26T20:58:15.361 INFO:teuthology.orchestra.run.smithi028.stdout: caps mds = "allow" 2024-04-26T20:58:15.361 INFO:teuthology.orchestra.run.smithi028.stdout: caps mon = "allow r" 2024-04-26T20:58:15.361 INFO:teuthology.orchestra.run.smithi028.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T20:58:15.361 INFO:teuthology.orchestra.run.smithi028.stderr:updated caps for client.0 2024-04-26T20:58:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.363+0000 7f3f04a00640 1 -- 172.21.15.28:0/3715083584 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3edc06c8f0 msgr2=0x7f3edc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.363+0000 7f3f04a00640 1 --2- 172.21.15.28:0/3715083584 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3edc06c8f0 0x7f3edc06edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3f00160980 tx=0x7f3ee8008040 comp rx=0 tx=0).stop 2024-04-26T20:58:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.363+0000 7f3f04a00640 1 -- 172.21.15.28:0/3715083584 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ee40039e0 msgr2=0x7f3f0013fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.363+0000 7f3f04a00640 1 --2- 172.21.15.28:0/3715083584 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ee40039e0 0x7f3f0013fa30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3ef002af10 tx=0x7f3ef0009750 comp rx=0 tx=0).stop 2024-04-26T20:58:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.363+0000 7f3f04a00640 1 -- 172.21.15.28:0/3715083584 shutdown_connections 2024-04-26T20:58:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.363+0000 7f3f04a00640 1 --2- 172.21.15.28:0/3715083584 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3edc06c8f0 0x7f3edc06edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.363+0000 7f3f04a00640 1 --2- 172.21.15.28:0/3715083584 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3f000b5cd0 0x7f3f00150640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.363+0000 7f3f04a00640 1 --2- 172.21.15.28:0/3715083584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f00160580 0x7f3f000b5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.363+0000 7f3f04a00640 1 --2- 172.21.15.28:0/3715083584 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ee40039e0 0x7f3f0013fa30 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.363+0000 7f3f04a00640 1 -- 172.21.15.28:0/3715083584 >> 172.21.15.28:0/3715083584 conn(0x7f3f0006c530 msgr2=0x7f3f00147d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.364+0000 7f3f04a00640 1 -- 172.21.15.28:0/3715083584 shutdown_connections 2024-04-26T20:58:15.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.364+0000 7f3f04a00640 1 -- 172.21.15.28:0/3715083584 wait complete. 2024-04-26T20:58:15.386 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:58:15.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.598+0000 7f6648e00640 1 Processor -- start 2024-04-26T20:58:15.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.598+0000 7f6648e00640 1 -- start start 2024-04-26T20:58:15.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.599+0000 7f6648e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f66440b43c0 0x7f66440d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:15.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.599+0000 7f6648e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66440b47a0 con 0x7f66440b6b10 2024-04-26T20:58:15.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.599+0000 7f6648e00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6644160c90 con 0x7f66441607e0 2024-04-26T20:58:15.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.599+0000 7f6648e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f66440d0740 con 0x7f66440b43c0 2024-04-26T20:58:15.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.600+0000 7f6642800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f66440b6b10 0x7f66440d3880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50120/0 (socket says 172.21.15.28:50120) 2024-04-26T20:58:15.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.600+0000 7f6642800640 1 -- 172.21.15.28:0/1188681023 learned_addr learned my addr 172.21.15.28:0/1188681023 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:15.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.600+0000 7f6641e00640 1 --2- 172.21.15.28:0/1188681023 >> v2:172.21.15.103:3300/0 conn(0x7f66440b43c0 0x7f66440d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:15.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.600+0000 7f6641400640 1 -- 172.21.15.28:0/1188681023 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2943905538 0 0) 0x7f66440b47a0 con 0x7f66440b6b10 2024-04-26T20:58:15.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.600+0000 7f6641400640 1 -- 172.21.15.28:0/1188681023 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6628003520 con 0x7f66440b6b10 2024-04-26T20:58:15.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.601+0000 7f6641400640 1 -- 172.21.15.28:0/1188681023 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1505653980 0 0) 0x7f6628003520 con 0x7f66440b6b10 2024-04-26T20:58:15.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.601+0000 7f6641400640 1 -- 172.21.15.28:0/1188681023 >> v2:172.21.15.103:3300/0 conn(0x7f66440b43c0 msgr2=0x7f66440d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:15.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.601+0000 7f6641400640 1 --2- 172.21.15.28:0/1188681023 >> v2:172.21.15.103:3300/0 conn(0x7f66440b43c0 0x7f66440d00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.601+0000 7f6641400640 1 -- 172.21.15.28:0/1188681023 >> v1:172.21.15.103:6789/0 conn(0x7f66441607e0 legacy=0x7f6644163cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:15.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.601+0000 7f6641400640 1 -- 172.21.15.28:0/1188681023 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66440d0a20 con 0x7f66440b6b10 2024-04-26T20:58:15.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.601+0000 7f6641400640 1 -- 172.21.15.28:0/1188681023 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f663c0019f0 con 0x7f66440b6b10 2024-04-26T20:58:15.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.601+0000 7f6641400640 1 -- 172.21.15.28:0/1188681023 >> v1:172.21.15.28:6789/0 conn(0x7f66440b6b10 legacy=0x7f66440d3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:15.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.601+0000 7f6641400640 1 --2- 172.21.15.28:0/1188681023 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66280039b0 0x7f6628003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:15.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.601+0000 7f6641400640 1 --2- 172.21.15.28:0/1188681023 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66280043b0 0x7f6628004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:15.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.601+0000 7f6641400640 1 --2- 172.21.15.28:0/1188681023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6628005120 0x7f662800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:15.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.601+0000 7f6641400640 1 -- 172.21.15.28:0/1188681023 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f66440d0740 con 0x7f66280043b0 2024-04-26T20:58:15.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.601+0000 7f6641400640 1 -- 172.21.15.28:0/1188681023 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f662800c060 con 0x7f6628005120 2024-04-26T20:58:15.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.601+0000 7f6641400640 1 -- 172.21.15.28:0/1188681023 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f662800c260 con 0x7f66280039b0 2024-04-26T20:58:15.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.601+0000 7f663be00640 1 --2- 172.21.15.28:0/1188681023 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66280043b0 0x7f6628004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:15.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.602+0000 7f6642800640 1 --2- 172.21.15.28:0/1188681023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6628005120 0x7f662800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:15.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.602+0000 7f6641e00640 1 --2- 172.21.15.28:0/1188681023 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66280039b0 0x7f6628003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:15.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.602+0000 7f663be00640 1 -- 172.21.15.28:0/1188681023 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66280039b0 msgr2=0x7f6628003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:15.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.602+0000 7f663be00640 1 --2- 172.21.15.28:0/1188681023 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66280039b0 0x7f6628003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.602+0000 7f663be00640 1 -- 172.21.15.28:0/1188681023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6628005120 msgr2=0x7f662800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:15.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.602+0000 7f663be00640 1 --2- 172.21.15.28:0/1188681023 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6628005120 0x7f662800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.602+0000 7f663be00640 1 -- 172.21.15.28:0/1188681023 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6628003520 con 0x7f66280043b0 2024-04-26T20:58:15.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.602+0000 7f6642800640 1 -- 172.21.15.28:0/1188681023 reap_dead start 2024-04-26T20:58:15.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.603+0000 7f663be00640 1 --2- 172.21.15.28:0/1188681023 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66280043b0 0x7f6628004880 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f662c001860 tx=0x7f6628005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3a423e4ebe9627 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:15.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.603+0000 7f6641400640 1 -- 172.21.15.28:0/1188681023 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c037070 con 0x7f66280043b0 2024-04-26T20:58:15.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.603+0000 7f6641400640 1 -- 172.21.15.28:0/1188681023 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f662c02abe0 con 0x7f66280043b0 2024-04-26T20:58:15.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.603+0000 7f6641400640 1 -- 172.21.15.28:0/1188681023 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c001430 con 0x7f66280043b0 2024-04-26T20:58:15.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.604+0000 7f6648e00640 1 -- 172.21.15.28:0/1188681023 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66280043b0 msgr2=0x7f6628004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:15.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.604+0000 7f6648e00640 1 --2- 172.21.15.28:0/1188681023 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66280043b0 0x7f6628004880 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f662c001860 tx=0x7f6628005830 comp rx=0 tx=0).stop 2024-04-26T20:58:15.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.604+0000 7f6648e00640 1 -- 172.21.15.28:0/1188681023 shutdown_connections 2024-04-26T20:58:15.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.604+0000 7f6648e00640 1 --2- 172.21.15.28:0/1188681023 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66280043b0 0x7f6628004880 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.604+0000 7f6648e00640 1 -- 172.21.15.28:0/1188681023 >> 172.21.15.28:0/1188681023 conn(0x7f66440d1b10 msgr2=0x7f66440d1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:15.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.604+0000 7f6648e00640 1 -- 172.21.15.28:0/1188681023 shutdown_connections 2024-04-26T20:58:15.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.604+0000 7f6648e00640 1 -- 172.21.15.28:0/1188681023 wait complete. 2024-04-26T20:58:15.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.606+0000 7f6648e00640 1 Processor -- start 2024-04-26T20:58:15.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.606+0000 7f6648e00640 1 -- start start 2024-04-26T20:58:15.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.607+0000 7f6648e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66280043b0 0x7f664414e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:15.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.607+0000 7f6648e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66440d5070 0x7f66440d54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:15.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.607+0000 7f6648e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66441f5650 0x7f66441f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:15.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.607+0000 7f6648e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f66440d00f0 con 0x7f66441f5650 2024-04-26T20:58:15.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.607+0000 7f6648e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6644163790 con 0x7f66280043b0 2024-04-26T20:58:15.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.607+0000 7f6648e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f66440d0270 con 0x7f66440d5070 2024-04-26T20:58:15.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.608+0000 7f6642800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66441f5650 0x7f66441f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:15.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.608+0000 7f6642800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66441f5650 0x7f66441f5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52052/0 (socket says 172.21.15.28:52052) 2024-04-26T20:58:15.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.608+0000 7f6642800640 1 -- 172.21.15.28:0/1798313058 learned_addr learned my addr 172.21.15.28:0/1798313058 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:15.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.608+0000 7f6641e00640 1 --2- 172.21.15.28:0/1798313058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66280043b0 0x7f664414e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:15.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.608+0000 7f663be00640 1 --2- 172.21.15.28:0/1798313058 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66440d5070 0x7f66440d54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:15.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.608+0000 7f6642800640 1 -- 172.21.15.28:0/1798313058 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66440d5070 msgr2=0x7f66440d54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:15.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.608+0000 7f6642800640 1 --2- 172.21.15.28:0/1798313058 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66440d5070 0x7f66440d54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.608+0000 7f6642800640 1 -- 172.21.15.28:0/1798313058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66280043b0 msgr2=0x7f664414e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:15.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.608+0000 7f6642800640 1 --2- 172.21.15.28:0/1798313058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66280043b0 0x7f664414e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.608+0000 7f6642800640 1 -- 172.21.15.28:0/1798313058 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664414eaf0 con 0x7f66441f5650 2024-04-26T20:58:15.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.608+0000 7f6641e00640 1 --2- 172.21.15.28:0/1798313058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66280043b0 0x7f664414e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:58:15.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.608+0000 7f6642800640 1 --2- 172.21.15.28:0/1798313058 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66441f5650 0x7f66441f5b00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6644160bc0 tx=0x7f663c007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:15.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.609+0000 7f663a000640 1 -- 172.21.15.28:0/1798313058 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c00ea80 con 0x7f66441f5650 2024-04-26T20:58:15.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.609+0000 7f663a000640 1 -- 172.21.15.28:0/1798313058 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f663c018070 con 0x7f66441f5650 2024-04-26T20:58:15.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.609+0000 7f663a000640 1 -- 172.21.15.28:0/1798313058 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c00ee60 con 0x7f66441f5650 2024-04-26T20:58:15.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.609+0000 7f6648e00640 1 -- 172.21.15.28:0/1798313058 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f664414ed60 con 0x7f66441f5650 2024-04-26T20:58:15.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.609+0000 7f6648e00640 1 -- 172.21.15.28:0/1798313058 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f664414f320 con 0x7f66441f5650 2024-04-26T20:58:15.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.610+0000 7f6648e00640 1 -- 172.21.15.28:0/1798313058 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66440d5a30 con 0x7f66441f5650 2024-04-26T20:58:15.614 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.611+0000 7f663a000640 1 -- 172.21.15.28:0/1798313058 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f663c01d460 con 0x7f66441f5650 2024-04-26T20:58:15.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.613+0000 7f663a000640 1 --2- 172.21.15.28:0/1798313058 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f662006c920 0x7f662006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:15.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.613+0000 7f663a000640 1 -- 172.21.15.28:0/1798313058 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f663c08bdf0 con 0x7f66441f5650 2024-04-26T20:58:15.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.613+0000 7f6641e00640 1 --2- 172.21.15.28:0/1798313058 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f662006c920 0x7f662006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:15.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.614+0000 7f6641e00640 1 --2- 172.21.15.28:0/1798313058 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f662006c920 0x7f662006ede0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f66440c8f30 tx=0x7f6634008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:15.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.617+0000 7f663a000640 1 -- 172.21.15.28:0/1798313058 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f663c05a180 con 0x7f66441f5650 2024-04-26T20:58:15.858 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:58:15.866 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:58:15.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.864+0000 7f6648e00640 1 -- 172.21.15.28:0/1798313058 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f664406a400 con 0x7f66441f5650 2024-04-26T20:58:15.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.865+0000 7f663a000640 1 -- 172.21.15.28:0/1798313058 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7f663c059a80 con 0x7f66441f5650 2024-04-26T20:58:15.869 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:15.869 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/1364678224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1364678224},{"type":"v1","addr":"172.21.15.103:6834","nonce":1364678224}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4081926466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":4081926466},{"type":"v1","addr":"172.21.15.28:6837","nonce":4081926466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5175,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/1370252546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":1370252546},{"type":"v1","addr":"172.21.15.28:6835","nonce":1370252546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:58:09.226353+0000","modified":"2024-04-26T20:58:12.319110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5184":{"gid":5184,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6835/1781724684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":1781724684},{"type":"v1","addr":"172.21.15.103:6835","nonce":1781724684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5184,"qdb_cluster":[5184]},"id":2}]} 2024-04-26T20:58:15.870 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-04-26T20:58:15.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.868+0000 7f6648e00640 1 -- 172.21.15.28:0/1798313058 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f662006c920 msgr2=0x7f662006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:15.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.868+0000 7f6648e00640 1 --2- 172.21.15.28:0/1798313058 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f662006c920 0x7f662006ede0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f66440c8f30 tx=0x7f6634008040 comp rx=0 tx=0).stop 2024-04-26T20:58:15.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.868+0000 7f6648e00640 1 -- 172.21.15.28:0/1798313058 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66441f5650 msgr2=0x7f66441f5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:15.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.868+0000 7f6648e00640 1 --2- 172.21.15.28:0/1798313058 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66441f5650 0x7f66441f5b00 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6644160bc0 tx=0x7f663c007bc0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.868+0000 7f6648e00640 1 -- 172.21.15.28:0/1798313058 shutdown_connections 2024-04-26T20:58:15.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.868+0000 7f6648e00640 1 --2- 172.21.15.28:0/1798313058 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f662006c920 0x7f662006ede0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.868+0000 7f6648e00640 1 --2- 172.21.15.28:0/1798313058 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66441f5650 0x7f66441f5b00 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.868+0000 7f6648e00640 1 --2- 172.21.15.28:0/1798313058 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66440d5070 0x7f66440d54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.868+0000 7f6648e00640 1 --2- 172.21.15.28:0/1798313058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66280043b0 0x7f664414e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:15.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.868+0000 7f6648e00640 1 -- 172.21.15.28:0/1798313058 >> 172.21.15.28:0/1798313058 conn(0x7f66440d1b10 msgr2=0x7f664415f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:15.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.868+0000 7f6648e00640 1 -- 172.21.15.28:0/1798313058 shutdown_connections 2024-04-26T20:58:15.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:15.868+0000 7f6648e00640 1 -- 172.21.15.28:0/1798313058 wait complete. 2024-04-26T20:58:16.292 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:58:09.226353+0000', 'modified': '2024-04-26T20:58:12.319110+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5184}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5184': {'gid': 5184, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.103:6835/1781724684', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6833', 'nonce': 1781724684}, {'type': 'v1', 'addr': '172.21.15.103:6835', 'nonce': 1781724684}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5184, 'qdb_cluster': [5184]} 2024-04-26T20:58:16.293 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T20:58:16.293 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-26T20:58:16.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.520+0000 7f0a96e00640 1 Processor -- start 2024-04-26T20:58:16.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.520+0000 7f0a96e00640 1 -- start start 2024-04-26T20:58:16.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.521+0000 7f0a96e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f0a90073990 0x7f0a90073d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.521+0000 7f0a96e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a9015c430 0x7f0a900b5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.521+0000 7f0a96e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a900b41b0 con 0x7f0a900b5790 2024-04-26T20:58:16.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.521+0000 7f0a96e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f0a900b4490 con 0x7f0a90073990 2024-04-26T20:58:16.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.521+0000 7f0a96e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f0a900b4770 con 0x7f0a9015c430 2024-04-26T20:58:16.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.522+0000 7f0a94e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f0a900b5790 0x7f0a900b5be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50132/0 (socket says 172.21.15.28:50132) 2024-04-26T20:58:16.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.522+0000 7f0a94e00640 1 -- 172.21.15.28:0/1457180261 learned_addr learned my addr 172.21.15.28:0/1457180261 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:16.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.522+0000 7f0a8fe00640 1 --2- 172.21.15.28:0/1457180261 >> v2:172.21.15.103:3300/0 conn(0x7f0a90073990 0x7f0a90073d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.522+0000 7f0a8f400640 1 --2- 172.21.15.28:0/1457180261 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a9015c430 0x7f0a900b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.522+0000 7f0a8ea00640 1 -- 172.21.15.28:0/1457180261 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3408984964 0 0) 0x7f0a900b41b0 con 0x7f0a900b5790 2024-04-26T20:58:16.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.522+0000 7f0a8ea00640 1 -- 172.21.15.28:0/1457180261 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a74003520 con 0x7f0a900b5790 2024-04-26T20:58:16.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.523+0000 7f0a8ea00640 1 -- 172.21.15.28:0/1457180261 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1982834433 0 0) 0x7f0a74003520 con 0x7f0a900b5790 2024-04-26T20:58:16.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.523+0000 7f0a8ea00640 1 -- 172.21.15.28:0/1457180261 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a9015c430 msgr2=0x7f0a900b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:16.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.523+0000 7f0a8ea00640 1 --2- 172.21.15.28:0/1457180261 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a9015c430 0x7f0a900b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.523+0000 7f0a8ea00640 1 -- 172.21.15.28:0/1457180261 >> v2:172.21.15.103:3300/0 conn(0x7f0a90073990 msgr2=0x7f0a90073d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:16.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.523+0000 7f0a8ea00640 1 --2- 172.21.15.28:0/1457180261 >> v2:172.21.15.103:3300/0 conn(0x7f0a90073990 0x7f0a90073d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.523+0000 7f0a8ea00640 1 -- 172.21.15.28:0/1457180261 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a900b4a50 con 0x7f0a900b5790 2024-04-26T20:58:16.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.523+0000 7f0a8ea00640 1 -- 172.21.15.28:0/1457180261 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f0a88001670 con 0x7f0a900b5790 2024-04-26T20:58:16.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.523+0000 7f0a8ea00640 1 -- 172.21.15.28:0/1457180261 >> v1:172.21.15.28:6789/0 conn(0x7f0a900b5790 legacy=0x7f0a900b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:16.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.523+0000 7f0a8ea00640 1 --2- 172.21.15.28:0/1457180261 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a740039b0 0x7f0a74003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.523+0000 7f0a8ea00640 1 --2- 172.21.15.28:0/1457180261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a74004310 0x7f0a74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.523+0000 7f0a8ea00640 1 --2- 172.21.15.28:0/1457180261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a74004e50 0x7f0a740052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.523+0000 7f0a8ea00640 1 -- 172.21.15.28:0/1457180261 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0a900b4490 con 0x7f0a74004310 2024-04-26T20:58:16.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.523+0000 7f0a8ea00640 1 -- 172.21.15.28:0/1457180261 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a900b4770 con 0x7f0a74004e50 2024-04-26T20:58:16.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.523+0000 7f0a8ea00640 1 -- 172.21.15.28:0/1457180261 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f0a7400dff0 con 0x7f0a740039b0 2024-04-26T20:58:16.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.523+0000 7f0a8f400640 1 --2- 172.21.15.28:0/1457180261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a74004310 0x7f0a74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.524+0000 7f0a8f400640 1 -- 172.21.15.28:0/1457180261 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a740039b0 msgr2=0x7f0a74003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:16.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.524+0000 7f0a8f400640 1 --2- 172.21.15.28:0/1457180261 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a740039b0 0x7f0a74003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.524+0000 7f0a94e00640 1 --2- 172.21.15.28:0/1457180261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a74004e50 0x7f0a740052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.524+0000 7f0a8f400640 1 -- 172.21.15.28:0/1457180261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a74004e50 msgr2=0x7f0a740052d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:16.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.524+0000 7f0a8f400640 1 --2- 172.21.15.28:0/1457180261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a74004e50 0x7f0a740052d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.524+0000 7f0a8f400640 1 -- 172.21.15.28:0/1457180261 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a74003520 con 0x7f0a74004310 2024-04-26T20:58:16.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.524+0000 7f0a94e00640 1 --2- 172.21.15.28:0/1457180261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a74004e50 0x7f0a740052d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:16.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.525+0000 7f0a8f400640 1 --2- 172.21.15.28:0/1457180261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a74004310 0x7f0a74004800 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c009ac0 tx=0x7f0a74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92775799688e2ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:16.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.525+0000 7f0a8ea00640 1 -- 172.21.15.28:0/1457180261 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c036070 con 0x7f0a74004310 2024-04-26T20:58:16.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.525+0000 7f0a8ea00640 1 -- 172.21.15.28:0/1457180261 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a7c031400 con 0x7f0a74004310 2024-04-26T20:58:16.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.525+0000 7f0a8ea00640 1 -- 172.21.15.28:0/1457180261 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c009690 con 0x7f0a74004310 2024-04-26T20:58:16.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.526+0000 7f0a96e00640 1 -- 172.21.15.28:0/1457180261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a74004310 msgr2=0x7f0a74004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:16.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.526+0000 7f0a96e00640 1 --2- 172.21.15.28:0/1457180261 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a74004310 0x7f0a74004800 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c009ac0 tx=0x7f0a74005830 comp rx=0 tx=0).stop 2024-04-26T20:58:16.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.526+0000 7f0a94e00640 1 -- 172.21.15.28:0/1457180261 reap_dead start 2024-04-26T20:58:16.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.526+0000 7f0a96e00640 1 -- 172.21.15.28:0/1457180261 shutdown_connections 2024-04-26T20:58:16.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.526+0000 7f0a96e00640 1 -- 172.21.15.28:0/1457180261 >> 172.21.15.28:0/1457180261 conn(0x7f0a9006e220 msgr2=0x7f0a9006e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:16.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.526+0000 7f0a96e00640 1 -- 172.21.15.28:0/1457180261 shutdown_connections 2024-04-26T20:58:16.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.526+0000 7f0a96e00640 1 -- 172.21.15.28:0/1457180261 wait complete. 2024-04-26T20:58:16.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.528+0000 7f0a96e00640 1 Processor -- start 2024-04-26T20:58:16.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.528+0000 7f0a96e00640 1 -- start start 2024-04-26T20:58:16.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.529+0000 7f0a96e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a74004310 0x7f0a90140c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.529+0000 7f0a96e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a900afe20 0x7f0a900ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.529+0000 7f0a96e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a900addc0 0x7f0a900ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.529+0000 7f0a96e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0a90072220 con 0x7f0a74004310 2024-04-26T20:58:16.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.529+0000 7f0a96e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a900742b0 con 0x7f0a900addc0 2024-04-26T20:58:16.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.529+0000 7f0a96e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f0a900723a0 con 0x7f0a900afe20 2024-04-26T20:58:16.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.529+0000 7f0a8fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a74004310 0x7f0a90140c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.529+0000 7f0a8fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a74004310 0x7f0a90140c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52062/0 (socket says 172.21.15.28:52062) 2024-04-26T20:58:16.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.529+0000 7f0a8fe00640 1 -- 172.21.15.28:0/3475024692 learned_addr learned my addr 172.21.15.28:0/3475024692 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:16.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.529+0000 7f0a8f400640 1 --2- 172.21.15.28:0/3475024692 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a900afe20 0x7f0a900ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.529+0000 7f0a8fe00640 1 -- 172.21.15.28:0/3475024692 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a900afe20 msgr2=0x7f0a900ac4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:16.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.529+0000 7f0a8fe00640 1 --2- 172.21.15.28:0/3475024692 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a900afe20 0x7f0a900ac4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.529+0000 7f0a8fe00640 1 -- 172.21.15.28:0/3475024692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a900addc0 msgr2=0x7f0a900ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:16.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.529+0000 7f0a8fe00640 1 --2- 172.21.15.28:0/3475024692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a900addc0 0x7f0a900ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.529+0000 7f0a8fe00640 1 -- 172.21.15.28:0/3475024692 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a900aca00 con 0x7f0a74004310 2024-04-26T20:58:16.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.530+0000 7f0a8fe00640 1 --2- 172.21.15.28:0/3475024692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a74004310 0x7f0a90140c50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009db0 tx=0x7f0a8000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:16.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.530+0000 7f0a87e00640 1 -- 172.21.15.28:0/3475024692 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80018020 con 0x7f0a74004310 2024-04-26T20:58:16.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.530+0000 7f0a87e00640 1 -- 172.21.15.28:0/3475024692 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a80014480 con 0x7f0a74004310 2024-04-26T20:58:16.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.530+0000 7f0a87e00640 1 -- 172.21.15.28:0/3475024692 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80014870 con 0x7f0a74004310 2024-04-26T20:58:16.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.530+0000 7f0a96e00640 1 -- 172.21.15.28:0/3475024692 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a900acca0 con 0x7f0a74004310 2024-04-26T20:58:16.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.530+0000 7f0a96e00640 1 -- 172.21.15.28:0/3475024692 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a900ad260 con 0x7f0a74004310 2024-04-26T20:58:16.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.532+0000 7f0a87e00640 1 -- 172.21.15.28:0/3475024692 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a80005d10 con 0x7f0a74004310 2024-04-26T20:58:16.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.533+0000 7f0a87e00640 1 --2- 172.21.15.28:0/3475024692 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0a7006c8f0 0x7f0a7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.534+0000 7f0a87e00640 1 -- 172.21.15.28:0/3475024692 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f0a8008c720 con 0x7f0a74004310 2024-04-26T20:58:16.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.534+0000 7f0a8f400640 1 --2- 172.21.15.28:0/3475024692 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0a7006c8f0 0x7f0a7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.534+0000 7f0a99db4740 1 Processor -- start 2024-04-26T20:58:16.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.534+0000 7f0a99db4740 1 -- start start 2024-04-26T20:58:16.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.535+0000 7f0a8f400640 1 --2- 172.21.15.28:0/3475024692 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0a7006c8f0 0x7f0a7006edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c033000 tx=0x7f0a7c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:16.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.535+0000 7f0a99db4740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 0x560dc76e0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.535+0000 7f0a99db4740 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x560dc771acb0 0x560dc76de9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.535+0000 7f0a99db4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dc77090a0 0x560dc7709520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.535+0000 7f0a99db4740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x560dc7642600 con 0x560dc76de5e0 2024-04-26T20:58:16.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.535+0000 7f0a99db4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560dc75df100 con 0x560dc77090a0 2024-04-26T20:58:16.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.535+0000 7f0a99db4740 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x560dc763e5c0 con 0x560dc771acb0 2024-04-26T20:58:16.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.536+0000 7f0a8fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 0x560dc76e0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.536+0000 7f0a8fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 0x560dc76e0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52064/0 (socket says 172.21.15.28:52064) 2024-04-26T20:58:16.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.536+0000 7f0a8fe00640 1 -- 172.21.15.28:0/1813640886 learned_addr learned my addr 172.21.15.28:0/1813640886 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:16.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.536+0000 7f0a8fe00640 1 -- 172.21.15.28:0/1813640886 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x560dc771acb0 msgr2=0x560dc76de9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:16.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.536+0000 7f0a8fe00640 1 --2- 172.21.15.28:0/1813640886 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x560dc771acb0 0x560dc76de9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.536+0000 7f0a8fe00640 1 -- 172.21.15.28:0/1813640886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dc77090a0 msgr2=0x560dc7709520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:16.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.536+0000 7f0a8fe00640 1 --2- 172.21.15.28:0/1813640886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dc77090a0 0x560dc7709520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.536+0000 7f0a8fe00640 1 -- 172.21.15.28:0/1813640886 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dc73542d0 con 0x560dc76de5e0 2024-04-26T20:58:16.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.536+0000 7f0a8fe00640 1 --2- 172.21.15.28:0/1813640886 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 0x560dc76e0740 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0a80062a60 tx=0x7f0a8005a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813710cd878653fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:16.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.536+0000 7f0a84c00640 1 -- 172.21.15.28:0/1813640886 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a800586b0 con 0x560dc76de5e0 2024-04-26T20:58:16.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.537+0000 7f0a84c00640 1 -- 172.21.15.28:0/1813640886 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a8001ada0 con 0x560dc76de5e0 2024-04-26T20:58:16.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.537+0000 7f0a84c00640 1 -- 172.21.15.28:0/1813640886 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80059800 con 0x560dc76de5e0 2024-04-26T20:58:16.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.537+0000 7f0a99db4740 1 -- 172.21.15.28:0/1813640886 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 msgr2=0x560dc76e0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:16.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.537+0000 7f0a99db4740 1 --2- 172.21.15.28:0/1813640886 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 0x560dc76e0740 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0a80062a60 tx=0x7f0a8005a470 comp rx=0 tx=0).stop 2024-04-26T20:58:16.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.537+0000 7f0a99db4740 1 -- 172.21.15.28:0/1813640886 shutdown_connections 2024-04-26T20:58:16.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.537+0000 7f0a99db4740 1 --2- 172.21.15.28:0/1813640886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dc77090a0 0x560dc7709520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.537+0000 7f0a99db4740 1 --2- 172.21.15.28:0/1813640886 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x560dc771acb0 0x560dc76de9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.537+0000 7f0a99db4740 1 --2- 172.21.15.28:0/1813640886 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 0x560dc76e0740 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.537+0000 7f0a99db4740 1 -- 172.21.15.28:0/1813640886 >> 172.21.15.28:0/1813640886 conn(0x560dc7672d10 msgr2=0x560dc77240a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:16.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.537+0000 7f0a99db4740 1 -- 172.21.15.28:0/1813640886 shutdown_connections 2024-04-26T20:58:16.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.538+0000 7f0a99db4740 1 -- 172.21.15.28:0/1813640886 wait complete. 2024-04-26T20:58:16.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.538+0000 7f0a99db4740 1 -- start start 2024-04-26T20:58:16.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.538+0000 7f0a99db4740 1 Processor -- start 2024-04-26T20:58:16.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.539+0000 7f0a99db4740 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x560dc76de5e0 0x560dc771e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.539+0000 7f0a99db4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dc77090a0 0x560dc76f9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.539+0000 7f0a99db4740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc771acb0 0x560dc771f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.539+0000 7f0a99db4740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x560dc75c9f70 con 0x560dc771acb0 2024-04-26T20:58:16.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.539+0000 7f0a99db4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560dc7706bf0 con 0x560dc77090a0 2024-04-26T20:58:16.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.539+0000 7f0a99db4740 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x560dc76f9e80 con 0x560dc76de5e0 2024-04-26T20:58:16.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.539+0000 7f0a99db4740 10 client.0 _ll_register_callbacks cb 0x560dc7638480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:58:16.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.540+0000 7f0a94e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc771acb0 0x560dc771f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.540+0000 7f0a94e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc771acb0 0x560dc771f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52072/0 (socket says 172.21.15.28:52072) 2024-04-26T20:58:16.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.540+0000 7f0a94e00640 1 -- 172.21.15.28:0/3175019355 learned_addr learned my addr 172.21.15.28:0/3175019355 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:16.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.540+0000 7f0a8fe00640 1 --2- 172.21.15.28:0/3175019355 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x560dc76de5e0 0x560dc771e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.540+0000 7f0a8f400640 1 --2- 172.21.15.28:0/3175019355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dc77090a0 0x560dc76f9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.540+0000 7f0a94e00640 1 -- 172.21.15.28:0/3175019355 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x560dc76de5e0 msgr2=0x560dc771e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:16.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.540+0000 7f0a94e00640 1 --2- 172.21.15.28:0/3175019355 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x560dc76de5e0 0x560dc771e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.540+0000 7f0a94e00640 1 -- 172.21.15.28:0/3175019355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dc77090a0 msgr2=0x560dc76f9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:16.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.540+0000 7f0a94e00640 1 --2- 172.21.15.28:0/3175019355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dc77090a0 0x560dc76f9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.540+0000 7f0a94e00640 1 -- 172.21.15.28:0/3175019355 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dc741e6b0 con 0x560dc771acb0 2024-04-26T20:58:16.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.541+0000 7f0a94e00640 1 --2- 172.21.15.28:0/3175019355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc771acb0 0x560dc771f430 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0a88001050 tx=0x7f0a8800d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ec60ef7fef3ffb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:16.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.541+0000 7f0a79600640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-04-26T20:58:16.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.541+0000 7f0a79600640 1 -- 172.21.15.28:0/3175019355 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a880072d0 con 0x560dc771acb0 2024-04-26T20:58:16.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.541+0000 7f0a79600640 1 -- 172.21.15.28:0/3175019355 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f0a88007470 con 0x560dc771acb0 2024-04-26T20:58:16.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.541+0000 7f0a99db4740 1 -- 172.21.15.28:0/3175019355 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a8001ada0 con 0x560dc771acb0 2024-04-26T20:58:16.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.541+0000 7f0a79600640 1 -- 172.21.15.28:0/3175019355 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f0a88007670 con 0x560dc771acb0 2024-04-26T20:58:16.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.541+0000 7f0a99db4740 10 client.5358 fetch_fsmap learned FSMap version 17 2024-04-26T20:58:16.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.541+0000 7f0a99db4740 1 -- 172.21.15.28:0/3175019355 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x560dc77095c0 con 0x560dc771acb0 2024-04-26T20:58:16.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.541+0000 7f0a79600640 1 -- 172.21.15.28:0/3175019355 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a8001ada0 con 0x560dc771acb0 2024-04-26T20:58:16.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.542+0000 7f0a79600640 1 -- 172.21.15.28:0/3175019355 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f0a8801c050 con 0x560dc771acb0 2024-04-26T20:58:16.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.542+0000 7f0a99db4740 10 client.5358 fetch_fsmap finished waiting for FSMap version 17 2024-04-26T20:58:16.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.542+0000 7f0a99db4740 10 client.5358 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T20:58:16.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.542+0000 7f0a99db4740 20 client.5358 populate_metadata read hostname 'smithi028' 2024-04-26T20:58:16.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.542+0000 7f0a99db4740 1 --2- 172.21.15.28:0/3175019355 >> [v2:172.21.15.103:6833/1781724684,v1:172.21.15.103:6835/1781724684] conn(0x560dc77373e0 0x560dc7709cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.542+0000 7f0a99db4740 4 client.5358 mds_command: new command op to 5184 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:58:16.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.542+0000 7f0a99db4740 1 -- 172.21.15.28:0/3175019355 --> [v2:172.21.15.103:6833/1781724684,v1:172.21.15.103:6835/1781724684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dc7650210 con 0x560dc77373e0 2024-04-26T20:58:16.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.543+0000 7f0a8fe00640 1 --2- 172.21.15.28:0/3175019355 >> [v2:172.21.15.103:6833/1781724684,v1:172.21.15.103:6835/1781724684] conn(0x560dc77373e0 0x560dc7709cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.544+0000 7f0a8fe00640 1 --2- 172.21.15.28:0/3175019355 >> [v2:172.21.15.103:6833/1781724684,v1:172.21.15.103:6835/1781724684] conn(0x560dc77373e0 0x560dc7709cf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e0364fac9a8f82d server_cookie=fc792d58fd3f7bd6 in_seq=0 out_seq=0 2024-04-26T20:58:16.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.544+0000 7f0a79600640 10 client.5358 ms_handle_connect on v2:172.21.15.103:6833/1781724684 2024-04-26T20:58:16.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.545+0000 7f0a79600640 1 -- 172.21.15.28:0/3175019355 <== mds.0 v2:172.21.15.103:6833/1781724684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560dc7650210 con 0x560dc77373e0 2024-04-26T20:58:16.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.545+0000 7f0a79600640 10 client.5358 handle_command_reply: tid=0 2024-04-26T20:58:16.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.546+0000 7f0a99db4740 1 client.5358 shutdown 2024-04-26T20:58:16.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.546+0000 7f0a99db4740 1 -- 172.21.15.28:0/3175019355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc771acb0 msgr2=0x560dc771f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:16.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.546+0000 7f0a99db4740 1 --2- 172.21.15.28:0/3175019355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc771acb0 0x560dc771f430 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0a88001050 tx=0x7f0a8800d5e0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.546+0000 7f0a99db4740 1 -- 172.21.15.28:0/3175019355 shutdown_connections 2024-04-26T20:58:16.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.546+0000 7f0a99db4740 1 --2- 172.21.15.28:0/3175019355 >> [v2:172.21.15.103:6833/1781724684,v1:172.21.15.103:6835/1781724684] conn(0x560dc77373e0 0x560dc7709cf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.546+0000 7f0a99db4740 1 --2- 172.21.15.28:0/3175019355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc771acb0 0x560dc771f430 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.546+0000 7f0a99db4740 1 --2- 172.21.15.28:0/3175019355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dc77090a0 0x560dc76f9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.546+0000 7f0a99db4740 1 --2- 172.21.15.28:0/3175019355 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x560dc76de5e0 0x560dc771e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.546+0000 7f0a79600640 0 client.5358 ms_handle_reset on v2:172.21.15.103:6833/1781724684 2024-04-26T20:58:16.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.546+0000 7f0a99db4740 1 -- 172.21.15.28:0/3175019355 >> 172.21.15.28:0/3175019355 conn(0x560dc7672d10 msgr2=0x560dc7727590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:16.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.546+0000 7f0a99db4740 1 -- 172.21.15.28:0/3175019355 shutdown_connections 2024-04-26T20:58:16.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.546+0000 7f0a99db4740 1 -- 172.21.15.28:0/3175019355 wait complete. 2024-04-26T20:58:16.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.546+0000 7f0a99db4740 20 client.5358 trim_cache size 0 max 16384 2024-04-26T20:58:16.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.566+0000 7f0a99db4740 1 Processor -- start 2024-04-26T20:58:16.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.567+0000 7f0a99db4740 1 -- start start 2024-04-26T20:58:16.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.567+0000 7f0a99db4740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 0x560dc7707c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.567+0000 7f0a99db4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dc77090a0 0x560dc7704240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.567+0000 7f0a99db4740 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x560dc771acb0 0x560dc771e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.567+0000 7f0a99db4740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0a90200a30 con 0x560dc76de5e0 2024-04-26T20:58:16.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.567+0000 7f0a99db4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a901ffda0 con 0x560dc77090a0 2024-04-26T20:58:16.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.567+0000 7f0a99db4740 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f0a902003c0 con 0x560dc771acb0 2024-04-26T20:58:16.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.568+0000 7f0a8fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 0x560dc7707c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.568+0000 7f0a8fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 0x560dc7707c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52088/0 (socket says 172.21.15.28:52088) 2024-04-26T20:58:16.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.568+0000 7f0a8fe00640 1 -- 172.21.15.28:0/3225539403 learned_addr learned my addr 172.21.15.28:0/3225539403 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:16.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.568+0000 7f0a8f400640 1 --2- 172.21.15.28:0/3225539403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dc77090a0 0x560dc7704240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.568+0000 7f0a8fe00640 1 -- 172.21.15.28:0/3225539403 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x560dc771acb0 msgr2=0x560dc771e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:16.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.568+0000 7f0a94e00640 1 --2- 172.21.15.28:0/3225539403 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x560dc771acb0 0x560dc771e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.568+0000 7f0a8fe00640 1 --2- 172.21.15.28:0/3225539403 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x560dc771acb0 0x560dc771e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.568+0000 7f0a8fe00640 1 -- 172.21.15.28:0/3225539403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dc77090a0 msgr2=0x560dc7704240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:16.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.568+0000 7f0a8fe00640 1 --2- 172.21.15.28:0/3225539403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dc77090a0 0x560dc7704240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.568+0000 7f0a8fe00640 1 -- 172.21.15.28:0/3225539403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dc76f9cb0 con 0x560dc76de5e0 2024-04-26T20:58:16.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.568+0000 7f0a8f400640 1 --2- 172.21.15.28:0/3225539403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dc77090a0 0x560dc7704240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:16.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.568+0000 7f0a8fe00640 1 --2- 172.21.15.28:0/3225539403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 0x560dc7707c60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0a8008abb0 tx=0x7f0a8005d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f31f805765ee084 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:16.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.569+0000 7f0a7be00640 1 -- 172.21.15.28:0/3225539403 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80058770 con 0x560dc76de5e0 2024-04-26T20:58:16.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.569+0000 7f0a7be00640 1 -- 172.21.15.28:0/3225539403 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a80072b90 con 0x560dc76de5e0 2024-04-26T20:58:16.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.569+0000 7f0a7be00640 1 -- 172.21.15.28:0/3225539403 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80059800 con 0x560dc76de5e0 2024-04-26T20:58:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.569+0000 7f0a99db4740 1 -- 172.21.15.28:0/3225539403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 msgr2=0x560dc7707c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.569+0000 7f0a99db4740 1 --2- 172.21.15.28:0/3225539403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 0x560dc7707c60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0a8008abb0 tx=0x7f0a8005d000 comp rx=0 tx=0).stop 2024-04-26T20:58:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.569+0000 7f0a99db4740 1 -- 172.21.15.28:0/3225539403 shutdown_connections 2024-04-26T20:58:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.569+0000 7f0a99db4740 1 --2- 172.21.15.28:0/3225539403 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x560dc771acb0 0x560dc771e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.569+0000 7f0a99db4740 1 --2- 172.21.15.28:0/3225539403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dc77090a0 0x560dc7704240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.569+0000 7f0a99db4740 1 --2- 172.21.15.28:0/3225539403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 0x560dc7707c60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.569+0000 7f0a99db4740 1 -- 172.21.15.28:0/3225539403 >> 172.21.15.28:0/3225539403 conn(0x560dc7672d10 msgr2=0x560dc7708620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.570+0000 7f0a99db4740 1 -- 172.21.15.28:0/3225539403 shutdown_connections 2024-04-26T20:58:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.570+0000 7f0a99db4740 1 -- 172.21.15.28:0/3225539403 wait complete. 2024-04-26T20:58:16.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.570+0000 7f0a99db4740 1 -- start start 2024-04-26T20:58:16.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.570+0000 7f0a99db4740 1 Processor -- start 2024-04-26T20:58:16.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.571+0000 7f0a99db4740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 0x560dc7736500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.571+0000 7f0a99db4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dc77090a0 0x560dc771d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.571+0000 7f0a99db4740 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x560dc771acb0 0x560dc7709790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.571+0000 7f0a99db4740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x560dc7737260 con 0x560dc76de5e0 2024-04-26T20:58:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.571+0000 7f0a99db4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560dc770ac20 con 0x560dc77090a0 2024-04-26T20:58:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.571+0000 7f0a99db4740 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x560dc77378b0 con 0x560dc771acb0 2024-04-26T20:58:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.571+0000 7f0a99db4740 10 client.0 _ll_register_callbacks cb 0x7f0a88001cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:58:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.572+0000 7f0a8fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 0x560dc7736500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.572+0000 7f0a8fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 0x560dc7736500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52102/0 (socket says 172.21.15.28:52102) 2024-04-26T20:58:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.572+0000 7f0a8fe00640 1 -- 172.21.15.28:0/645398087 learned_addr learned my addr 172.21.15.28:0/645398087 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.572+0000 7f0a8fe00640 1 -- 172.21.15.28:0/645398087 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x560dc771acb0 msgr2=0x560dc7709790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.572+0000 7f0a8fe00640 1 --2- 172.21.15.28:0/645398087 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x560dc771acb0 0x560dc7709790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.572+0000 7f0a8fe00640 1 -- 172.21.15.28:0/645398087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dc77090a0 msgr2=0x560dc771d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.572+0000 7f0a8fe00640 1 --2- 172.21.15.28:0/645398087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dc77090a0 0x560dc771d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.572+0000 7f0a8fe00640 1 -- 172.21.15.28:0/645398087 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dc773e1e0 con 0x560dc76de5e0 2024-04-26T20:58:16.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.572+0000 7f0a8fe00640 1 --2- 172.21.15.28:0/645398087 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 0x560dc7736500 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0a800094c0 tx=0x7f0a80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d20a5dde078072 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:16.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.572+0000 7f0a79600640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-04-26T20:58:16.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.572+0000 7f0a79600640 1 -- 172.21.15.28:0/645398087 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a80059a00 con 0x560dc76de5e0 2024-04-26T20:58:16.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.573+0000 7f0a79600640 1 -- 172.21.15.28:0/645398087 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f0a80056100 con 0x560dc76de5e0 2024-04-26T20:58:16.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.573+0000 7f0a99db4740 1 -- 172.21.15.28:0/645398087 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a80072b90 con 0x560dc76de5e0 2024-04-26T20:58:16.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.573+0000 7f0a79600640 1 -- 172.21.15.28:0/645398087 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f0a80058770 con 0x560dc76de5e0 2024-04-26T20:58:16.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.573+0000 7f0a99db4740 10 client.5364 fetch_fsmap learned FSMap version 17 2024-04-26T20:58:16.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.573+0000 7f0a99db4740 1 -- 172.21.15.28:0/645398087 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x560dc771e5e0 con 0x560dc76de5e0 2024-04-26T20:58:16.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.573+0000 7f0a79600640 1 -- 172.21.15.28:0/645398087 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a80072b90 con 0x560dc76de5e0 2024-04-26T20:58:16.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.573+0000 7f0a79600640 1 -- 172.21.15.28:0/645398087 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 17) ==== 3494+0+0 (secure 0 0 0) 0x7f0a80056300 con 0x560dc76de5e0 2024-04-26T20:58:16.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.573+0000 7f0a99db4740 10 client.5364 fetch_fsmap finished waiting for FSMap version 17 2024-04-26T20:58:16.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.573+0000 7f0a99db4740 10 client.5364 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T20:58:16.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.573+0000 7f0a99db4740 20 client.5364 populate_metadata read hostname 'smithi028' 2024-04-26T20:58:16.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.573+0000 7f0a99db4740 1 --2- 172.21.15.28:0/645398087 >> [v2:172.21.15.103:6833/1781724684,v1:172.21.15.103:6835/1781724684] conn(0x560dc7709cd0 0x560dc77466f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.573+0000 7f0a99db4740 4 client.5364 mds_command: new command op to 5184 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:58:16.576 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.573+0000 7f0a99db4740 1 -- 172.21.15.28:0/645398087 --> [v2:172.21.15.103:6833/1781724684,v1:172.21.15.103:6835/1781724684] -- command(tid 0: {"prefix": "status"}) -- 0x560dc764a080 con 0x560dc7709cd0 2024-04-26T20:58:16.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.574+0000 7f0a8f400640 1 --2- 172.21.15.28:0/645398087 >> [v2:172.21.15.103:6833/1781724684,v1:172.21.15.103:6835/1781724684] conn(0x560dc7709cd0 0x560dc77466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.575+0000 7f0a8f400640 1 --2- 172.21.15.28:0/645398087 >> [v2:172.21.15.103:6833/1781724684,v1:172.21.15.103:6835/1781724684] conn(0x560dc7709cd0 0x560dc77466f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=908df943ef9a397a server_cookie=4a859f4deb264996 in_seq=0 out_seq=0 2024-04-26T20:58:16.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.575+0000 7f0a79600640 10 client.5364 ms_handle_connect on v2:172.21.15.103:6833/1781724684 2024-04-26T20:58:16.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.576+0000 7f0a79600640 1 -- 172.21.15.28:0/645398087 <== mds.0 v2:172.21.15.103:6833/1781724684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x560dc764a080 con 0x560dc7709cd0 2024-04-26T20:58:16.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.576+0000 7f0a79600640 10 client.5364 handle_command_reply: tid=0 2024-04-26T20:58:16.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.576+0000 7f0a99db4740 1 client.5364 shutdown 2024-04-26T20:58:16.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.576+0000 7f0a99db4740 1 -- 172.21.15.28:0/645398087 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 msgr2=0x560dc7736500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:16.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.576+0000 7f0a99db4740 1 --2- 172.21.15.28:0/645398087 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 0x560dc7736500 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0a800094c0 tx=0x7f0a80004a40 comp rx=0 tx=0).stop 2024-04-26T20:58:16.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.576+0000 7f0a99db4740 1 -- 172.21.15.28:0/645398087 shutdown_connections 2024-04-26T20:58:16.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.576+0000 7f0a99db4740 1 --2- 172.21.15.28:0/645398087 >> [v2:172.21.15.103:6833/1781724684,v1:172.21.15.103:6835/1781724684] conn(0x560dc7709cd0 0x560dc77466f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.578 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.576+0000 7f0a99db4740 1 --2- 172.21.15.28:0/645398087 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x560dc771acb0 0x560dc7709790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.576+0000 7f0a99db4740 1 --2- 172.21.15.28:0/645398087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560dc77090a0 0x560dc771d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.576+0000 7f0a99db4740 1 --2- 172.21.15.28:0/645398087 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x560dc76de5e0 0x560dc7736500 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.576+0000 7f0a79600640 0 client.5364 ms_handle_reset on v2:172.21.15.103:6833/1781724684 2024-04-26T20:58:16.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.576+0000 7f0a99db4740 1 -- 172.21.15.28:0/645398087 >> 172.21.15.28:0/645398087 conn(0x560dc7672d10 msgr2=0x560dc7727660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:16.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.576+0000 7f0a99db4740 1 -- 172.21.15.28:0/645398087 shutdown_connections 2024-04-26T20:58:16.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.576+0000 7f0a99db4740 1 -- 172.21.15.28:0/645398087 wait complete. 2024-04-26T20:58:16.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.576+0000 7f0a99db4740 20 client.5364 trim_cache size 0 max 16384 2024-04-26T20:58:16.579 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T20:58:16.579 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "79845bd7-2c94-4fa6-8082-292285d530c9", 2024-04-26T20:58:16.579 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-04-26T20:58:16.579 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 5184, 2024-04-26T20:58:16.579 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-04-26T20:58:16.579 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-04-26T20:58:16.580 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-04-26T20:58:16.580 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 6.7799311109999998, 2024-04-26T20:58:16.580 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 17, 2024-04-26T20:58:16.580 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 34, 2024-04-26T20:58:16.580 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 34, 2024-04-26T20:58:16.580 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 15.706993584999999 2024-04-26T20:58:16.580 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T20:58:16.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.577+0000 7f0a96e00640 1 -- 172.21.15.28:0/3475024692 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0a7006c8f0 msgr2=0x7f0a7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:16.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.577+0000 7f0a96e00640 1 --2- 172.21.15.28:0/3475024692 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0a7006c8f0 0x7f0a7006edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c033000 tx=0x7f0a7c008030 comp rx=0 tx=0).stop 2024-04-26T20:58:16.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.577+0000 7f0a96e00640 1 -- 172.21.15.28:0/3475024692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a74004310 msgr2=0x7f0a90140c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:16.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.577+0000 7f0a96e00640 1 --2- 172.21.15.28:0/3475024692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a74004310 0x7f0a90140c50 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009db0 tx=0x7f0a8000c490 comp rx=0 tx=0).stop 2024-04-26T20:58:16.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.577+0000 7f0a96e00640 1 -- 172.21.15.28:0/3475024692 shutdown_connections 2024-04-26T20:58:16.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.577+0000 7f0a96e00640 1 --2- 172.21.15.28:0/3475024692 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0a7006c8f0 0x7f0a7006edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.577+0000 7f0a96e00640 1 --2- 172.21.15.28:0/3475024692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a900addc0 0x7f0a900ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.577+0000 7f0a96e00640 1 --2- 172.21.15.28:0/3475024692 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a900afe20 0x7f0a900ac4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.577+0000 7f0a96e00640 1 --2- 172.21.15.28:0/3475024692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a74004310 0x7f0a90140c50 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.577+0000 7f0a96e00640 1 -- 172.21.15.28:0/3475024692 >> 172.21.15.28:0/3475024692 conn(0x7f0a9006e220 msgr2=0x7f0a90142880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:16.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.577+0000 7f0a96e00640 1 -- 172.21.15.28:0/3475024692 shutdown_connections 2024-04-26T20:58:16.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.577+0000 7f0a96e00640 1 -- 172.21.15.28:0/3475024692 wait complete. 2024-04-26T20:58:16.596 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:58:16.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.793+0000 7f9a2be00640 1 Processor -- start 2024-04-26T20:58:16.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.793+0000 7f9a2be00640 1 -- start start 2024-04-26T20:58:16.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.794+0000 7f9a2be00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f9a2c15f2e0 0x7f9a2c06c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.794+0000 7f9a2be00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9a2c0d38f0 0x7f9a2c06c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.795+0000 7f9a2be00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a2c0d3db0 con 0x7f9a2c0d2a00 2024-04-26T20:58:16.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.795+0000 7f9a2be00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f9a2c0d4090 con 0x7f9a2c15f2e0 2024-04-26T20:58:16.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.795+0000 7f9a2be00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9a2c0d4370 con 0x7f9a2c0d38f0 2024-04-26T20:58:16.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.795+0000 7f9a2b400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f9a2c0d2a00 0x7f9a2c0b38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:50136/0 (socket says 172.21.15.28:50136) 2024-04-26T20:58:16.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.795+0000 7f9a2b400640 1 -- 172.21.15.28:0/106193524 learned_addr learned my addr 172.21.15.28:0/106193524 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:16.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.795+0000 7f9a23e00640 1 --2- 172.21.15.28:0/106193524 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9a2c0d38f0 0x7f9a2c06c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.795+0000 7f9a2aa00640 1 --2- 172.21.15.28:0/106193524 >> v2:172.21.15.103:3300/0 conn(0x7f9a2c15f2e0 0x7f9a2c06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.795+0000 7f9a2a000640 1 -- 172.21.15.28:0/106193524 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1815807916 0 0) 0x7f9a2c0d3db0 con 0x7f9a2c0d2a00 2024-04-26T20:58:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.796+0000 7f9a2a000640 1 -- 172.21.15.28:0/106193524 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a10003520 con 0x7f9a2c0d2a00 2024-04-26T20:58:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.796+0000 7f9a2a000640 1 -- 172.21.15.28:0/106193524 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3643172025 0 0) 0x7f9a10003520 con 0x7f9a2c0d2a00 2024-04-26T20:58:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.796+0000 7f9a2a000640 1 -- 172.21.15.28:0/106193524 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9a2c0d38f0 msgr2=0x7f9a2c06c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.796+0000 7f9a2a000640 1 --2- 172.21.15.28:0/106193524 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9a2c0d38f0 0x7f9a2c06c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.796+0000 7f9a2a000640 1 -- 172.21.15.28:0/106193524 >> v2:172.21.15.103:3300/0 conn(0x7f9a2c15f2e0 msgr2=0x7f9a2c06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.796+0000 7f9a2a000640 1 --2- 172.21.15.28:0/106193524 >> v2:172.21.15.103:3300/0 conn(0x7f9a2c15f2e0 0x7f9a2c06c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.796+0000 7f9a2a000640 1 -- 172.21.15.28:0/106193524 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2c0d4650 con 0x7f9a2c0d2a00 2024-04-26T20:58:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.796+0000 7f9a2a000640 1 -- 172.21.15.28:0/106193524 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f9a240019f0 con 0x7f9a2c0d2a00 2024-04-26T20:58:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.796+0000 7f9a2a000640 1 -- 172.21.15.28:0/106193524 >> v1:172.21.15.28:6789/0 conn(0x7f9a2c0d2a00 legacy=0x7f9a2c0b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.796+0000 7f9a2a000640 1 --2- 172.21.15.28:0/106193524 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9a2c0d38f0 0x7f9a100039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.796+0000 7f9a2a000640 1 --2- 172.21.15.28:0/106193524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a10003f80 0x7f9a10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.796+0000 7f9a2a000640 1 --2- 172.21.15.28:0/106193524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a10004d40 0x7f9a100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.796+0000 7f9a2a000640 1 -- 172.21.15.28:0/106193524 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c0d4090 con 0x7f9a10003f80 2024-04-26T20:58:16.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.796+0000 7f9a2a000640 1 -- 172.21.15.28:0/106193524 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c0d4370 con 0x7f9a10004d40 2024-04-26T20:58:16.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.796+0000 7f9a2a000640 1 -- 172.21.15.28:0/106193524 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9a1000b9c0 con 0x7f9a2c0d38f0 2024-04-26T20:58:16.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.796+0000 7f9a23e00640 1 --2- 172.21.15.28:0/106193524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a10003f80 0x7f9a10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.797+0000 7f9a2aa00640 1 --2- 172.21.15.28:0/106193524 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9a2c0d38f0 0x7f9a100039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.797+0000 7f9a23e00640 1 -- 172.21.15.28:0/106193524 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9a2c0d38f0 msgr2=0x7f9a100039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:16.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.797+0000 7f9a23e00640 1 --2- 172.21.15.28:0/106193524 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9a2c0d38f0 0x7f9a100039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.797+0000 7f9a23e00640 1 -- 172.21.15.28:0/106193524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a10004d40 msgr2=0x7f9a100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:16.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.797+0000 7f9a23e00640 1 --2- 172.21.15.28:0/106193524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a10004d40 0x7f9a100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.797+0000 7f9a23e00640 1 -- 172.21.15.28:0/106193524 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a10003520 con 0x7f9a10003f80 2024-04-26T20:58:16.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.798+0000 7f9a23e00640 1 --2- 172.21.15.28:0/106193524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a10003f80 0x7f9a10004800 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009af0 tx=0x7f9a10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8709e75f9a45162a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:16.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.798+0000 7f9a2a000640 1 -- 172.21.15.28:0/106193524 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14036070 con 0x7f9a10003f80 2024-04-26T20:58:16.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.798+0000 7f9a2a000640 1 -- 172.21.15.28:0/106193524 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a14031400 con 0x7f9a10003f80 2024-04-26T20:58:16.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.798+0000 7f9a2a000640 1 -- 172.21.15.28:0/106193524 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14009690 con 0x7f9a10003f80 2024-04-26T20:58:16.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.799+0000 7f9a2be00640 1 -- 172.21.15.28:0/106193524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a10003f80 msgr2=0x7f9a10004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:16.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.799+0000 7f9a2be00640 1 --2- 172.21.15.28:0/106193524 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a10003f80 0x7f9a10004800 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9a14009af0 tx=0x7f9a10005830 comp rx=0 tx=0).stop 2024-04-26T20:58:16.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.799+0000 7f9a2b400640 1 -- 172.21.15.28:0/106193524 reap_dead start 2024-04-26T20:58:16.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.799+0000 7f9a2be00640 1 -- 172.21.15.28:0/106193524 shutdown_connections 2024-04-26T20:58:16.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.799+0000 7f9a2be00640 1 -- 172.21.15.28:0/106193524 >> 172.21.15.28:0/106193524 conn(0x7f9a2c0d4e30 msgr2=0x7f9a2c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:16.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.799+0000 7f9a2be00640 1 -- 172.21.15.28:0/106193524 shutdown_connections 2024-04-26T20:58:16.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.799+0000 7f9a2be00640 1 -- 172.21.15.28:0/106193524 wait complete. 2024-04-26T20:58:16.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.801+0000 7f9a2be00640 1 Processor -- start 2024-04-26T20:58:16.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.801+0000 7f9a2be00640 1 -- start start 2024-04-26T20:58:16.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.802+0000 7f9a2be00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9a10003f80 0x7f9a2c0cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.802+0000 7f9a2be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a2c0ce990 0x7f9a2c15fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.802+0000 7f9a2be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a2c160050 0x7f9a2c158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.802+0000 7f9a2be00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c06d430 con 0x7f9a2c0ce990 2024-04-26T20:58:16.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.802+0000 7f9a2be00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c06d2b0 con 0x7f9a2c160050 2024-04-26T20:58:16.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.802+0000 7f9a2be00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9a2c06c890 con 0x7f9a10003f80 2024-04-26T20:58:16.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.802+0000 7f9a23e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a2c0ce990 0x7f9a2c15fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.802+0000 7f9a23e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a2c0ce990 0x7f9a2c15fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52122/0 (socket says 172.21.15.28:52122) 2024-04-26T20:58:16.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.802+0000 7f9a23e00640 1 -- 172.21.15.28:0/2970859080 learned_addr learned my addr 172.21.15.28:0/2970859080 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:16.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.803+0000 7f9a23e00640 1 -- 172.21.15.28:0/2970859080 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9a10003f80 msgr2=0x7f9a2c0cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:16.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.803+0000 7f9a23e00640 1 --2- 172.21.15.28:0/2970859080 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9a10003f80 0x7f9a2c0cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.803+0000 7f9a23e00640 1 -- 172.21.15.28:0/2970859080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a2c160050 msgr2=0x7f9a2c158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:16.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.803+0000 7f9a23e00640 1 --2- 172.21.15.28:0/2970859080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a2c160050 0x7f9a2c158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:16.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.803+0000 7f9a23e00640 1 -- 172.21.15.28:0/2970859080 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2c0cdbf0 con 0x7f9a2c0ce990 2024-04-26T20:58:16.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.803+0000 7f9a23e00640 1 --2- 172.21.15.28:0/2970859080 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a2c0ce990 0x7f9a2c15fb10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9a1402abe0 tx=0x7f9a14004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:16.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.803+0000 7f9a22a00640 1 -- 172.21.15.28:0/2970859080 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14040020 con 0x7f9a2c0ce990 2024-04-26T20:58:16.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.803+0000 7f9a22a00640 1 -- 172.21.15.28:0/2970859080 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a140038d0 con 0x7f9a2c0ce990 2024-04-26T20:58:16.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.803+0000 7f9a22a00640 1 -- 172.21.15.28:0/2970859080 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a14036040 con 0x7f9a2c0ce990 2024-04-26T20:58:16.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.803+0000 7f9a2be00640 1 -- 172.21.15.28:0/2970859080 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2c0cdeb0 con 0x7f9a2c0ce990 2024-04-26T20:58:16.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.803+0000 7f9a2be00640 1 -- 172.21.15.28:0/2970859080 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a2c0cee10 con 0x7f9a2c0ce990 2024-04-26T20:58:16.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.804+0000 7f9a2be00640 1 -- 172.21.15.28:0/2970859080 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a2c160470 con 0x7f9a2c0ce990 2024-04-26T20:58:16.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.806+0000 7f9a22a00640 1 -- 172.21.15.28:0/2970859080 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a14043020 con 0x7f9a2c0ce990 2024-04-26T20:58:16.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.806+0000 7f9a22a00640 1 --2- 172.21.15.28:0/2970859080 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9a0806c8f0 0x7f9a0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:16.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.807+0000 7f9a22a00640 1 -- 172.21.15.28:0/2970859080 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f9a140abf60 con 0x7f9a2c0ce990 2024-04-26T20:58:16.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.807+0000 7f9a2aa00640 1 --2- 172.21.15.28:0/2970859080 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9a0806c8f0 0x7f9a0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:16.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.807+0000 7f9a2aa00640 1 --2- 172.21.15.28:0/2970859080 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9a0806c8f0 0x7f9a0806edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0099d0 tx=0x7f9a1c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:16.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:16.811+0000 7f9a22a00640 1 -- 172.21.15.28:0/2970859080 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a1404a030 con 0x7f9a2c0ce990 2024-04-26T20:58:17.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:17.058+0000 7f9a2be00640 1 -- 172.21.15.28:0/2970859080 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9a2c06a360 con 0x7f9a2c0ce990 2024-04-26T20:58:17.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:17.059+0000 7f9a22a00640 1 -- 172.21.15.28:0/2970859080 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f9a1407a2f0 con 0x7f9a2c0ce990 2024-04-26T20:58:17.060 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:58:17.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:17.061+0000 7f9a2be00640 1 -- 172.21.15.28:0/2970859080 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9a0806c8f0 msgr2=0x7f9a0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:17.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:17.061+0000 7f9a2be00640 1 --2- 172.21.15.28:0/2970859080 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9a0806c8f0 0x7f9a0806edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0099d0 tx=0x7f9a1c008040 comp rx=0 tx=0).stop 2024-04-26T20:58:17.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:17.061+0000 7f9a2be00640 1 -- 172.21.15.28:0/2970859080 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a2c0ce990 msgr2=0x7f9a2c15fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:17.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:17.061+0000 7f9a2be00640 1 --2- 172.21.15.28:0/2970859080 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a2c0ce990 0x7f9a2c15fb10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9a1402abe0 tx=0x7f9a14004730 comp rx=0 tx=0).stop 2024-04-26T20:58:17.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:17.062+0000 7f9a2be00640 1 -- 172.21.15.28:0/2970859080 shutdown_connections 2024-04-26T20:58:17.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:17.062+0000 7f9a2be00640 1 --2- 172.21.15.28:0/2970859080 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9a0806c8f0 0x7f9a0806edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:17.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:17.062+0000 7f9a2be00640 1 --2- 172.21.15.28:0/2970859080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a2c160050 0x7f9a2c158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:17.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:17.062+0000 7f9a2be00640 1 --2- 172.21.15.28:0/2970859080 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a2c0ce990 0x7f9a2c15fb10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:17.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:17.062+0000 7f9a2be00640 1 --2- 172.21.15.28:0/2970859080 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9a10003f80 0x7f9a2c0cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:17.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:17.062+0000 7f9a2be00640 1 -- 172.21.15.28:0/2970859080 >> 172.21.15.28:0/2970859080 conn(0x7f9a2c0d4e30 msgr2=0x7f9a2c0b8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:17.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:17.062+0000 7f9a2be00640 1 -- 172.21.15.28:0/2970859080 shutdown_connections 2024-04-26T20:58:17.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:17.062+0000 7f9a2be00640 1 -- 172.21.15.28:0/2970859080 wait complete. 2024-04-26T20:58:17.082 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:58:17.082 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:58:17.082 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-04-26T20:58:17.082 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T20:58:17.082 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:58:17.082 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:58:17.082 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:58:17.082 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:58:17.082 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:58:17.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:17.083 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-04-26T20:58:17.134 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:58:17.135 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:58:17.135 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:58:17.135 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:58:17.135 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-04-26T20:58:17.135 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:58:17.135 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:58:17.135 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:58:17.135 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-04-26T20:58:17.135 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:58:17.135 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:58:17.135 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-04-26T20:58:17.135 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:58:17.135 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T20:58:17.136 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-04-26T20:58:17.136 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T20:58:17.136 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42539sec preferred_lft 42539sec 2024-04-26T20:58:17.136 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link noprefixroute 2024-04-26T20:58:17.136 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:58:17.136 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:58:17.136 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:58:17.136 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-04-26T20:58:17.136 INFO:teuthology.orchestra.run.smithi028.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:58:17.136 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 4a:2a:43:26:cb:b2 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:58:17.136 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:58:17.136 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:58:17.136 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::608e:e5ff:fecb:cfa2/64 scope link 2024-04-26T20:58:17.136 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:58:17.136 INFO:teuthology.orchestra.run.smithi028.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:58:17.137 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 4a:2a:43:26:cb:b2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:58:17.137 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::482a:43ff:fe26:cbb2/64 scope link 2024-04-26T20:58:17.137 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:58:17.137 INFO:teuthology.orchestra.run.smithi028.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:58:17.137 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 56:33:e8:86:4a:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:58:17.137 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::5433:e8ff:fe86:4afb/64 scope link 2024-04-26T20:58:17.137 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:58:17.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:17.138 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:58:17.217 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-04-26T20:58:17.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:17.223 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-04-26T20:58:17.279 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-04-26T20:58:17.279 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:58:17.279 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T20:58:17.279 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T20:58:17.279 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:58:17.280 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:58:17.280 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-04-26T20:58:17.280 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T20:58:17.280 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T20:58:17.281 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T20:58:17.281 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-04-26T20:58:17.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:58:17.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:58:17.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:17.507 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-04-26T20:58:17.553 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:58:17.553 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T20:58:17.553 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:58:17.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:17.554 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-04-26T20:58:17.630 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-26T20:58:17.630 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:58:17.630 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T20:58:17.680 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:58:17.680 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:58:17.680 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:58:17.734 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2024-04-26T20:58:17.819 DEBUG:teuthology.orchestra.run.smithi028:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:58:17.875 INFO:teuthology.orchestra.run.smithi028.stdout:/proc 2024-04-26T20:58:17.876 INFO:teuthology.orchestra.run.smithi028.stdout:/sys 2024-04-26T20:58:17.876 INFO:teuthology.orchestra.run.smithi028.stdout:/dev 2024-04-26T20:58:17.876 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/security 2024-04-26T20:58:17.876 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/shm 2024-04-26T20:58:17.876 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/pts 2024-04-26T20:58:17.876 INFO:teuthology.orchestra.run.smithi028.stdout:/run 2024-04-26T20:58:17.876 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/cgroup 2024-04-26T20:58:17.876 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/pstore 2024-04-26T20:58:17.876 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/bpf 2024-04-26T20:58:17.876 INFO:teuthology.orchestra.run.smithi028.stdout:/ 2024-04-26T20:58:17.876 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/selinux 2024-04-26T20:58:17.876 INFO:teuthology.orchestra.run.smithi028.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:58:17.876 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/mqueue 2024-04-26T20:58:17.877 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/hugepages 2024-04-26T20:58:17.877 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/debug 2024-04-26T20:58:17.877 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/tracing 2024-04-26T20:58:17.877 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/fuse/connections 2024-04-26T20:58:17.877 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:58:17.877 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/config 2024-04-26T20:58:17.877 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:58:17.877 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:58:17.877 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:58:17.877 INFO:teuthology.orchestra.run.smithi028.stdout:/run/user/1000 2024-04-26T20:58:17.877 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph 2024-04-26T20:58:17.877 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:58:17.877 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:58:17.877 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:58:17.877 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:58:17.878 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns 2024-04-26T20:58:17.878 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:58:17.878 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:58:17.878 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:58:17.878 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:58:17.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:17.879 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2024-04-26T20:58:17.928 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T20:58:17.928 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-26T20:58:17.933 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2024-04-26T20:58:17.999 DEBUG:teuthology.orchestra.run.smithi028:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:58:18.043 INFO:teuthology.orchestra.run.smithi028.stdout:/proc 2024-04-26T20:58:18.043 INFO:teuthology.orchestra.run.smithi028.stdout:/sys 2024-04-26T20:58:18.043 INFO:teuthology.orchestra.run.smithi028.stdout:/dev 2024-04-26T20:58:18.043 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/security 2024-04-26T20:58:18.043 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/shm 2024-04-26T20:58:18.044 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/pts 2024-04-26T20:58:18.044 INFO:teuthology.orchestra.run.smithi028.stdout:/run 2024-04-26T20:58:18.044 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/cgroup 2024-04-26T20:58:18.044 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/pstore 2024-04-26T20:58:18.044 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/bpf 2024-04-26T20:58:18.044 INFO:teuthology.orchestra.run.smithi028.stdout:/ 2024-04-26T20:58:18.044 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/selinux 2024-04-26T20:58:18.044 INFO:teuthology.orchestra.run.smithi028.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:58:18.044 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/mqueue 2024-04-26T20:58:18.044 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/hugepages 2024-04-26T20:58:18.044 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/debug 2024-04-26T20:58:18.044 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/tracing 2024-04-26T20:58:18.044 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/fuse/connections 2024-04-26T20:58:18.044 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:58:18.044 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/config 2024-04-26T20:58:18.045 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:58:18.045 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:58:18.045 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:58:18.045 INFO:teuthology.orchestra.run.smithi028.stdout:/run/user/1000 2024-04-26T20:58:18.045 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph 2024-04-26T20:58:18.045 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:58:18.045 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:58:18.045 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:58:18.045 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:58:18.045 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns 2024-04-26T20:58:18.045 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:58:18.045 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:58:18.045 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:58:18.046 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:58:18.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:18.047 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2024-04-26T20:58:18.107 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:2024-04-26T20:58:18.106+0000 7f6a0280b480 -1 init, newargv = 0x55e5191bdca0 newargc=15 2024-04-26T20:58:18.107 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:ceph-fuse[45804]: starting ceph client 2024-04-26T20:58:18.117 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:ceph-fuse[45804]: starting fuse 2024-04-26T20:58:18.131 INFO:teuthology.orchestra.run.smithi028.stdout:41 2024-04-26T20:58:18.132 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-26T20:58:18.132 DEBUG:teuthology.orchestra.run.smithi028:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:58:18.132 DEBUG:teuthology.orchestra.run.smithi028:> import glob 2024-04-26T20:58:18.132 DEBUG:teuthology.orchestra.run.smithi028:> import re 2024-04-26T20:58:18.132 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:58:18.132 DEBUG:teuthology.orchestra.run.smithi028:> import subprocess 2024-04-26T20:58:18.132 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:58:18.132 DEBUG:teuthology.orchestra.run.smithi028:> def _find_admin_socket(client_name): 2024-04-26T20:58:18.132 DEBUG:teuthology.orchestra.run.smithi028:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-26T20:58:18.132 DEBUG:teuthology.orchestra.run.smithi028:> files = glob.glob(asok_path) 2024-04-26T20:58:18.132 DEBUG:teuthology.orchestra.run.smithi028:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-26T20:58:18.132 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:58:18.133 DEBUG:teuthology.orchestra.run.smithi028:> # Given a non-glob path, it better be there 2024-04-26T20:58:18.133 DEBUG:teuthology.orchestra.run.smithi028:> if "*" not in asok_path: 2024-04-26T20:58:18.133 DEBUG:teuthology.orchestra.run.smithi028:> assert(len(files) == 1) 2024-04-26T20:58:18.133 DEBUG:teuthology.orchestra.run.smithi028:> return files[0] 2024-04-26T20:58:18.133 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:58:18.133 DEBUG:teuthology.orchestra.run.smithi028:> for f in files: 2024-04-26T20:58:18.133 DEBUG:teuthology.orchestra.run.smithi028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:58:18.133 DEBUG:teuthology.orchestra.run.smithi028:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:58:18.133 DEBUG:teuthology.orchestra.run.smithi028:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:58:18.133 DEBUG:teuthology.orchestra.run.smithi028:> contents = proc_f.read() 2024-04-26T20:58:18.133 DEBUG:teuthology.orchestra.run.smithi028:> if mountpoint in contents: 2024-04-26T20:58:18.133 DEBUG:teuthology.orchestra.run.smithi028:> return f 2024-04-26T20:58:18.133 DEBUG:teuthology.orchestra.run.smithi028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:58:18.133 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:58:18.133 DEBUG:teuthology.orchestra.run.smithi028:> print(_find_admin_socket("client.0")) 2024-04-26T20:58:18.133 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:58:18.325 INFO:teuthology.orchestra.run.smithi028.stdout:/var/run/ceph/ceph-client.0.45804.asok 2024-04-26T20:58:18.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:58:18.343 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45804.asok 2024-04-26T20:58:18.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:18.343 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45804.asok status 2024-04-26T20:58:18.581 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T20:58:18.581 INFO:teuthology.orchestra.run.smithi028.stdout: "metadata": { 2024-04-26T20:58:18.581 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:58:18.581 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:58:18.581 INFO:teuthology.orchestra.run.smithi028.stdout: "entity_id": "0", 2024-04-26T20:58:18.581 INFO:teuthology.orchestra.run.smithi028.stdout: "hostname": "smithi028", 2024-04-26T20:58:18.582 INFO:teuthology.orchestra.run.smithi028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-26T20:58:18.582 INFO:teuthology.orchestra.run.smithi028.stdout: "pid": "45804", 2024-04-26T20:58:18.582 INFO:teuthology.orchestra.run.smithi028.stdout: "root": "/" 2024-04-26T20:58:18.582 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:18.582 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_count": 0, 2024-04-26T20:58:18.582 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_pinned_count": 0, 2024-04-26T20:58:18.582 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 5376, 2024-04-26T20:58:18.582 INFO:teuthology.orchestra.run.smithi028.stdout: "inst": { 2024-04-26T20:58:18.582 INFO:teuthology.orchestra.run.smithi028.stdout: "name": { 2024-04-26T20:58:18.582 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "client", 2024-04-26T20:58:18.582 INFO:teuthology.orchestra.run.smithi028.stdout: "num": 5376 2024-04-26T20:58:18.582 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:18.582 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2024-04-26T20:58:18.582 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T20:58:18.583 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:58:18.583 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3918289377 2024-04-26T20:58:18.583 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:18.583 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:18.583 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2024-04-26T20:58:18.583 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T20:58:18.583 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:58:18.583 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3918289377 2024-04-26T20:58:18.583 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:18.583 INFO:teuthology.orchestra.run.smithi028.stdout: "inst_str": "client.5376 192.168.0.1:0/3918289377", 2024-04-26T20:58:18.583 INFO:teuthology.orchestra.run.smithi028.stdout: "addr_str": "192.168.0.1:0/3918289377", 2024-04-26T20:58:18.583 INFO:teuthology.orchestra.run.smithi028.stdout: "inode_count": 1, 2024-04-26T20:58:18.583 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_epoch": 17, 2024-04-26T20:58:18.583 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch": 34, 2024-04-26T20:58:18.583 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:58:18.584 INFO:teuthology.orchestra.run.smithi028.stdout: "blocklisted": false, 2024-04-26T20:58:18.584 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs" 2024-04-26T20:58:18.584 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T20:58:18.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:18.596 DEBUG:teuthology.orchestra.run.smithi028:> stat --file-system '--printf=%T 2024-04-26T20:58:18.596 DEBUG:teuthology.orchestra.run.smithi028:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-26T20:58:18.649 INFO:teuthology.orchestra.run.smithi028.stdout:fuseblk 2024-04-26T20:58:18.650 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-26T20:58:18.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:18.650 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:58:18.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:18.737 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent) 2024-04-26T20:58:18.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:18.809 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/child) 2024-04-26T20:58:18.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:18.874 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent/child/file) 2024-04-26T20:58:18.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:18.938 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/child/grandchild) 2024-04-26T20:58:19.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:19.004 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent/child/grandchild/file) 2024-04-26T20:58:19.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:19.068 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-26T20:58:19.167 INFO:teuthology.orchestra.run.smithi028.stdout:./ 2024-04-26T20:58:19.167 INFO:teuthology.orchestra.run.smithi028.stdout:./parent 2024-04-26T20:58:19.167 INFO:teuthology.orchestra.run.smithi028.stdout:./parent/child 2024-04-26T20:58:19.167 INFO:teuthology.orchestra.run.smithi028.stdout:./parent/child/grandchild 2024-04-26T20:58:19.167 INFO:teuthology.orchestra.run.smithi028.stdout:./parent/child/grandchild/file 2024-04-26T20:58:19.167 INFO:teuthology.orchestra.run.smithi028.stdout:./parent/child/file 2024-04-26T20:58:19.169 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:58:19.169 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:58:19.169 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:58:19.169 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:58:19.169 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-26T20:58:19.169 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:58:19.311 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-04-26T20:58:19.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:58:19.325 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:58:19.325 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:58:19.325 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:58:19.325 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:58:19.325 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-26T20:58:19.325 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:58:19.521 INFO:teuthology.orchestra.run.smithi028.stdout:1099511627776 2024-04-26T20:58:19.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:58:19.536 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:58:19.536 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:58:19.536 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:58:19.536 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:58:19.536 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./parent/child").st_ino) 2024-04-26T20:58:19.536 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:58:19.718 INFO:teuthology.orchestra.run.smithi028.stdout:1099511627777 2024-04-26T20:58:19.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:58:19.733 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:58:19.733 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:58:19.733 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:58:19.733 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:58:19.733 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./parent/child/grandchild").st_ino) 2024-04-26T20:58:19.733 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:58:19.929 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628279 2024-04-26T20:58:19.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:58:19.943 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:58:19.944 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:58:19.944 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:58:19.944 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:58:19.944 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./parent/child/grandchild/file").st_ino) 2024-04-26T20:58:19.944 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:58:20.140 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628280 2024-04-26T20:58:20.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:58:20.155 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:58:20.155 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:58:20.155 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:58:20.155 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:58:20.155 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./parent/child/file").st_ino) 2024-04-26T20:58:20.155 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:58:20.350 INFO:teuthology.orchestra.run.smithi028.stdout:1099511627778 2024-04-26T20:58:20.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:58:20.366 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:58:20.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.625+0000 7f3276800640 1 Processor -- start 2024-04-26T20:58:20.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.625+0000 7f3276800640 1 -- start start 2024-04-26T20:58:20.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.626+0000 7f3276800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3270146670 0x7f3270071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:20.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.626+0000 7f3276800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f32701610c0 0x7f32700cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:20.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.626+0000 7f3276800640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3270161530 con 0x7f3270146060 2024-04-26T20:58:20.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.626+0000 7f3276800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f327006f510 con 0x7f32701610c0 2024-04-26T20:58:20.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.626+0000 7f3276800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f3270146440 con 0x7f3270146670 2024-04-26T20:58:20.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.626+0000 7f3275e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f32701610c0 0x7f32700cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:20.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.626+0000 7f3275e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f32701610c0 0x7f32700cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36278/0 (socket says 172.21.15.28:36278) 2024-04-26T20:58:20.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.626+0000 7f3275e00640 1 -- 172.21.15.28:0/3125444059 learned_addr learned my addr 172.21.15.28:0/3125444059 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:20.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.626+0000 7f3274a00640 1 --2- 172.21.15.28:0/3125444059 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3270146670 0x7f3270071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:20.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.627+0000 7f3275e00640 1 -- 172.21.15.28:0/3125444059 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3270146670 msgr2=0x7f3270071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:20.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.627+0000 7f3275e00640 1 --2- 172.21.15.28:0/3125444059 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3270146670 0x7f3270071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:20.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.627+0000 7f3275e00640 1 -- 172.21.15.28:0/3125444059 >> v1:172.21.15.103:6789/0 conn(0x7f3270146060 legacy=0x7f32701609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:20.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.627+0000 7f3275e00640 1 -- 172.21.15.28:0/3125444059 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327006f010 con 0x7f32701610c0 2024-04-26T20:58:20.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.628+0000 7f3275e00640 1 --2- 172.21.15.28:0/3125444059 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32701610c0 0x7f32700cb0a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3268009870 tx=0x7f326802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942446466274c761 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.628+0000 7f326fe00640 1 -- 172.21.15.28:0/3125444059 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268037070 con 0x7f32701610c0 2024-04-26T20:58:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.628+0000 7f326fe00640 1 -- 172.21.15.28:0/3125444059 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3268032440 con 0x7f32701610c0 2024-04-26T20:58:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.628+0000 7f326fe00640 1 -- 172.21.15.28:0/3125444059 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268032810 con 0x7f32701610c0 2024-04-26T20:58:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.629+0000 7f3276800640 1 -- 172.21.15.28:0/3125444059 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32701610c0 msgr2=0x7f32700cb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.629+0000 7f3276800640 1 --2- 172.21.15.28:0/3125444059 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32701610c0 0x7f32700cb0a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3268009870 tx=0x7f326802a180 comp rx=0 tx=0).stop 2024-04-26T20:58:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.629+0000 7f3276800640 1 -- 172.21.15.28:0/3125444059 shutdown_connections 2024-04-26T20:58:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.629+0000 7f3276800640 1 --2- 172.21.15.28:0/3125444059 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32701610c0 0x7f32700cb0a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.629+0000 7f3276800640 1 --2- 172.21.15.28:0/3125444059 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3270146670 0x7f3270071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.629+0000 7f3276800640 1 -- 172.21.15.28:0/3125444059 >> 172.21.15.28:0/3125444059 conn(0x7f32700d0a50 msgr2=0x7f32700d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:20.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.629+0000 7f3276800640 1 -- 172.21.15.28:0/3125444059 shutdown_connections 2024-04-26T20:58:20.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.629+0000 7f3276800640 1 -- 172.21.15.28:0/3125444059 wait complete. 2024-04-26T20:58:20.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.631+0000 7f3276800640 1 Processor -- start 2024-04-26T20:58:20.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.631+0000 7f3276800640 1 -- start start 2024-04-26T20:58:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.632+0000 7f3276800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3270146060 0x7f327015de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.632+0000 7f3276800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3270146670 0x7f32700aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.632+0000 7f3276800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32701610c0 0x7f32700aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.632+0000 7f3276800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f327006ff60 con 0x7f32701610c0 2024-04-26T20:58:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.632+0000 7f3276800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f327006fc80 con 0x7f3270146060 2024-04-26T20:58:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.632+0000 7f3276800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f32700cb5e0 con 0x7f3270146670 2024-04-26T20:58:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.632+0000 7f3275e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32701610c0 0x7f32700aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:20.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.632+0000 7f3275e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32701610c0 0x7f32700aa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36292/0 (socket says 172.21.15.28:36292) 2024-04-26T20:58:20.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.632+0000 7f3275e00640 1 -- 172.21.15.28:0/2006119859 learned_addr learned my addr 172.21.15.28:0/2006119859 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:20.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.632+0000 7f3275400640 1 --2- 172.21.15.28:0/2006119859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3270146060 0x7f327015de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:20.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.632+0000 7f3274a00640 1 --2- 172.21.15.28:0/2006119859 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3270146670 0x7f32700aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:20.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.633+0000 7f3275e00640 1 -- 172.21.15.28:0/2006119859 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3270146670 msgr2=0x7f32700aa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:20.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.633+0000 7f3275e00640 1 --2- 172.21.15.28:0/2006119859 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3270146670 0x7f32700aa4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:20.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.633+0000 7f3275e00640 1 -- 172.21.15.28:0/2006119859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3270146060 msgr2=0x7f327015de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:20.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.633+0000 7f3275e00640 1 --2- 172.21.15.28:0/2006119859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3270146060 0x7f327015de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:20.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.633+0000 7f3275e00640 1 -- 172.21.15.28:0/2006119859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32701433f0 con 0x7f32701610c0 2024-04-26T20:58:20.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.633+0000 7f3275400640 1 --2- 172.21.15.28:0/2006119859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3270146060 0x7f327015de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:20.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.633+0000 7f3275e00640 1 --2- 172.21.15.28:0/2006119859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32701610c0 0x7f32700aa9f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f326802a1c0 tx=0x7f32680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:20.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.633+0000 7f326d600640 1 -- 172.21.15.28:0/2006119859 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268041020 con 0x7f32701610c0 2024-04-26T20:58:20.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.633+0000 7f326d600640 1 -- 172.21.15.28:0/2006119859 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32680306c0 con 0x7f32701610c0 2024-04-26T20:58:20.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.633+0000 7f326d600640 1 -- 172.21.15.28:0/2006119859 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3268037040 con 0x7f32701610c0 2024-04-26T20:58:20.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.633+0000 7f3276800640 1 -- 172.21.15.28:0/2006119859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32701436d0 con 0x7f32701610c0 2024-04-26T20:58:20.636 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.634+0000 7f3276800640 1 -- 172.21.15.28:0/2006119859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f327015e380 con 0x7f32701610c0 2024-04-26T20:58:20.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.635+0000 7f3276800640 1 -- 172.21.15.28:0/2006119859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f327006a360 con 0x7f32701610c0 2024-04-26T20:58:20.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.636+0000 7f326d600640 1 -- 172.21.15.28:0/2006119859 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3268044020 con 0x7f32701610c0 2024-04-26T20:58:20.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.638+0000 7f326d600640 1 --2- 172.21.15.28:0/2006119859 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f325006c8f0 0x7f325006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:20.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.638+0000 7f326d600640 1 -- 172.21.15.28:0/2006119859 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f32680ac340 con 0x7f32701610c0 2024-04-26T20:58:20.638 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.638+0000 7f3275400640 1 --2- 172.21.15.28:0/2006119859 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f325006c8f0 0x7f325006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:20.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.638+0000 7f3275400640 1 --2- 172.21.15.28:0/2006119859 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f325006c8f0 0x7f325006edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3264004520 tx=0x7f3264009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:20.642 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.643+0000 7f326d600640 1 -- 172.21.15.28:0/2006119859 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f326804b030 con 0x7f32701610c0 2024-04-26T20:58:20.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.891+0000 7f3276800640 1 -- 172.21.15.28:0/2006119859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3270072cf0 con 0x7f32701610c0 2024-04-26T20:58:20.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.893+0000 7f326d600640 1 -- 172.21.15.28:0/2006119859 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7f326807a760 con 0x7f32701610c0 2024-04-26T20:58:20.894 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:20.894 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/1364678224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1364678224},{"type":"v1","addr":"172.21.15.103:6834","nonce":1364678224}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4081926466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":4081926466},{"type":"v1","addr":"172.21.15.28:6837","nonce":4081926466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5175,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/1370252546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":1370252546},{"type":"v1","addr":"172.21.15.28:6835","nonce":1370252546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:58:09.226353+0000","modified":"2024-04-26T20:58:12.319110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5184":{"gid":5184,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6835/1781724684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":1781724684},{"type":"v1","addr":"172.21.15.103:6835","nonce":1781724684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5184,"qdb_cluster":[5184]},"id":2}]} 2024-04-26T20:58:20.894 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-04-26T20:58:20.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.896+0000 7f3276800640 1 -- 172.21.15.28:0/2006119859 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f325006c8f0 msgr2=0x7f325006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:20.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.896+0000 7f3276800640 1 --2- 172.21.15.28:0/2006119859 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f325006c8f0 0x7f325006edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3264004520 tx=0x7f3264009040 comp rx=0 tx=0).stop 2024-04-26T20:58:20.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.896+0000 7f3276800640 1 -- 172.21.15.28:0/2006119859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32701610c0 msgr2=0x7f32700aa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:20.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.896+0000 7f3276800640 1 --2- 172.21.15.28:0/2006119859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32701610c0 0x7f32700aa9f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f326802a1c0 tx=0x7f32680046c0 comp rx=0 tx=0).stop 2024-04-26T20:58:20.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.896+0000 7f3276800640 1 -- 172.21.15.28:0/2006119859 shutdown_connections 2024-04-26T20:58:20.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.896+0000 7f3276800640 1 --2- 172.21.15.28:0/2006119859 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f325006c8f0 0x7f325006edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:20.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.896+0000 7f3276800640 1 --2- 172.21.15.28:0/2006119859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32701610c0 0x7f32700aa9f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:20.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.896+0000 7f3276800640 1 --2- 172.21.15.28:0/2006119859 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3270146670 0x7f32700aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:20.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.896+0000 7f3276800640 1 --2- 172.21.15.28:0/2006119859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3270146060 0x7f327015de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:20.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.896+0000 7f3276800640 1 -- 172.21.15.28:0/2006119859 >> 172.21.15.28:0/2006119859 conn(0x7f32700d0a50 msgr2=0x7f327006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:20.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.896+0000 7f3276800640 1 -- 172.21.15.28:0/2006119859 shutdown_connections 2024-04-26T20:58:20.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:20.896+0000 7f3276800640 1 -- 172.21.15.28:0/2006119859 wait complete. 2024-04-26T20:58:20.918 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json dump tree /parent/child 1 2024-04-26T20:58:21.145 INFO:teuthology.orchestra.run.smithi103.stdout:[{"ino":1099511628279,"rdev":0,"ctime":"2024-04-26T20:58:19.063054+0000","btime":"2024-04-26T20:58:19.000120+0000","mode":16877,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T20:58:19.063054+0000","atime":"2024-04-26T20:58:19.000120+0000","time_warp_seq":0,"change_attr":1,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-26T20:58:19.063054+0000","num_files":1,"num_subdirs":0,"change_attr":1},"rstat":{"version":0,"rbytes":0,"rfiles":1,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T20:58:19.063054+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":1,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T20:58:19.063054+0000"},"version":6,"file_data_version":0,"xattr_version":1,"backtrace_version":4,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1,"waiter":0,"authpin":0},"nref":4,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","type":"ifile","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":5376,"pending":"pAsLsXsFsx","issued":"pAsLsXsFsx","wanted":"AsLsXsFsx","last_sent":8}],"loner":5376,"want_loner":5376,"mds_caps_wanted":[],"dirfrags":[{"path":"/parent/child/grandchild","dirfrag":"0x100000001f7","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"dirty":1,"authpin":0},"nref":2,"dentries":[{"path":"parent/child/grandchild/file","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511628280,"remote_type":"","version":2,"projected_version":2,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false},{"ino":1099511627778,"rdev":0,"ctime":"2024-04-26T20:58:18.932451+0000","btime":"2024-04-26T20:58:18.932451+0000","mode":33188,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":0,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":5,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T20:58:18.932451+0000","atime":"2024-04-26T20:58:18.932451+0000","time_warp_seq":0,"change_attr":0,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[{"client":5376,"byte range":{"first":0,"last":4194304},"follows":1}],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":1,"rsubdirs":0,"rsnaps":0,"rctime":"0.000000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":1,"rsubdirs":0,"rsnaps":0,"rctime":"0.000000"},"version":2,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"caps":1,"dirtyparent":1,"dirty":1},"nref":3,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"excl","type":"iauth","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","type":"ifile","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{"gather_set":[],"state":"excl","type":"ixattr","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":5376,"pending":"pAsxLsXsxFsxcrwb","issued":"pAsxLsXsxFsxcrwb","wanted":"pAsxXsxFxwb","last_sent":3}],"loner":5376,"want_loner":5376,"mds_caps_wanted":[],"dirfrags":[],"is_subvolume":false},{"ino":1099511627777,"rdev":0,"ctime":"2024-04-26T20:58:19.000120+0000","btime":"2024-04-26T20:58:18.868780+0000","mode":16877,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T20:58:19.000120+0000","atime":"2024-04-26T20:58:18.868780+0000","time_warp_seq":0,"change_attr":2,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-26T20:58:19.000120+0000","num_files":1,"num_subdirs":1,"change_attr":2},"rstat":{"version":0,"rbytes":0,"rfiles":1,"rsubdirs":2,"rsnaps":0,"rctime":"2024-04-26T20:58:19.000120+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":1,"rsubdirs":2,"rsnaps":0,"rctime":"2024-04-26T20:58:19.000120+0000"},"version":6,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirtyscattered":1,"request":0,"lock":0,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1,"waiter":0,"authpin":0},"nref":5,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","type":"ifile","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":5376,"pending":"pAsLsXsFsx","issued":"pAsLsXsFsx","wanted":"AsLsXsFsx","last_sent":10}],"loner":5376,"want_loner":5376,"mds_caps_wanted":[],"dirfrags":[{"path":"/parent/child","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"7","version":"7","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"dirty":1,"authpin":0},"nref":2,"dentries":[{"path":"parent/child/grandchild","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511628279,"remote_type":"","version":6,"projected_version":6,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"parent/child/file","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511627778,"remote_type":"","version":2,"projected_version":2,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false}] 2024-04-26T20:58:21.147 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 0, "rctime": "2024-04-26T20:58:19.063054+0000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2024-04-26T20:58:19.000120+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 4, "btime": "2024-04-26T20:58:19.000120+0000", "change_attr": 1, "client_caps": [ { "client_id": 5376, "issued": "pAsLsXsFsx", "last_sent": 8, "pending": "pAsLsXsFsx", "wanted": "AsLsXsFsx" } ], "client_ranges": [], "ctime": "2024-04-26T20:58:19.063054+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dentries": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511628280, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "parent/child/grandchild/file", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 2, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 2, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x100000001f7", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/parent/child/grandchild", "pins": { "authpin": 0, "child": 1, "dirty": 1 }, "projected_version": "3", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "3" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 1, "mtime": "2024-04-26T20:58:19.063054+0000", "num_files": 1, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "type": "ifile", "xlock_by": {} }, "flocklock": {}, "gid": 1316, "ino": 1099511628279, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 5376, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-04-26T20:58:19.063054+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "lock": 0, "request": 0, "waiter": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2024-04-26T20:58:19.063054+0000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 6, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": 5376, "xattr_version": 1, "xattrlock": {}, "xattrs": [] }, { "accounted_rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 0, "version": 0 }, "atime": "2024-04-26T20:58:18.932451+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "type": "iauth", "xlock_by": {} }, "backtrace_version": 2, "btime": "2024-04-26T20:58:18.932451+0000", "change_attr": 0, "client_caps": [ { "client_id": 5376, "issued": "pAsxLsXsxFsxcrwb", "last_sent": 3, "pending": "pAsxLsXsxFsxcrwb", "wanted": "pAsxXsxFxwb" } ], "client_ranges": [ { "byte range": { "first": 0, "last": 4194304 }, "client": 5376, "follows": 1 } ], "ctime": "2024-04-26T20:58:18.932451+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 0, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "type": "ifile", "xlock_by": {} }, "flocklock": {}, "gid": 1316, "ino": 1099511627778, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 5, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": 5376, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 33188, "mtime": "2024-04-26T20:58:18.932451+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 3, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "caps": 1, "dirty": 1, "dirtyparent": 1, "request": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 0, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 2, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": 5376, "xattr_version": 1, "xattrlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "type": "ixattr", "xlock_by": {} }, "xattrs": [] }, { "accounted_rstat": { "rbytes": 0, "rctime": "2024-04-26T20:58:19.000120+0000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "atime": "2024-04-26T20:58:18.868780+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-26T20:58:18.868780+0000", "change_attr": 2, "client_caps": [ { "client_id": 5376, "issued": "pAsLsXsFsx", "last_sent": 10, "pending": "pAsLsXsFsx", "wanted": "AsLsXsFsx" } ], "client_ranges": [], "ctime": "2024-04-26T20:58:19.000120+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dentries": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511628279, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "parent/child/grandchild", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 6, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 6, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511627778, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "parent/child/file", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 2, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 2, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/parent/child", "pins": { "authpin": 0, "child": 1, "dirty": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 2, "mtime": "2024-04-26T20:58:19.000120+0000", "num_files": 1, "num_subdirs": 1, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "type": "ifile", "xlock_by": {} }, "flocklock": {}, "gid": 1316, "ino": 1099511627777, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 5376, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-04-26T20:58:19.000120+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 5, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "dirtyscattered": 1, "lock": 0, "request": 0, "waiter": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2024-04-26T20:58:19.000120+0000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 2, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 6, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": 5376, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } ] 2024-04-26T20:58:21.147 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T20:58:21.148 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:58:21.266 DEBUG:teuthology.orchestra.run.smithi103:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:58:21.381 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T20:58:21.381 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:58:21.381 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:58:21.381 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:58:21.445 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:58:21.445 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:58:21.499 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:58:21.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.716+0000 7ff6d8000640 1 Processor -- start 2024-04-26T20:58:21.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.716+0000 7ff6d8000640 1 -- start start 2024-04-26T20:58:21.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.717+0000 7ff6d8000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff6d01469f0 0x7ff6d00b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:21.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.717+0000 7ff6d8000640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff6d006da70 0x7ff6d00ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:21.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.717+0000 7ff6d8000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6d0161190 con 0x7ff6d0160a40 2024-04-26T20:58:21.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.717+0000 7ff6d8000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff6d0161470 con 0x7ff6d01469f0 2024-04-26T20:58:21.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.717+0000 7ff6d8000640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7ff6d0161750 con 0x7ff6d006da70 2024-04-26T20:58:21.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.717+0000 7ff6d5600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff6d01469f0 0x7ff6d00b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:21.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.718+0000 7ff6d5600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff6d01469f0 0x7ff6d00b11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36302/0 (socket says 172.21.15.28:36302) 2024-04-26T20:58:21.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.718+0000 7ff6d4c00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff6d0160a40 0x7ff6d006d360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37018/0 (socket says 172.21.15.28:37018) 2024-04-26T20:58:21.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.718+0000 7ff6d5600640 1 -- 172.21.15.28:0/3400809396 learned_addr learned my addr 172.21.15.28:0/3400809396 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:21.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.718+0000 7ff6d6000640 1 --2- 172.21.15.28:0/3400809396 >> v2:172.21.15.103:3300/0 conn(0x7ff6d006da70 0x7ff6d00ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:21.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.718+0000 7ff6d5600640 1 -- 172.21.15.28:0/3400809396 >> v2:172.21.15.103:3300/0 conn(0x7ff6d006da70 msgr2=0x7ff6d00ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:21.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.718+0000 7ff6d5600640 1 --2- 172.21.15.28:0/3400809396 >> v2:172.21.15.103:3300/0 conn(0x7ff6d006da70 0x7ff6d00ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:21.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.718+0000 7ff6d5600640 1 -- 172.21.15.28:0/3400809396 >> v1:172.21.15.28:6789/0 conn(0x7ff6d0160a40 legacy=0x7ff6d006d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:21.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.718+0000 7ff6cfe00640 1 -- 172.21.15.28:0/3400809396 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2505470442 0 0) 0x7ff6d0161190 con 0x7ff6d0160a40 2024-04-26T20:58:21.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.718+0000 7ff6d5600640 1 -- 172.21.15.28:0/3400809396 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d00cc770 con 0x7ff6d01469f0 2024-04-26T20:58:21.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.718+0000 7ff6d6000640 1 --2- 172.21.15.28:0/3400809396 >> v2:172.21.15.103:3300/0 conn(0x7ff6d006da70 0x7ff6d00ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:21.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.719+0000 7ff6d5600640 1 --2- 172.21.15.28:0/3400809396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6d01469f0 0x7ff6d00b11d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff6d00b0550 tx=0x7ff6c802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331ec2d88499d3d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:21.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.719+0000 7ff6cfe00640 1 -- 172.21.15.28:0/3400809396 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8036070 con 0x7ff6d01469f0 2024-04-26T20:58:21.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.719+0000 7ff6cfe00640 1 -- 172.21.15.28:0/3400809396 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6c8031600 con 0x7ff6d01469f0 2024-04-26T20:58:21.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.719+0000 7ff6cfe00640 1 -- 172.21.15.28:0/3400809396 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c80319d0 con 0x7ff6d01469f0 2024-04-26T20:58:21.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.720+0000 7ff6d8000640 1 -- 172.21.15.28:0/3400809396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6d01469f0 msgr2=0x7ff6d00b11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:21.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.720+0000 7ff6d8000640 1 --2- 172.21.15.28:0/3400809396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6d01469f0 0x7ff6d00b11d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff6d00b0550 tx=0x7ff6c802a6a0 comp rx=0 tx=0).stop 2024-04-26T20:58:21.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.720+0000 7ff6d8000640 1 -- 172.21.15.28:0/3400809396 shutdown_connections 2024-04-26T20:58:21.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.720+0000 7ff6d8000640 1 --2- 172.21.15.28:0/3400809396 >> v2:172.21.15.103:3300/0 conn(0x7ff6d006da70 0x7ff6d00ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:21.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.720+0000 7ff6d8000640 1 --2- 172.21.15.28:0/3400809396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6d01469f0 0x7ff6d00b11d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:21.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.720+0000 7ff6d8000640 1 -- 172.21.15.28:0/3400809396 >> 172.21.15.28:0/3400809396 conn(0x7ff6d0073140 msgr2=0x7ff6d0073550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:21.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.720+0000 7ff6d8000640 1 -- 172.21.15.28:0/3400809396 shutdown_connections 2024-04-26T20:58:21.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.720+0000 7ff6d8000640 1 -- 172.21.15.28:0/3400809396 wait complete. 2024-04-26T20:58:21.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.722+0000 7ff6d8000640 1 Processor -- start 2024-04-26T20:58:21.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.722+0000 7ff6d8000640 1 -- start start 2024-04-26T20:58:21.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.723+0000 7ff6d8000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6d006da70 0x7ff6d0142c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:21.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.723+0000 7ff6d8000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6d01469f0 0x7ff6d00b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:21.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.723+0000 7ff6d8000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6d0160a40 0x7ff6d00cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:21.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.723+0000 7ff6d8000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d006f1a0 con 0x7ff6d006da70 2024-04-26T20:58:21.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.723+0000 7ff6d8000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff6d00cd1d0 con 0x7ff6d01469f0 2024-04-26T20:58:21.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.723+0000 7ff6d8000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff6d00b6d90 con 0x7ff6d0160a40 2024-04-26T20:58:21.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.724+0000 7ff6d5600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6d006da70 0x7ff6d0142c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:21.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.724+0000 7ff6d5600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6d006da70 0x7ff6d0142c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36312/0 (socket says 172.21.15.28:36312) 2024-04-26T20:58:21.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.724+0000 7ff6d5600640 1 -- 172.21.15.28:0/3886949352 learned_addr learned my addr 172.21.15.28:0/3886949352 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:21.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.724+0000 7ff6d4c00640 1 --2- 172.21.15.28:0/3886949352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6d01469f0 0x7ff6d00b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:21.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.724+0000 7ff6d5600640 1 -- 172.21.15.28:0/3886949352 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6d0160a40 msgr2=0x7ff6d00cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:21.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.724+0000 7ff6d5600640 1 --2- 172.21.15.28:0/3886949352 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6d0160a40 0x7ff6d00cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:21.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.724+0000 7ff6d5600640 1 -- 172.21.15.28:0/3886949352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6d01469f0 msgr2=0x7ff6d00b5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:21.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.724+0000 7ff6d5600640 1 --2- 172.21.15.28:0/3886949352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6d01469f0 0x7ff6d00b5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:21.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.724+0000 7ff6d5600640 1 -- 172.21.15.28:0/3886949352 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6d00b64f0 con 0x7ff6d006da70 2024-04-26T20:58:21.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.724+0000 7ff6d5600640 1 --2- 172.21.15.28:0/3886949352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6d006da70 0x7ff6d0142c50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8034040 tx=0x7ff6c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:21.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.725+0000 7ff6cd600640 1 -- 172.21.15.28:0/3886949352 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8036050 con 0x7ff6d006da70 2024-04-26T20:58:21.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.725+0000 7ff6cd600640 1 -- 172.21.15.28:0/3886949352 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6c8042070 con 0x7ff6d006da70 2024-04-26T20:58:21.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.725+0000 7ff6cd600640 1 -- 172.21.15.28:0/3886949352 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8009510 con 0x7ff6d006da70 2024-04-26T20:58:21.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.725+0000 7ff6d8000640 1 -- 172.21.15.28:0/3886949352 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6d00b67d0 con 0x7ff6d006da70 2024-04-26T20:58:21.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.725+0000 7ff6d8000640 1 -- 172.21.15.28:0/3886949352 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6d0143190 con 0x7ff6d006da70 2024-04-26T20:58:21.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.726+0000 7ff6d8000640 1 -- 172.21.15.28:0/3886949352 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6d00b37b0 con 0x7ff6d006da70 2024-04-26T20:58:21.727 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.727+0000 7ff6cd600640 1 -- 172.21.15.28:0/3886949352 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff6c8007c50 con 0x7ff6d006da70 2024-04-26T20:58:21.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.729+0000 7ff6cd600640 1 --2- 172.21.15.28:0/3886949352 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff6b006c8f0 0x7ff6b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:21.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.729+0000 7ff6cd600640 1 -- 172.21.15.28:0/3886949352 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7ff6c80accd0 con 0x7ff6d006da70 2024-04-26T20:58:21.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.729+0000 7ff6d4c00640 1 --2- 172.21.15.28:0/3886949352 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff6b006c8f0 0x7ff6b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:21.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.730+0000 7ff6d4c00640 1 --2- 172.21.15.28:0/3886949352 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff6b006c8f0 0x7ff6b006edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff6d00cc290 tx=0x7ff6c4007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:21.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.734+0000 7ff6cd600640 1 -- 172.21.15.28:0/3886949352 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff6c8045360 con 0x7ff6d006da70 2024-04-26T20:58:21.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.940+0000 7ff6d8000640 1 -- 172.21.15.28:0/3886949352 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6d0073fe0 con 0x7ff6d006da70 2024-04-26T20:58:21.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.942+0000 7ff6cd600640 1 -- 172.21.15.28:0/3886949352 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16645 (secure 0 0 0) 0x7ff6c807afd0 con 0x7ff6d006da70 2024-04-26T20:58:21.942 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:21.943 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":34,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:58:10.759775+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:58:07.143890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:58:08.193405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:58:21.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.944+0000 7ff6d8000640 1 -- 172.21.15.28:0/3886949352 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff6b006c8f0 msgr2=0x7ff6b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:21.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.944+0000 7ff6d8000640 1 --2- 172.21.15.28:0/3886949352 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff6b006c8f0 0x7ff6b006edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff6d00cc290 tx=0x7ff6c4007c00 comp rx=0 tx=0).stop 2024-04-26T20:58:21.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.944+0000 7ff6d8000640 1 -- 172.21.15.28:0/3886949352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6d006da70 msgr2=0x7ff6d0142c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.944+0000 7ff6d8000640 1 --2- 172.21.15.28:0/3886949352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6d006da70 0x7ff6d0142c50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8034040 tx=0x7ff6c8004a40 comp rx=0 tx=0).stop 2024-04-26T20:58:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.945+0000 7ff6d8000640 1 -- 172.21.15.28:0/3886949352 shutdown_connections 2024-04-26T20:58:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.945+0000 7ff6d8000640 1 --2- 172.21.15.28:0/3886949352 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff6b006c8f0 0x7ff6b006edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.945+0000 7ff6d8000640 1 --2- 172.21.15.28:0/3886949352 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff6d0160a40 0x7ff6d00cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.945+0000 7ff6d8000640 1 --2- 172.21.15.28:0/3886949352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff6d01469f0 0x7ff6d00b5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.945+0000 7ff6d8000640 1 --2- 172.21.15.28:0/3886949352 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6d006da70 0x7ff6d0142c50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.945+0000 7ff6d8000640 1 -- 172.21.15.28:0/3886949352 >> 172.21.15.28:0/3886949352 conn(0x7ff6d0073140 msgr2=0x7ff6d00b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:21.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.945+0000 7ff6d8000640 1 -- 172.21.15.28:0/3886949352 shutdown_connections 2024-04-26T20:58:21.947 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:21.945+0000 7ff6d8000640 1 -- 172.21.15.28:0/3886949352 wait complete. 2024-04-26T20:58:21.966 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:58:07.143890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:58:08.193405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:58:21.966 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:58:22.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.184+0000 7f2656c00640 1 Processor -- start 2024-04-26T20:58:22.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.185+0000 7f2656c00640 1 -- start start 2024-04-26T20:58:22.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.186+0000 7f2656c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2650073f50 0x7f26500ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:22.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.186+0000 7f2656c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f26500ce8d0 0x7f265006e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:22.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.186+0000 7f2656c00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2650074400 con 0x7f265006fc10 2024-04-26T20:58:22.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.186+0000 7f2656c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f26500ced00 con 0x7f2650073f50 2024-04-26T20:58:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.186+0000 7f2656c00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f265015e310 con 0x7f26500ce8d0 2024-04-26T20:58:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.187+0000 7f264f400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2650073f50 0x7f26500ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.187+0000 7f264f400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2650073f50 0x7f26500ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36316/0 (socket says 172.21.15.28:36316) 2024-04-26T20:58:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.187+0000 7f264f400640 1 -- 172.21.15.28:0/2539241069 learned_addr learned my addr 172.21.15.28:0/2539241069 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.187+0000 7f264f400640 1 -- 172.21.15.28:0/2539241069 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f26500ce8d0 msgr2=0x7f265006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.187+0000 7f264f400640 1 --2- 172.21.15.28:0/2539241069 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f26500ce8d0 0x7f265006e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.187+0000 7f264f400640 1 -- 172.21.15.28:0/2539241069 >> v1:172.21.15.103:6789/0 conn(0x7f265006fc10 legacy=0x7f2650073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:22.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.187+0000 7f264f400640 1 -- 172.21.15.28:0/2539241069 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2650072aa0 con 0x7f2650073f50 2024-04-26T20:58:22.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.188+0000 7f264f400640 1 --2- 172.21.15.28:0/2539241069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2650073f50 0x7f26500ce390 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2650070460 tx=0x7f263802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5881f8dc181e282 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:22.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.188+0000 7f264ea00640 1 -- 172.21.15.28:0/2539241069 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638037070 con 0x7f2650073f50 2024-04-26T20:58:22.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.189+0000 7f264ea00640 1 -- 172.21.15.28:0/2539241069 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f263802ae50 con 0x7f2650073f50 2024-04-26T20:58:22.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.189+0000 7f264ea00640 1 -- 172.21.15.28:0/2539241069 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638032660 con 0x7f2650073f50 2024-04-26T20:58:22.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.189+0000 7f2656c00640 1 -- 172.21.15.28:0/2539241069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2650073f50 msgr2=0x7f26500ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:22.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.189+0000 7f2656c00640 1 --2- 172.21.15.28:0/2539241069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2650073f50 0x7f26500ce390 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2650070460 tx=0x7f263802a140 comp rx=0 tx=0).stop 2024-04-26T20:58:22.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.189+0000 7f2656c00640 1 -- 172.21.15.28:0/2539241069 shutdown_connections 2024-04-26T20:58:22.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.189+0000 7f2656c00640 1 --2- 172.21.15.28:0/2539241069 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f26500ce8d0 0x7f265006e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:22.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.189+0000 7f2656c00640 1 --2- 172.21.15.28:0/2539241069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2650073f50 0x7f26500ce390 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:22.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.189+0000 7f2656c00640 1 -- 172.21.15.28:0/2539241069 >> 172.21.15.28:0/2539241069 conn(0x7f265006c770 msgr2=0x7f265006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:22.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.189+0000 7f2656c00640 1 -- 172.21.15.28:0/2539241069 shutdown_connections 2024-04-26T20:58:22.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.189+0000 7f2656c00640 1 -- 172.21.15.28:0/2539241069 wait complete. 2024-04-26T20:58:22.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.191+0000 7f2656c00640 1 Processor -- start 2024-04-26T20:58:22.191 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.191+0000 7f2656c00640 1 -- start start 2024-04-26T20:58:22.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.192+0000 7f2656c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f265006fc10 0x7f2650150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:22.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.192+0000 7f2656c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2650073f50 0x7f26501f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:22.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.192+0000 7f2656c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26500ce8d0 0x7f26501583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:22.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.192+0000 7f2656c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f265015dfa0 con 0x7f26500ce8d0 2024-04-26T20:58:22.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.192+0000 7f2656c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2650073060 con 0x7f2650073f50 2024-04-26T20:58:22.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.192+0000 7f2656c00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f26500d07d0 con 0x7f265006fc10 2024-04-26T20:58:22.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.193+0000 7f2654c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26500ce8d0 0x7f26501583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:22.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.193+0000 7f2654c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26500ce8d0 0x7f26501583e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36330/0 (socket says 172.21.15.28:36330) 2024-04-26T20:58:22.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.193+0000 7f2654c00640 1 -- 172.21.15.28:0/1634121370 learned_addr learned my addr 172.21.15.28:0/1634121370 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:22.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.193+0000 7f264f400640 1 --2- 172.21.15.28:0/1634121370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2650073f50 0x7f26501f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:22.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.193+0000 7f264fe00640 1 --2- 172.21.15.28:0/1634121370 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f265006fc10 0x7f2650150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.193+0000 7f2654c00640 1 -- 172.21.15.28:0/1634121370 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f265006fc10 msgr2=0x7f2650150b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.193+0000 7f2654c00640 1 --2- 172.21.15.28:0/1634121370 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f265006fc10 0x7f2650150b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.193+0000 7f2654c00640 1 -- 172.21.15.28:0/1634121370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2650073f50 msgr2=0x7f26501f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.193+0000 7f2654c00640 1 --2- 172.21.15.28:0/1634121370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2650073f50 0x7f26501f65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.193+0000 7f2654c00640 1 -- 172.21.15.28:0/1634121370 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26501510b0 con 0x7f26500ce8d0 2024-04-26T20:58:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.194+0000 7f2654c00640 1 --2- 172.21.15.28:0/1634121370 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26500ce8d0 0x7f26501583e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f264800b450 tx=0x7f264800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.194+0000 7f2647e00640 1 -- 172.21.15.28:0/1634121370 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648015020 con 0x7f26500ce8d0 2024-04-26T20:58:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.194+0000 7f2647e00640 1 -- 172.21.15.28:0/1634121370 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2648011890 con 0x7f26500ce8d0 2024-04-26T20:58:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.194+0000 7f2656c00640 1 -- 172.21.15.28:0/1634121370 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26501f6e20 con 0x7f26500ce8d0 2024-04-26T20:58:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.194+0000 7f2647e00640 1 -- 172.21.15.28:0/1634121370 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2648011c60 con 0x7f26500ce8d0 2024-04-26T20:58:22.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.194+0000 7f2656c00640 1 -- 172.21.15.28:0/1634121370 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26501f73e0 con 0x7f26500ce8d0 2024-04-26T20:58:22.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.195+0000 7f2656c00640 1 -- 172.21.15.28:0/1634121370 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f265006a360 con 0x7f26500ce8d0 2024-04-26T20:58:22.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.196+0000 7f2647e00640 1 -- 172.21.15.28:0/1634121370 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2648018020 con 0x7f26500ce8d0 2024-04-26T20:58:22.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.197+0000 7f2647e00640 1 --2- 172.21.15.28:0/1634121370 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f262c06c8f0 0x7f262c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:22.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.197+0000 7f2647e00640 1 -- 172.21.15.28:0/1634121370 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f264808bd40 con 0x7f26500ce8d0 2024-04-26T20:58:22.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.197+0000 7f264fe00640 1 --2- 172.21.15.28:0/1634121370 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f262c06c8f0 0x7f262c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:22.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.198+0000 7f264fe00640 1 --2- 172.21.15.28:0/1634121370 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f262c06c8f0 0x7f262c06edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2640009f20 tx=0x7f26400094d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:22.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.201+0000 7f2647e00640 1 -- 172.21.15.28:0/1634121370 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f264801f020 con 0x7f26500ce8d0 2024-04-26T20:58:22.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.405+0000 7f2656c00640 1 -- 172.21.15.28:0/1634121370 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f26500695b0 con 0x7f26500ce8d0 2024-04-26T20:58:22.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.406+0000 7f2647e00640 1 -- 172.21.15.28:0/1634121370 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f26501510b0 con 0x7f26500ce8d0 2024-04-26T20:58:22.406 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:58:22.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.409+0000 7f2656c00640 1 -- 172.21.15.28:0/1634121370 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f262c06c8f0 msgr2=0x7f262c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:22.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.409+0000 7f2656c00640 1 --2- 172.21.15.28:0/1634121370 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f262c06c8f0 0x7f262c06edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2640009f20 tx=0x7f26400094d0 comp rx=0 tx=0).stop 2024-04-26T20:58:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.409+0000 7f2656c00640 1 -- 172.21.15.28:0/1634121370 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26500ce8d0 msgr2=0x7f26501583e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.409+0000 7f2656c00640 1 --2- 172.21.15.28:0/1634121370 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26500ce8d0 0x7f26501583e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f264800b450 tx=0x7f264800b910 comp rx=0 tx=0).stop 2024-04-26T20:58:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.409+0000 7f2656c00640 1 -- 172.21.15.28:0/1634121370 shutdown_connections 2024-04-26T20:58:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.409+0000 7f2656c00640 1 --2- 172.21.15.28:0/1634121370 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f262c06c8f0 0x7f262c06edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.409+0000 7f2656c00640 1 --2- 172.21.15.28:0/1634121370 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26500ce8d0 0x7f26501583e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.409+0000 7f2656c00640 1 --2- 172.21.15.28:0/1634121370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2650073f50 0x7f26501f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.409+0000 7f2656c00640 1 --2- 172.21.15.28:0/1634121370 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f265006fc10 0x7f2650150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:22.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.409+0000 7f2656c00640 1 -- 172.21.15.28:0/1634121370 >> 172.21.15.28:0/1634121370 conn(0x7f265006c770 msgr2=0x7f265006ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:22.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.409+0000 7f2656c00640 1 -- 172.21.15.28:0/1634121370 shutdown_connections 2024-04-26T20:58:22.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.409+0000 7f2656c00640 1 -- 172.21.15.28:0/1634121370 wait complete. 2024-04-26T20:58:22.430 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:58:22.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.643+0000 7f2d16800640 1 Processor -- start 2024-04-26T20:58:22.644 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.643+0000 7f2d16800640 1 -- start start 2024-04-26T20:58:22.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.645+0000 7f2d16800640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f2d100b4f90 0x7f2d100735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:22.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.645+0000 7f2d16800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d1006f2d0 con 0x7f2d1006ed20 2024-04-26T20:58:22.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.645+0000 7f2d16800640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d1006f5b0 con 0x7f2d100731f0 2024-04-26T20:58:22.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.645+0000 7f2d16800640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f2d1006f890 con 0x7f2d100b4f90 2024-04-26T20:58:22.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.645+0000 7f2d0fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f2d1006ed20 0x7f2d10168ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37034/0 (socket says 172.21.15.28:37034) 2024-04-26T20:58:22.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.645+0000 7f2d0fe00640 1 -- 172.21.15.28:0/1502984234 learned_addr learned my addr 172.21.15.28:0/1502984234 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:22.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.645+0000 7f2d0ea00640 1 -- 172.21.15.28:0/1502984234 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3483748767 0 0) 0x7f2d1006f2d0 con 0x7f2d1006ed20 2024-04-26T20:58:22.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.645+0000 7f2d0ea00640 1 -- 172.21.15.28:0/1502984234 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2cf4003520 con 0x7f2d1006ed20 2024-04-26T20:58:22.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.646+0000 7f2d06a00640 1 --2- 172.21.15.28:0/1502984234 >> v2:172.21.15.103:3300/0 conn(0x7f2d100b4f90 0x7f2d100735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:22.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.646+0000 7f2d0ea00640 1 -- 172.21.15.28:0/1502984234 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2373079733 0 0) 0x7f2cf4003520 con 0x7f2d1006ed20 2024-04-26T20:58:22.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.646+0000 7f2d0ea00640 1 -- 172.21.15.28:0/1502984234 >> v2:172.21.15.103:3300/0 conn(0x7f2d100b4f90 msgr2=0x7f2d100735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.646+0000 7f2d0ea00640 1 --2- 172.21.15.28:0/1502984234 >> v2:172.21.15.103:3300/0 conn(0x7f2d100b4f90 0x7f2d100735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.646+0000 7f2d0ea00640 1 -- 172.21.15.28:0/1502984234 >> v1:172.21.15.103:6789/0 conn(0x7f2d100731f0 legacy=0x7f2d1006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.646+0000 7f2d0ea00640 1 -- 172.21.15.28:0/1502984234 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1006fb70 con 0x7f2d1006ed20 2024-04-26T20:58:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.646+0000 7f2d0ea00640 1 -- 172.21.15.28:0/1502984234 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f2d08001670 con 0x7f2d1006ed20 2024-04-26T20:58:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.646+0000 7f2d0ea00640 1 -- 172.21.15.28:0/1502984234 >> v1:172.21.15.28:6789/0 conn(0x7f2d1006ed20 legacy=0x7f2d10168ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.646+0000 7f2d0ea00640 1 --2- 172.21.15.28:0/1502984234 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cf4003a00 0x7f2cf4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.646+0000 7f2d0ea00640 1 --2- 172.21.15.28:0/1502984234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cf4004800 0x7f2cf4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.646+0000 7f2d0ea00640 1 --2- 172.21.15.28:0/1502984234 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2cf4005210 0x7f2cf400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.646+0000 7f2d0ea00640 1 -- 172.21.15.28:0/1502984234 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1006f890 con 0x7f2cf4005210 2024-04-26T20:58:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.646+0000 7f2d0ea00640 1 -- 172.21.15.28:0/1502984234 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2cf400c160 con 0x7f2cf4004800 2024-04-26T20:58:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.646+0000 7f2d0ea00640 1 -- 172.21.15.28:0/1502984234 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f2cf400c360 con 0x7f2cf4003a00 2024-04-26T20:58:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.646+0000 7f2d0fe00640 1 --2- 172.21.15.28:0/1502984234 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2cf4005210 0x7f2cf400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.647+0000 7f2d06a00640 1 --2- 172.21.15.28:0/1502984234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cf4004800 0x7f2cf4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:22.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.647+0000 7f2d0f400640 1 --2- 172.21.15.28:0/1502984234 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cf4003a00 0x7f2cf4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:22.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.647+0000 7f2d0fe00640 1 -- 172.21.15.28:0/1502984234 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cf4003a00 msgr2=0x7f2cf4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:22.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.647+0000 7f2d0fe00640 1 --2- 172.21.15.28:0/1502984234 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cf4003a00 0x7f2cf4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:22.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.647+0000 7f2d0fe00640 1 -- 172.21.15.28:0/1502984234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cf4004800 msgr2=0x7f2cf4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:22.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.647+0000 7f2d0fe00640 1 --2- 172.21.15.28:0/1502984234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2cf4004800 0x7f2cf4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:22.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.647+0000 7f2d0fe00640 1 -- 172.21.15.28:0/1502984234 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cf4003520 con 0x7f2cf4005210 2024-04-26T20:58:22.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.647+0000 7f2d0f400640 1 --2- 172.21.15.28:0/1502984234 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cf4003a00 0x7f2cf4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:22.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.648+0000 7f2d0fe00640 1 -- 172.21.15.28:0/1502984234 reap_dead start 2024-04-26T20:58:22.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.648+0000 7f2d0fe00640 1 --2- 172.21.15.28:0/1502984234 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2cf4005210 0x7f2cf400b9c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800c0e0 tx=0x7f2cf4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b721248ce560880 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:22.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.648+0000 7f2d0ea00640 1 -- 172.21.15.28:0/1502984234 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08007c20 con 0x7f2cf4005210 2024-04-26T20:58:22.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.648+0000 7f2d0ea00640 1 -- 172.21.15.28:0/1502984234 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d08005af0 con 0x7f2cf4005210 2024-04-26T20:58:22.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.648+0000 7f2d0ea00640 1 -- 172.21.15.28:0/1502984234 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d08007070 con 0x7f2cf4005210 2024-04-26T20:58:22.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.649+0000 7f2d16800640 1 -- 172.21.15.28:0/1502984234 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2cf4005210 msgr2=0x7f2cf400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:22.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.649+0000 7f2d16800640 1 --2- 172.21.15.28:0/1502984234 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2cf4005210 0x7f2cf400b9c0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800c0e0 tx=0x7f2cf4005830 comp rx=0 tx=0).stop 2024-04-26T20:58:22.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.649+0000 7f2d16800640 1 -- 172.21.15.28:0/1502984234 shutdown_connections 2024-04-26T20:58:22.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.649+0000 7f2d16800640 1 --2- 172.21.15.28:0/1502984234 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2cf4005210 0x7f2cf400b9c0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:22.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.649+0000 7f2d16800640 1 -- 172.21.15.28:0/1502984234 >> 172.21.15.28:0/1502984234 conn(0x7f2d1006c770 msgr2=0x7f2d1006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:22.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.649+0000 7f2d16800640 1 -- 172.21.15.28:0/1502984234 shutdown_connections 2024-04-26T20:58:22.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.649+0000 7f2d16800640 1 -- 172.21.15.28:0/1502984234 wait complete. 2024-04-26T20:58:22.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.651+0000 7f2d16800640 1 Processor -- start 2024-04-26T20:58:22.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.651+0000 7f2d16800640 1 -- start start 2024-04-26T20:58:22.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.652+0000 7f2d16800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cf4005210 0x7f2d100cf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:22.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.652+0000 7f2d16800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d100c6d70 0x7f2d100c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:22.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.652+0000 7f2d16800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d10141c30 0x7f2d101420c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:22.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.652+0000 7f2d16800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1006e790 con 0x7f2d100c6d70 2024-04-26T20:58:22.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.652+0000 7f2d16800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1006e610 con 0x7f2d10141c30 2024-04-26T20:58:22.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.652+0000 7f2d16800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f2d1006e910 con 0x7f2cf4005210 2024-04-26T20:58:22.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.652+0000 7f2d06a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d100c6d70 0x7f2d100c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:22.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.652+0000 7f2d06a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d100c6d70 0x7f2d100c71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36340/0 (socket says 172.21.15.28:36340) 2024-04-26T20:58:22.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.652+0000 7f2d06a00640 1 -- 172.21.15.28:0/1256658718 learned_addr learned my addr 172.21.15.28:0/1256658718 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:22.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.653+0000 7f2d0f400640 1 --2- 172.21.15.28:0/1256658718 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cf4005210 0x7f2d100cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:22.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.653+0000 7f2d0fe00640 1 --2- 172.21.15.28:0/1256658718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d10141c30 0x7f2d101420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:22.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.653+0000 7f2d06a00640 1 -- 172.21.15.28:0/1256658718 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cf4005210 msgr2=0x7f2d100cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:22.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.653+0000 7f2d06a00640 1 --2- 172.21.15.28:0/1256658718 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cf4005210 0x7f2d100cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:22.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.653+0000 7f2d06a00640 1 -- 172.21.15.28:0/1256658718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d10141c30 msgr2=0x7f2d101420c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:22.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.653+0000 7f2d06a00640 1 --2- 172.21.15.28:0/1256658718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d10141c30 0x7f2d101420c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:22.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.653+0000 7f2d06a00640 1 -- 172.21.15.28:0/1256658718 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d100c7910 con 0x7f2d100c6d70 2024-04-26T20:58:22.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.653+0000 7f2d0f400640 1 --2- 172.21.15.28:0/1256658718 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cf4005210 0x7f2d100cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:22.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.653+0000 7f2d0fe00640 1 --2- 172.21.15.28:0/1256658718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d10141c30 0x7f2d101420c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:22.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.653+0000 7f2d06a00640 1 --2- 172.21.15.28:0/1256658718 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d100c6d70 0x7f2d100c71f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009c80 tx=0x7f2cf8007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:22.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.653+0000 7f2d07e00640 1 -- 172.21.15.28:0/1256658718 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8014410 con 0x7f2d100c6d70 2024-04-26T20:58:22.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.654+0000 7f2d07e00640 1 -- 172.21.15.28:0/1256658718 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf8019070 con 0x7f2d100c6d70 2024-04-26T20:58:22.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.654+0000 7f2d07e00640 1 -- 172.21.15.28:0/1256658718 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf80147f0 con 0x7f2d100c6d70 2024-04-26T20:58:22.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.654+0000 7f2d16800640 1 -- 172.21.15.28:0/1256658718 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d101f59b0 con 0x7f2d100c6d70 2024-04-26T20:58:22.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.654+0000 7f2d16800640 1 -- 172.21.15.28:0/1256658718 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d101f4bf0 con 0x7f2d100c6d70 2024-04-26T20:58:22.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.655+0000 7f2d16800640 1 -- 172.21.15.28:0/1256658718 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d1006a360 con 0x7f2d100c6d70 2024-04-26T20:58:22.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.656+0000 7f2d07e00640 1 -- 172.21.15.28:0/1256658718 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2cf8014990 con 0x7f2d100c6d70 2024-04-26T20:58:22.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.657+0000 7f2d07e00640 1 --2- 172.21.15.28:0/1256658718 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2cec06c8f0 0x7f2cec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:22.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.657+0000 7f2d07e00640 1 -- 172.21.15.28:0/1256658718 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f2cf808c4b0 con 0x7f2d100c6d70 2024-04-26T20:58:22.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.657+0000 7f2d0f400640 1 --2- 172.21.15.28:0/1256658718 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2cec06c8f0 0x7f2cec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:22.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.658+0000 7f2d0f400640 1 --2- 172.21.15.28:0/1256658718 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2cec06c8f0 0x7f2cec06edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2d100af950 tx=0x7f2d00009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:22.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.661+0000 7f2d07e00640 1 -- 172.21.15.28:0/1256658718 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2cf805a840 con 0x7f2d100c6d70 2024-04-26T20:58:22.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.872+0000 7f2d16800640 1 -- 172.21.15.28:0/1256658718 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2d100cfc50 con 0x7f2d100c6d70 2024-04-26T20:58:22.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.877+0000 7f2d07e00640 1 -- 172.21.15.28:0/1256658718 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7f2cf805a140 con 0x7f2d100c6d70 2024-04-26T20:58:22.877 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-04-26T20:58:22.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.879+0000 7f2d16800640 1 -- 172.21.15.28:0/1256658718 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2cec06c8f0 msgr2=0x7f2cec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:22.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.879+0000 7f2d16800640 1 --2- 172.21.15.28:0/1256658718 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2cec06c8f0 0x7f2cec06edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2d100af950 tx=0x7f2d00009040 comp rx=0 tx=0).stop 2024-04-26T20:58:22.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.879+0000 7f2d16800640 1 -- 172.21.15.28:0/1256658718 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d100c6d70 msgr2=0x7f2d100c71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:22.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.879+0000 7f2d16800640 1 --2- 172.21.15.28:0/1256658718 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d100c6d70 0x7f2d100c71f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8009c80 tx=0x7f2cf8007700 comp rx=0 tx=0).stop 2024-04-26T20:58:22.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.880+0000 7f2d16800640 1 -- 172.21.15.28:0/1256658718 shutdown_connections 2024-04-26T20:58:22.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.880+0000 7f2d16800640 1 --2- 172.21.15.28:0/1256658718 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2cec06c8f0 0x7f2cec06edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:22.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.880+0000 7f2d16800640 1 --2- 172.21.15.28:0/1256658718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2d10141c30 0x7f2d101420c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:22.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.880+0000 7f2d16800640 1 --2- 172.21.15.28:0/1256658718 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2d100c6d70 0x7f2d100c71f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:22.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.880+0000 7f2d16800640 1 --2- 172.21.15.28:0/1256658718 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2cf4005210 0x7f2d100cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:22.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.880+0000 7f2d16800640 1 -- 172.21.15.28:0/1256658718 >> 172.21.15.28:0/1256658718 conn(0x7f2d1006c770 msgr2=0x7f2d100b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:22.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.880+0000 7f2d16800640 1 -- 172.21.15.28:0/1256658718 shutdown_connections 2024-04-26T20:58:22.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:22.880+0000 7f2d16800640 1 -- 172.21.15.28:0/1256658718 wait complete. 2024-04-26T20:58:22.899 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:58:23.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.116+0000 7ffa64800640 1 Processor -- start 2024-04-26T20:58:23.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.116+0000 7ffa64800640 1 -- start start 2024-04-26T20:58:23.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.118+0000 7ffa64800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffa5c146860 0x7ffa5c06f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:23.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.118+0000 7ffa64800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa5c0b4730 0x7ffa5c15e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:23.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.118+0000 7ffa64800640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa5c0b4d90 con 0x7ffa5c06f810 2024-04-26T20:58:23.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.118+0000 7ffa64800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ffa5c06fc70 con 0x7ffa5c146860 2024-04-26T20:58:23.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.118+0000 7ffa64800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ffa5c15d850 con 0x7ffa5c0b4730 2024-04-26T20:58:23.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.118+0000 7ffa61e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffa5c146860 0x7ffa5c06f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:23.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.118+0000 7ffa61e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffa5c146860 0x7ffa5c06f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36354/0 (socket says 172.21.15.28:36354) 2024-04-26T20:58:23.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.118+0000 7ffa61e00640 1 -- 172.21.15.28:0/2042135552 learned_addr learned my addr 172.21.15.28:0/2042135552 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:23.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.119+0000 7ffa62800640 1 --2- 172.21.15.28:0/2042135552 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa5c0b4730 0x7ffa5c15e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:23.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.119+0000 7ffa61e00640 1 -- 172.21.15.28:0/2042135552 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa5c0b4730 msgr2=0x7ffa5c15e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:23.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.119+0000 7ffa61e00640 1 --2- 172.21.15.28:0/2042135552 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa5c0b4730 0x7ffa5c15e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.119+0000 7ffa61e00640 1 -- 172.21.15.28:0/2042135552 >> v1:172.21.15.103:6789/0 conn(0x7ffa5c06f810 legacy=0x7ffa5c0b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:23.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.119+0000 7ffa61e00640 1 -- 172.21.15.28:0/2042135552 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5c15db30 con 0x7ffa5c146860 2024-04-26T20:58:23.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.120+0000 7ffa61e00640 1 --2- 172.21.15.28:0/2042135552 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c146860 0x7ffa5c06f2d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ffa48009870 tx=0x7ffa4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18afa5ec84ae56d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:23.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.121+0000 7ffa60a00640 1 -- 172.21.15.28:0/2042135552 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa48037070 con 0x7ffa5c146860 2024-04-26T20:58:23.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.121+0000 7ffa60a00640 1 -- 172.21.15.28:0/2042135552 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa48032440 con 0x7ffa5c146860 2024-04-26T20:58:23.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.121+0000 7ffa60a00640 1 -- 172.21.15.28:0/2042135552 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa48032810 con 0x7ffa5c146860 2024-04-26T20:58:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.121+0000 7ffa64800640 1 -- 172.21.15.28:0/2042135552 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c146860 msgr2=0x7ffa5c06f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.121+0000 7ffa64800640 1 --2- 172.21.15.28:0/2042135552 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c146860 0x7ffa5c06f2d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ffa48009870 tx=0x7ffa4802a270 comp rx=0 tx=0).stop 2024-04-26T20:58:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.121+0000 7ffa64800640 1 -- 172.21.15.28:0/2042135552 shutdown_connections 2024-04-26T20:58:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.121+0000 7ffa64800640 1 --2- 172.21.15.28:0/2042135552 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa5c0b4730 0x7ffa5c15e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.121+0000 7ffa64800640 1 --2- 172.21.15.28:0/2042135552 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c146860 0x7ffa5c06f2d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.121+0000 7ffa64800640 1 -- 172.21.15.28:0/2042135552 >> 172.21.15.28:0/2042135552 conn(0x7ffa5c0b2b60 msgr2=0x7ffa5c0b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.122+0000 7ffa64800640 1 -- 172.21.15.28:0/2042135552 shutdown_connections 2024-04-26T20:58:23.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.122+0000 7ffa64800640 1 -- 172.21.15.28:0/2042135552 wait complete. 2024-04-26T20:58:23.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.123+0000 7ffa64800640 1 Processor -- start 2024-04-26T20:58:23.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.124+0000 7ffa64800640 1 -- start start 2024-04-26T20:58:23.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.125+0000 7ffa64800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c06f810 0x7ffa5c1e9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:23.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.125+0000 7ffa64800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa5c0b4730 0x7ffa5c1e7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:23.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.125+0000 7ffa64800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa5c146860 0x7ffa5c142770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:23.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.125+0000 7ffa64800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c15e9f0 con 0x7ffa5c06f810 2024-04-26T20:58:23.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.125+0000 7ffa64800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c15e870 con 0x7ffa5c0b4730 2024-04-26T20:58:23.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.125+0000 7ffa64800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ffa5c15eb70 con 0x7ffa5c146860 2024-04-26T20:58:23.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.125+0000 7ffa61e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c06f810 0x7ffa5c1e9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:23.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.125+0000 7ffa61e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c06f810 0x7ffa5c1e9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36366/0 (socket says 172.21.15.28:36366) 2024-04-26T20:58:23.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.125+0000 7ffa61e00640 1 -- 172.21.15.28:0/1005185696 learned_addr learned my addr 172.21.15.28:0/1005185696 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:23.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.125+0000 7ffa61e00640 1 -- 172.21.15.28:0/1005185696 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa5c146860 msgr2=0x7ffa5c142770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:23.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.125+0000 7ffa61e00640 1 --2- 172.21.15.28:0/1005185696 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa5c146860 0x7ffa5c142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.125+0000 7ffa61e00640 1 -- 172.21.15.28:0/1005185696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa5c0b4730 msgr2=0x7ffa5c1e7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:23.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.125+0000 7ffa61e00640 1 --2- 172.21.15.28:0/1005185696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa5c0b4730 0x7ffa5c1e7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.125+0000 7ffa61e00640 1 -- 172.21.15.28:0/1005185696 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5c1e81e0 con 0x7ffa5c06f810 2024-04-26T20:58:23.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.126+0000 7ffa61e00640 1 --2- 172.21.15.28:0/1005185696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c06f810 0x7ffa5c1e9a90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ffa48000c00 tx=0x7ffa480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:23.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.126+0000 7ffa56000640 1 -- 172.21.15.28:0/1005185696 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa48037040 con 0x7ffa5c06f810 2024-04-26T20:58:23.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.126+0000 7ffa56000640 1 -- 172.21.15.28:0/1005185696 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa48042070 con 0x7ffa5c06f810 2024-04-26T20:58:23.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.126+0000 7ffa56000640 1 -- 172.21.15.28:0/1005185696 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4803b870 con 0x7ffa5c06f810 2024-04-26T20:58:23.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.126+0000 7ffa64800640 1 -- 172.21.15.28:0/1005185696 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5c1e84c0 con 0x7ffa5c06f810 2024-04-26T20:58:23.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.126+0000 7ffa64800640 1 -- 172.21.15.28:0/1005185696 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa5c1e8a80 con 0x7ffa5c06f810 2024-04-26T20:58:23.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.127+0000 7ffa64800640 1 -- 172.21.15.28:0/1005185696 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa5c076520 con 0x7ffa5c06f810 2024-04-26T20:58:23.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.128+0000 7ffa56000640 1 -- 172.21.15.28:0/1005185696 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa4803ba10 con 0x7ffa5c06f810 2024-04-26T20:58:23.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.129+0000 7ffa56000640 1 --2- 172.21.15.28:0/1005185696 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ffa3c06c8f0 0x7ffa3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:23.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.129+0000 7ffa56000640 1 -- 172.21.15.28:0/1005185696 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7ffa480ac410 con 0x7ffa5c06f810 2024-04-26T20:58:23.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.129+0000 7ffa61400640 1 --2- 172.21.15.28:0/1005185696 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ffa3c06c8f0 0x7ffa3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:23.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.130+0000 7ffa61400640 1 --2- 172.21.15.28:0/1005185696 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ffa3c06c8f0 0x7ffa3c06edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffa50000f60 tx=0x7ffa500079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:23.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.134+0000 7ffa56000640 1 -- 172.21.15.28:0/1005185696 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa4807a7a0 con 0x7ffa5c06f810 2024-04-26T20:58:23.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.339+0000 7ffa64800640 1 -- 172.21.15.28:0/1005185696 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffa5c15cbf0 con 0x7ffa5c06f810 2024-04-26T20:58:23.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.341+0000 7ffa56000640 1 -- 172.21.15.28:0/1005185696 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7ffa4807a0a0 con 0x7ffa5c06f810 2024-04-26T20:58:23.341 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-04-26T20:58:23.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.343+0000 7ffa64800640 1 -- 172.21.15.28:0/1005185696 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ffa3c06c8f0 msgr2=0x7ffa3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:23.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.343+0000 7ffa64800640 1 --2- 172.21.15.28:0/1005185696 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ffa3c06c8f0 0x7ffa3c06edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffa50000f60 tx=0x7ffa500079d0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.343+0000 7ffa64800640 1 -- 172.21.15.28:0/1005185696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c06f810 msgr2=0x7ffa5c1e9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:23.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.343+0000 7ffa64800640 1 --2- 172.21.15.28:0/1005185696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c06f810 0x7ffa5c1e9a90 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ffa48000c00 tx=0x7ffa480046c0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.343+0000 7ffa64800640 1 -- 172.21.15.28:0/1005185696 shutdown_connections 2024-04-26T20:58:23.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.343+0000 7ffa64800640 1 --2- 172.21.15.28:0/1005185696 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ffa3c06c8f0 0x7ffa3c06edb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.343+0000 7ffa64800640 1 --2- 172.21.15.28:0/1005185696 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa5c146860 0x7ffa5c142770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.343+0000 7ffa64800640 1 --2- 172.21.15.28:0/1005185696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa5c0b4730 0x7ffa5c1e7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.343+0000 7ffa64800640 1 --2- 172.21.15.28:0/1005185696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa5c06f810 0x7ffa5c1e9a90 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.343+0000 7ffa64800640 1 -- 172.21.15.28:0/1005185696 >> 172.21.15.28:0/1005185696 conn(0x7ffa5c0b2b60 msgr2=0x7ffa5c0b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:23.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.343+0000 7ffa64800640 1 -- 172.21.15.28:0/1005185696 shutdown_connections 2024-04-26T20:58:23.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.343+0000 7ffa64800640 1 -- 172.21.15.28:0/1005185696 wait complete. 2024-04-26T20:58:23.364 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T20:58:23.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.580+0000 7f83b3200640 1 Processor -- start 2024-04-26T20:58:23.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.580+0000 7f83b3200640 1 -- start start 2024-04-26T20:58:23.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.582+0000 7f83b3200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f83b4146060 0x7f83b4071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:23.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.582+0000 7f83b3200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83b4146670 0x7f83b41609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:23.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.582+0000 7f83b3200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83b4146440 con 0x7f83b4160ef0 2024-04-26T20:58:23.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.582+0000 7f83b3200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f83b406f510 con 0x7f83b4146060 2024-04-26T20:58:23.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.582+0000 7f83b3200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f83b406f010 con 0x7f83b4146670 2024-04-26T20:58:23.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.583+0000 7f83b2800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f83b4160ef0 0x7f83b406de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37050/0 (socket says 172.21.15.28:37050) 2024-04-26T20:58:23.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.583+0000 7f83b2800640 1 -- 172.21.15.28:0/3316576064 learned_addr learned my addr 172.21.15.28:0/3316576064 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:23.584 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.583+0000 7f83b1400640 1 --2- 172.21.15.28:0/3316576064 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83b4146670 0x7f83b41609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.583+0000 7f83b1e00640 1 --2- 172.21.15.28:0/3316576064 >> v2:172.21.15.103:3300/0 conn(0x7f83b4146060 0x7f83b4071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.583+0000 7f83b0a00640 1 -- 172.21.15.28:0/3316576064 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1029096683 0 0) 0x7f83b4146440 con 0x7f83b4160ef0 2024-04-26T20:58:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.583+0000 7f83b0a00640 1 -- 172.21.15.28:0/3316576064 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8398003520 con 0x7f83b4160ef0 2024-04-26T20:58:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.583+0000 7f83b0a00640 1 -- 172.21.15.28:0/3316576064 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 254859200 0 0) 0x7f8398003520 con 0x7f83b4160ef0 2024-04-26T20:58:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.583+0000 7f83b0a00640 1 -- 172.21.15.28:0/3316576064 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83b4146670 msgr2=0x7f83b41609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.583+0000 7f83b0a00640 1 --2- 172.21.15.28:0/3316576064 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83b4146670 0x7f83b41609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.583+0000 7f83b0a00640 1 -- 172.21.15.28:0/3316576064 >> v2:172.21.15.103:3300/0 conn(0x7f83b4146060 msgr2=0x7f83b4071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.583+0000 7f83b0a00640 1 --2- 172.21.15.28:0/3316576064 >> v2:172.21.15.103:3300/0 conn(0x7f83b4146060 0x7f83b4071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.583+0000 7f83b0a00640 1 -- 172.21.15.28:0/3316576064 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b41617a0 con 0x7f83b4160ef0 2024-04-26T20:58:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.583+0000 7f83b0a00640 1 -- 172.21.15.28:0/3316576064 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f83ac0019f0 con 0x7f83b4160ef0 2024-04-26T20:58:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.583+0000 7f83b0a00640 1 -- 172.21.15.28:0/3316576064 >> v1:172.21.15.28:6789/0 conn(0x7f83b4160ef0 legacy=0x7f83b406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.584+0000 7f83b0a00640 1 --2- 172.21.15.28:0/3316576064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83980039b0 0x7f8398003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.584+0000 7f83b0a00640 1 --2- 172.21.15.28:0/3316576064 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83b4146670 0x7f8398004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:23.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.584+0000 7f83b0a00640 1 --2- 172.21.15.28:0/3316576064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8398004d40 0x7f83980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:23.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.584+0000 7f83b0a00640 1 -- 172.21.15.28:0/3316576064 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f83b406f510 con 0x7f83980039b0 2024-04-26T20:58:23.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.584+0000 7f83b0a00640 1 -- 172.21.15.28:0/3316576064 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f83b406f010 con 0x7f8398004d40 2024-04-26T20:58:23.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.584+0000 7f83b0a00640 1 -- 172.21.15.28:0/3316576064 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f839800b9f0 con 0x7f83b4146670 2024-04-26T20:58:23.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.584+0000 7f83b1e00640 1 --2- 172.21.15.28:0/3316576064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83980039b0 0x7f8398003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:23.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.584+0000 7f83b1e00640 1 -- 172.21.15.28:0/3316576064 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83b4146670 msgr2=0x7f8398004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:23.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.584+0000 7f83b1e00640 1 --2- 172.21.15.28:0/3316576064 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83b4146670 0x7f8398004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.584+0000 7f83b1e00640 1 -- 172.21.15.28:0/3316576064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8398004d40 msgr2=0x7f83980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:23.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.584+0000 7f83b1e00640 1 --2- 172.21.15.28:0/3316576064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8398004d40 0x7f83980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.586 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.584+0000 7f83b1e00640 1 -- 172.21.15.28:0/3316576064 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8398003520 con 0x7f83980039b0 2024-04-26T20:58:23.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.585+0000 7f83b1e00640 1 --2- 172.21.15.28:0/3316576064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83980039b0 0x7f8398003dd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009ac0 tx=0x7f8398005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a0db1f06a1f3d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:23.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.585+0000 7f83b0a00640 1 -- 172.21.15.28:0/3316576064 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a4036070 con 0x7f83980039b0 2024-04-26T20:58:23.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.586+0000 7f83b0a00640 1 -- 172.21.15.28:0/3316576064 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83a4031400 con 0x7f83980039b0 2024-04-26T20:58:23.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.586+0000 7f83b0a00640 1 -- 172.21.15.28:0/3316576064 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a4009690 con 0x7f83980039b0 2024-04-26T20:58:23.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.586+0000 7f83b3200640 1 -- 172.21.15.28:0/3316576064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83980039b0 msgr2=0x7f8398003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:23.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.586+0000 7f83b3200640 1 --2- 172.21.15.28:0/3316576064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83980039b0 0x7f8398003dd0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009ac0 tx=0x7f8398005830 comp rx=0 tx=0).stop 2024-04-26T20:58:23.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.586+0000 7f83b2800640 1 -- 172.21.15.28:0/3316576064 reap_dead start 2024-04-26T20:58:23.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.586+0000 7f83b3200640 1 -- 172.21.15.28:0/3316576064 shutdown_connections 2024-04-26T20:58:23.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.586+0000 7f83b3200640 1 -- 172.21.15.28:0/3316576064 >> 172.21.15.28:0/3316576064 conn(0x7f83b40d0a50 msgr2=0x7f83b40d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:23.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.586+0000 7f83b3200640 1 -- 172.21.15.28:0/3316576064 shutdown_connections 2024-04-26T20:58:23.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.586+0000 7f83b3200640 1 -- 172.21.15.28:0/3316576064 wait complete. 2024-04-26T20:58:23.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.588+0000 7f83b3200640 1 Processor -- start 2024-04-26T20:58:23.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.588+0000 7f83b3200640 1 -- start start 2024-04-26T20:58:23.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.589+0000 7f83b3200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83980039b0 0x7f83b40b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:23.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.589+0000 7f83b3200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83b415de20 0x7f83b40b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:23.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.589+0000 7f83b3200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83b41f0be0 0x7f83b41f1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:23.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.589+0000 7f83b3200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f83b40cc0b0 con 0x7f83b41f0be0 2024-04-26T20:58:23.590 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.589+0000 7f83b3200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f83b406ff60 con 0x7f83b415de20 2024-04-26T20:58:23.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.589+0000 7f83b3200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f83b40cc230 con 0x7f83980039b0 2024-04-26T20:58:23.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.589+0000 7f83b2800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83b41f0be0 0x7f83b41f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:23.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.589+0000 7f83b2800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83b41f0be0 0x7f83b41f1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36378/0 (socket says 172.21.15.28:36378) 2024-04-26T20:58:23.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.589+0000 7f83b2800640 1 -- 172.21.15.28:0/2959688956 learned_addr learned my addr 172.21.15.28:0/2959688956 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:23.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.590+0000 7f83b1400640 1 --2- 172.21.15.28:0/2959688956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83b415de20 0x7f83b40b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:23.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.590+0000 7f83b1e00640 1 --2- 172.21.15.28:0/2959688956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83980039b0 0x7f83b40b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:23.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.590+0000 7f83b2800640 1 -- 172.21.15.28:0/2959688956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83980039b0 msgr2=0x7f83b40b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:23.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.590+0000 7f83b2800640 1 --2- 172.21.15.28:0/2959688956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83980039b0 0x7f83b40b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.590+0000 7f83b2800640 1 -- 172.21.15.28:0/2959688956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83b415de20 msgr2=0x7f83b40b0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:23.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.590+0000 7f83b2800640 1 --2- 172.21.15.28:0/2959688956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83b415de20 0x7f83b40b0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.590+0000 7f83b2800640 1 -- 172.21.15.28:0/2959688956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b40b4060 con 0x7f83b41f0be0 2024-04-26T20:58:23.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.590+0000 7f83b1e00640 1 --2- 172.21.15.28:0/2959688956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83980039b0 0x7f83b40b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:23.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.590+0000 7f83b2800640 1 --2- 172.21.15.28:0/2959688956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83b41f0be0 0x7f83b41f1060 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f83ac000f80 tx=0x7f83ac00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:23.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.590+0000 7f83aa000640 1 -- 172.21.15.28:0/2959688956 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac00a6d0 con 0x7f83b41f0be0 2024-04-26T20:58:23.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.591+0000 7f83aa000640 1 -- 172.21.15.28:0/2959688956 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83ac00a870 con 0x7f83b41f0be0 2024-04-26T20:58:23.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.591+0000 7f83b3200640 1 -- 172.21.15.28:0/2959688956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b40b4340 con 0x7f83b41f0be0 2024-04-26T20:58:23.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.591+0000 7f83aa000640 1 -- 172.21.15.28:0/2959688956 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac00ac60 con 0x7f83b41f0be0 2024-04-26T20:58:23.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.591+0000 7f83b3200640 1 -- 172.21.15.28:0/2959688956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b413dc40 con 0x7f83b41f0be0 2024-04-26T20:58:23.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.592+0000 7f83b3200640 1 -- 172.21.15.28:0/2959688956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83b4071900 con 0x7f83b41f0be0 2024-04-26T20:58:23.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.593+0000 7f83aa000640 1 -- 172.21.15.28:0/2959688956 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83ac007070 con 0x7f83b41f0be0 2024-04-26T20:58:23.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.594+0000 7f83aa000640 1 --2- 172.21.15.28:0/2959688956 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f839406c8f0 0x7f839406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:23.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.594+0000 7f83aa000640 1 -- 172.21.15.28:0/2959688956 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f83ac08b700 con 0x7f83b41f0be0 2024-04-26T20:58:23.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.594+0000 7f83b1e00640 1 --2- 172.21.15.28:0/2959688956 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f839406c8f0 0x7f839406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:23.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.595+0000 7f83b1e00640 1 --2- 172.21.15.28:0/2959688956 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f839406c8f0 0x7f839406edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f83a4033000 tx=0x7f83a4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:23.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.598+0000 7f83aa000640 1 -- 172.21.15.28:0/2959688956 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f83b4071900 con 0x7f83b41f0be0 2024-04-26T20:58:23.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.817+0000 7f83b3200640 1 -- 172.21.15.28:0/2959688956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f83b406a360 con 0x7f83b41f0be0 2024-04-26T20:58:23.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.818+0000 7f83aa000640 1 -- 172.21.15.28:0/2959688956 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) ==== 80+0+0 (secure 0 0 0) 0x7f83ac0593b0 con 0x7f83b41f0be0 2024-04-26T20:58:23.818 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-04-26T20:58:23.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.821+0000 7f83b3200640 1 -- 172.21.15.28:0/2959688956 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f839406c8f0 msgr2=0x7f839406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:23.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.821+0000 7f83b3200640 1 --2- 172.21.15.28:0/2959688956 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f839406c8f0 0x7f839406edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f83a4033000 tx=0x7f83a4008030 comp rx=0 tx=0).stop 2024-04-26T20:58:23.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.821+0000 7f83b3200640 1 -- 172.21.15.28:0/2959688956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83b41f0be0 msgr2=0x7f83b41f1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:23.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.821+0000 7f83b3200640 1 --2- 172.21.15.28:0/2959688956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83b41f0be0 0x7f83b41f1060 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f83ac000f80 tx=0x7f83ac00cf00 comp rx=0 tx=0).stop 2024-04-26T20:58:23.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.821+0000 7f83b3200640 1 -- 172.21.15.28:0/2959688956 shutdown_connections 2024-04-26T20:58:23.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.821+0000 7f83b3200640 1 --2- 172.21.15.28:0/2959688956 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f839406c8f0 0x7f839406edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.821+0000 7f83b3200640 1 --2- 172.21.15.28:0/2959688956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83b41f0be0 0x7f83b41f1060 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.821+0000 7f83b3200640 1 --2- 172.21.15.28:0/2959688956 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83b415de20 0x7f83b40b0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.821+0000 7f83b3200640 1 --2- 172.21.15.28:0/2959688956 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f83980039b0 0x7f83b40b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:23.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.821+0000 7f83b3200640 1 -- 172.21.15.28:0/2959688956 >> 172.21.15.28:0/2959688956 conn(0x7f83b40d0a50 msgr2=0x7f83b415f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:23.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.821+0000 7f83b3200640 1 -- 172.21.15.28:0/2959688956 shutdown_connections 2024-04-26T20:58:23.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:23.821+0000 7f83b3200640 1 -- 172.21.15.28:0/2959688956 wait complete. 2024-04-26T20:58:23.841 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi028.front.sepia.ceph.com... 2024-04-26T20:58:23.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:23.841 DEBUG:teuthology.orchestra.run.smithi028:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-26T20:58:23.915 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:ceph-fuse[45804]: fuse finished with error 0 and tester_r 0 2024-04-26T20:58:23.920 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T20:58:23.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:23.921 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T20:58:24.098 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:58:24.099 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:58:24.099 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:58:24.143 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052734,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:58:24.143 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:58:24.143 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:58:24.143 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:58:24.199 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:58:24.199 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T20:58:24.199 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:58:24.199 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:58:24.254 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052734,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:58:24.255 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:58:24.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.476+0000 7f860a200640 1 Processor -- start 2024-04-26T20:58:24.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.476+0000 7f860a200640 1 -- start start 2024-04-26T20:58:24.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.478+0000 7f860a200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8604146670 0x7f8604071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:24.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.478+0000 7f860a200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f86041610c0 0x7f86040cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:24.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.478+0000 7f860a200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8604161530 con 0x7f8604146060 2024-04-26T20:58:24.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.478+0000 7f860a200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f860406f510 con 0x7f86041610c0 2024-04-26T20:58:24.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.478+0000 7f860a200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f8604146440 con 0x7f8604146670 2024-04-26T20:58:24.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.478+0000 7f8608e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8604146060 0x7f86041609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37060/0 (socket says 172.21.15.28:37060) 2024-04-26T20:58:24.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.478+0000 7f8608e00640 1 -- 172.21.15.28:0/3160109589 learned_addr learned my addr 172.21.15.28:0/3160109589 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:24.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.479+0000 7f8609800640 1 --2- 172.21.15.28:0/3160109589 >> v2:172.21.15.103:3300/0 conn(0x7f86041610c0 0x7f86040cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:24.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.479+0000 7f85fbe00640 1 --2- 172.21.15.28:0/3160109589 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8604146670 0x7f8604071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:24.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.479+0000 7f8603e00640 1 -- 172.21.15.28:0/3160109589 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4120944770 0 0) 0x7f8604161530 con 0x7f8604146060 2024-04-26T20:58:24.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.479+0000 7f8603e00640 1 -- 172.21.15.28:0/3160109589 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85e8003520 con 0x7f8604146060 2024-04-26T20:58:24.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.479+0000 7f8603e00640 1 -- 172.21.15.28:0/3160109589 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 181217056 0 0) 0x7f85e8003520 con 0x7f8604146060 2024-04-26T20:58:24.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.479+0000 7f8603e00640 1 -- 172.21.15.28:0/3160109589 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8604146670 msgr2=0x7f8604071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:24.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.479+0000 7f8603e00640 1 --2- 172.21.15.28:0/3160109589 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8604146670 0x7f8604071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:24.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.479+0000 7f8603e00640 1 -- 172.21.15.28:0/3160109589 >> v2:172.21.15.103:3300/0 conn(0x7f86041610c0 msgr2=0x7f86040cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:24.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.479+0000 7f8603e00640 1 --2- 172.21.15.28:0/3160109589 >> v2:172.21.15.103:3300/0 conn(0x7f86041610c0 0x7f86040cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:24.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.479+0000 7f8603e00640 1 -- 172.21.15.28:0/3160109589 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860406f010 con 0x7f8604146060 2024-04-26T20:58:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.479+0000 7f8603e00640 1 -- 172.21.15.28:0/3160109589 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f85f4001670 con 0x7f8604146060 2024-04-26T20:58:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.479+0000 7f8603e00640 1 -- 172.21.15.28:0/3160109589 >> v1:172.21.15.28:6789/0 conn(0x7f8604146060 legacy=0x7f86041609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.479+0000 7f8603e00640 1 --2- 172.21.15.28:0/3160109589 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8604146670 0x7f85e80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.480+0000 7f8603e00640 1 --2- 172.21.15.28:0/3160109589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85e8003f80 0x7f85e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.480+0000 7f8603e00640 1 --2- 172.21.15.28:0/3160109589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85e8004d40 0x7f85e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.480+0000 7f8603e00640 1 -- 172.21.15.28:0/3160109589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f860406f510 con 0x7f85e8003f80 2024-04-26T20:58:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.480+0000 7f8603e00640 1 -- 172.21.15.28:0/3160109589 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8604146440 con 0x7f85e8004d40 2024-04-26T20:58:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.480+0000 7f8603e00640 1 -- 172.21.15.28:0/3160109589 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f85e800ba70 con 0x7f8604146670 2024-04-26T20:58:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.480+0000 7f85fbe00640 1 --2- 172.21.15.28:0/3160109589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85e8003f80 0x7f85e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.480+0000 7f85fbe00640 1 -- 172.21.15.28:0/3160109589 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8604146670 msgr2=0x7f85e80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.480+0000 7f85fbe00640 1 --2- 172.21.15.28:0/3160109589 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f8604146670 0x7f85e80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:24.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.480+0000 7f85fbe00640 1 -- 172.21.15.28:0/3160109589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85e8004d40 msgr2=0x7f85e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:24.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.480+0000 7f8609800640 1 --2- 172.21.15.28:0/3160109589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85e8004d40 0x7f85e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:24.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.480+0000 7f85fbe00640 1 --2- 172.21.15.28:0/3160109589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85e8004d40 0x7f85e80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:24.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.480+0000 7f85fbe00640 1 -- 172.21.15.28:0/3160109589 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e8003520 con 0x7f85e8003f80 2024-04-26T20:58:24.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.481+0000 7f85fbe00640 1 --2- 172.21.15.28:0/3160109589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85e8003f80 0x7f85e8004800 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009ac0 tx=0x7f85e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cfefe13dded989 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:24.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.482+0000 7f8603e00640 1 -- 172.21.15.28:0/3160109589 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec036070 con 0x7f85e8003f80 2024-04-26T20:58:24.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.482+0000 7f8603e00640 1 -- 172.21.15.28:0/3160109589 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85ec031400 con 0x7f85e8003f80 2024-04-26T20:58:24.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.482+0000 7f8603e00640 1 -- 172.21.15.28:0/3160109589 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85ec009690 con 0x7f85e8003f80 2024-04-26T20:58:24.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.482+0000 7f860a200640 1 -- 172.21.15.28:0/3160109589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85e8003f80 msgr2=0x7f85e8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:24.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.482+0000 7f860a200640 1 --2- 172.21.15.28:0/3160109589 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85e8003f80 0x7f85e8004800 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f85ec009ac0 tx=0x7f85e8005830 comp rx=0 tx=0).stop 2024-04-26T20:58:24.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.482+0000 7f8609800640 1 -- 172.21.15.28:0/3160109589 reap_dead start 2024-04-26T20:58:24.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.482+0000 7f860a200640 1 -- 172.21.15.28:0/3160109589 shutdown_connections 2024-04-26T20:58:24.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.482+0000 7f860a200640 1 -- 172.21.15.28:0/3160109589 >> 172.21.15.28:0/3160109589 conn(0x7f86040d0a50 msgr2=0x7f86040d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:24.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.483+0000 7f860a200640 1 -- 172.21.15.28:0/3160109589 shutdown_connections 2024-04-26T20:58:24.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.483+0000 7f860a200640 1 -- 172.21.15.28:0/3160109589 wait complete. 2024-04-26T20:58:24.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.484+0000 7f860a200640 1 Processor -- start 2024-04-26T20:58:24.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.484+0000 7f860a200640 1 -- start start 2024-04-26T20:58:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.485+0000 7f860a200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85e8003f80 0x7f8604145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.485+0000 7f860a200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86041426d0 0x7f8604141510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.486+0000 7f860a200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86041f3360 0x7f86040b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.486+0000 7f860a200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f860406ff60 con 0x7f86041f3360 2024-04-26T20:58:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.486+0000 7f860a200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f860406fc80 con 0x7f86041426d0 2024-04-26T20:58:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.486+0000 7f860a200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f8604071e20 con 0x7f85e8003f80 2024-04-26T20:58:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.486+0000 7f8609800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86041f3360 0x7f86040b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.486+0000 7f8609800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86041f3360 0x7f86040b07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36410/0 (socket says 172.21.15.28:36410) 2024-04-26T20:58:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.486+0000 7f8609800640 1 -- 172.21.15.28:0/3986374944 learned_addr learned my addr 172.21.15.28:0/3986374944 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.486+0000 7f85fbe00640 1 --2- 172.21.15.28:0/3986374944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86041426d0 0x7f8604141510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.486+0000 7f8608e00640 1 --2- 172.21.15.28:0/3986374944 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85e8003f80 0x7f8604145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.486+0000 7f8609800640 1 -- 172.21.15.28:0/3986374944 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85e8003f80 msgr2=0x7f8604145030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.486+0000 7f8609800640 1 --2- 172.21.15.28:0/3986374944 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85e8003f80 0x7f8604145030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:24.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.486+0000 7f8609800640 1 -- 172.21.15.28:0/3986374944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86041426d0 msgr2=0x7f8604141510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:24.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.486+0000 7f8609800640 1 --2- 172.21.15.28:0/3986374944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86041426d0 0x7f8604141510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:24.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.486+0000 7f8609800640 1 -- 172.21.15.28:0/3986374944 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8604142e50 con 0x7f86041f3360 2024-04-26T20:58:24.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.487+0000 7f8609800640 1 --2- 172.21.15.28:0/3986374944 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86041f3360 0x7f86040b07b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8604072360 tx=0x7f85fc00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:24.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.487+0000 7f8601600640 1 -- 172.21.15.28:0/3986374944 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc016620 con 0x7f86041f3360 2024-04-26T20:58:24.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.487+0000 7f8601600640 1 -- 172.21.15.28:0/3986374944 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85fc01b070 con 0x7f86041f3360 2024-04-26T20:58:24.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.487+0000 7f8601600640 1 -- 172.21.15.28:0/3986374944 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc016a20 con 0x7f86041f3360 2024-04-26T20:58:24.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.487+0000 7f860a200640 1 -- 172.21.15.28:0/3986374944 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8604142050 con 0x7f86041f3360 2024-04-26T20:58:24.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.487+0000 7f860a200640 1 -- 172.21.15.28:0/3986374944 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8604144550 con 0x7f86041f3360 2024-04-26T20:58:24.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.489+0000 7f860a200640 1 -- 172.21.15.28:0/3986374944 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f860406a360 con 0x7f86041f3360 2024-04-26T20:58:24.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.490+0000 7f8601600640 1 -- 172.21.15.28:0/3986374944 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85fc005d10 con 0x7f86041f3360 2024-04-26T20:58:24.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.490+0000 7f8601600640 1 --2- 172.21.15.28:0/3986374944 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f85e006c920 0x7f85e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:24.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.491+0000 7f8601600640 1 -- 172.21.15.28:0/3986374944 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f85fc08c7a0 con 0x7f86041f3360 2024-04-26T20:58:24.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.491+0000 7f8608e00640 1 --2- 172.21.15.28:0/3986374944 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f85e006c920 0x7f85e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:24.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.491+0000 7f8608e00640 1 --2- 172.21.15.28:0/3986374944 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f85e006c920 0x7f85e006ede0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f85f4000ff0 tx=0x7f85f4001680 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:24.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.495+0000 7f8601600640 1 -- 172.21.15.28:0/3986374944 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f85fc025020 con 0x7f86041f3360 2024-04-26T20:58:24.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.740+0000 7f860a200640 1 -- 172.21.15.28:0/3986374944 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8604072b10 con 0x7f86041f3360 2024-04-26T20:58:24.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.741+0000 7f8601600640 1 -- 172.21.15.28:0/3986374944 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7f8604142e50 con 0x7f86041f3360 2024-04-26T20:58:24.742 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:24.742 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/1364678224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1364678224},{"type":"v1","addr":"172.21.15.103:6834","nonce":1364678224}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4081926466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":4081926466},{"type":"v1","addr":"172.21.15.28:6837","nonce":4081926466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5175,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/1370252546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":1370252546},{"type":"v1","addr":"172.21.15.28:6835","nonce":1370252546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:58:09.226353+0000","modified":"2024-04-26T20:58:12.319110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5184":{"gid":5184,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6835/1781724684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":1781724684},{"type":"v1","addr":"172.21.15.103:6835","nonce":1781724684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5184,"qdb_cluster":[5184]},"id":2}]} 2024-04-26T20:58:24.742 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-04-26T20:58:24.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.744+0000 7f860a200640 1 -- 172.21.15.28:0/3986374944 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f85e006c920 msgr2=0x7f85e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:24.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.744+0000 7f860a200640 1 --2- 172.21.15.28:0/3986374944 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f85e006c920 0x7f85e006ede0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f85f4000ff0 tx=0x7f85f4001680 comp rx=0 tx=0).stop 2024-04-26T20:58:24.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.744+0000 7f860a200640 1 -- 172.21.15.28:0/3986374944 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86041f3360 msgr2=0x7f86040b07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:24.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.744+0000 7f860a200640 1 --2- 172.21.15.28:0/3986374944 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86041f3360 0x7f86040b07b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8604072360 tx=0x7f85fc00f520 comp rx=0 tx=0).stop 2024-04-26T20:58:24.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.744+0000 7f860a200640 1 -- 172.21.15.28:0/3986374944 shutdown_connections 2024-04-26T20:58:24.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.744+0000 7f860a200640 1 --2- 172.21.15.28:0/3986374944 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f85e006c920 0x7f85e006ede0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:24.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.744+0000 7f860a200640 1 --2- 172.21.15.28:0/3986374944 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86041f3360 0x7f86040b07b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:24.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.744+0000 7f860a200640 1 --2- 172.21.15.28:0/3986374944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86041426d0 0x7f8604141510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:24.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.744+0000 7f860a200640 1 --2- 172.21.15.28:0/3986374944 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f85e8003f80 0x7f8604145030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:24.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.744+0000 7f860a200640 1 -- 172.21.15.28:0/3986374944 >> 172.21.15.28:0/3986374944 conn(0x7f86040d0a50 msgr2=0x7f86040cb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:24.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.744+0000 7f860a200640 1 -- 172.21.15.28:0/3986374944 shutdown_connections 2024-04-26T20:58:24.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.744+0000 7f860a200640 1 -- 172.21.15.28:0/3986374944 wait complete. 2024-04-26T20:58:24.766 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:58:24.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.974+0000 7f193da00640 1 Processor -- start 2024-04-26T20:58:24.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.974+0000 7f193da00640 1 -- start start 2024-04-26T20:58:24.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.976+0000 7f193da00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1938071e20 0x7f19381610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:24.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.976+0000 7f193da00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1938146440 con 0x7f1938146060 2024-04-26T20:58:24.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.976+0000 7f193da00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f193806f010 con 0x7f1938146670 2024-04-26T20:58:24.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.976+0000 7f193da00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f1938072220 con 0x7f1938071e20 2024-04-26T20:58:24.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.976+0000 7f193d000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1938071e20 0x7f19381610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:24.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.976+0000 7f193d000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1938071e20 0x7f19381610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36414/0 (socket says 172.21.15.28:36414) 2024-04-26T20:58:24.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.976+0000 7f193d000640 1 -- 172.21.15.28:0/3294344866 learned_addr learned my addr 172.21.15.28:0/3294344866 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:24.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.977+0000 7f193d000640 1 -- 172.21.15.28:0/3294344866 >> v1:172.21.15.103:6789/0 conn(0x7f1938146670 legacy=0x7f19380cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:24.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.977+0000 7f193d000640 1 -- 172.21.15.28:0/3294344866 >> v1:172.21.15.28:6789/0 conn(0x7f1938146060 legacy=0x7f19381609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:24.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.977+0000 7f193d000640 1 -- 172.21.15.28:0/3294344866 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19381617a0 con 0x7f1938071e20 2024-04-26T20:58:24.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.978+0000 7f193d000640 1 --2- 172.21.15.28:0/3294344866 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1938071e20 0x7f19381610c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1930009870 tx=0x7f193002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2857c42692eb708 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:24.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.978+0000 7f1937400640 1 -- 172.21.15.28:0/3294344866 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930037070 con 0x7f1938071e20 2024-04-26T20:58:24.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.978+0000 7f1937400640 1 -- 172.21.15.28:0/3294344866 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1930032440 con 0x7f1938071e20 2024-04-26T20:58:24.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.978+0000 7f1937400640 1 -- 172.21.15.28:0/3294344866 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930032810 con 0x7f1938071e20 2024-04-26T20:58:24.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.979+0000 7f193da00640 1 -- 172.21.15.28:0/3294344866 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1938071e20 msgr2=0x7f19381610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:24.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.979+0000 7f193da00640 1 --2- 172.21.15.28:0/3294344866 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1938071e20 0x7f19381610c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f1930009870 tx=0x7f193002a210 comp rx=0 tx=0).stop 2024-04-26T20:58:24.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.979+0000 7f193da00640 1 -- 172.21.15.28:0/3294344866 shutdown_connections 2024-04-26T20:58:24.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.979+0000 7f193da00640 1 --2- 172.21.15.28:0/3294344866 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1938071e20 0x7f19381610c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:24.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.979+0000 7f193da00640 1 -- 172.21.15.28:0/3294344866 >> 172.21.15.28:0/3294344866 conn(0x7f19380d0a50 msgr2=0x7f19380d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:24.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.979+0000 7f193da00640 1 -- 172.21.15.28:0/3294344866 shutdown_connections 2024-04-26T20:58:24.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.979+0000 7f193da00640 1 -- 172.21.15.28:0/3294344866 wait complete. 2024-04-26T20:58:24.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.981+0000 7f193da00640 1 Processor -- start 2024-04-26T20:58:24.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.981+0000 7f193da00640 1 -- start start 2024-04-26T20:58:24.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.982+0000 7f193da00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1938071e20 0x7f1938141580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:24.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.982+0000 7f193da00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1938146060 0x7f19381e6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:24.982 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.982+0000 7f193da00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1938146670 0x7f19380b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:24.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.982+0000 7f193da00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f193806ff60 con 0x7f1938146060 2024-04-26T20:58:24.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.982+0000 7f193da00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f193806f510 con 0x7f1938071e20 2024-04-26T20:58:24.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.982+0000 7f193da00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f19380cb7b0 con 0x7f1938146670 2024-04-26T20:58:24.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.982+0000 7f192f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1938146060 0x7f19381e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:24.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.982+0000 7f192f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1938146060 0x7f19381e6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36416/0 (socket says 172.21.15.28:36416) 2024-04-26T20:58:24.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.982+0000 7f192f400640 1 -- 172.21.15.28:0/2037051344 learned_addr learned my addr 172.21.15.28:0/2037051344 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:24.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.983+0000 7f192f400640 1 -- 172.21.15.28:0/2037051344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1938146670 msgr2=0x7f19380b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:24.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.983+0000 7f192f400640 1 --2- 172.21.15.28:0/2037051344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1938146670 0x7f19380b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:24.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.983+0000 7f192f400640 1 -- 172.21.15.28:0/2037051344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1938071e20 msgr2=0x7f1938141580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:24.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.983+0000 7f192f400640 1 --2- 172.21.15.28:0/2037051344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1938071e20 0x7f1938141580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:24.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.983+0000 7f192f400640 1 -- 172.21.15.28:0/2037051344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19381e7030 con 0x7f1938146060 2024-04-26T20:58:24.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.983+0000 7f192f400640 1 --2- 172.21.15.28:0/2037051344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1938146060 0x7f19381e6680 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f192000ba50 tx=0x7f192000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:24.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.983+0000 7f1934c00640 1 -- 172.21.15.28:0/2037051344 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1920019070 con 0x7f1938146060 2024-04-26T20:58:24.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.983+0000 7f1934c00640 1 -- 172.21.15.28:0/2037051344 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f192000cc20 con 0x7f1938146060 2024-04-26T20:58:24.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.983+0000 7f1934c00640 1 -- 172.21.15.28:0/2037051344 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1920014680 con 0x7f1938146060 2024-04-26T20:58:24.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.983+0000 7f193da00640 1 -- 172.21.15.28:0/2037051344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19381e7310 con 0x7f1938146060 2024-04-26T20:58:24.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.984+0000 7f193da00640 1 -- 172.21.15.28:0/2037051344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19381506a0 con 0x7f1938146060 2024-04-26T20:58:24.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.985+0000 7f193da00640 1 -- 172.21.15.28:0/2037051344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19381508a0 con 0x7f1938146060 2024-04-26T20:58:24.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.986+0000 7f1934c00640 1 -- 172.21.15.28:0/2037051344 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f192001c070 con 0x7f1938146060 2024-04-26T20:58:24.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.987+0000 7f1934c00640 1 --2- 172.21.15.28:0/2037051344 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f191406c8f0 0x7f191406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:24.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.987+0000 7f1934c00640 1 -- 172.21.15.28:0/2037051344 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f192008b910 con 0x7f1938146060 2024-04-26T20:58:24.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.987+0000 7f1937e00640 1 --2- 172.21.15.28:0/2037051344 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f191406c8f0 0x7f191406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:24.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.988+0000 7f1937e00640 1 --2- 172.21.15.28:0/2037051344 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f191406c8f0 0x7f191406edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1928000f60 tx=0x7f19280079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:24.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:24.991+0000 7f1934c00640 1 -- 172.21.15.28:0/2037051344 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f19381508a0 con 0x7f1938146060 2024-04-26T20:58:25.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.199+0000 7f193da00640 1 -- 172.21.15.28:0/2037051344 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f193806a360 con 0x7f1938146060 2024-04-26T20:58:25.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.202+0000 7f1934c00640 1 -- 172.21.15.28:0/2037051344 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16645 (secure 0 0 0) 0x7f1920059650 con 0x7f1938146060 2024-04-26T20:58:25.203 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:25.203 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":34,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:58:10.759775+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:58:07.143890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:58:08.193405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:58:25.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.205+0000 7f193da00640 1 -- 172.21.15.28:0/2037051344 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f191406c8f0 msgr2=0x7f191406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:25.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.205+0000 7f193da00640 1 --2- 172.21.15.28:0/2037051344 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f191406c8f0 0x7f191406edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1928000f60 tx=0x7f19280079d0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.205+0000 7f193da00640 1 -- 172.21.15.28:0/2037051344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1938146060 msgr2=0x7f19381e6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:25.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.205+0000 7f193da00640 1 --2- 172.21.15.28:0/2037051344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1938146060 0x7f19381e6680 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f192000ba50 tx=0x7f192000bf10 comp rx=0 tx=0).stop 2024-04-26T20:58:25.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.205+0000 7f193da00640 1 -- 172.21.15.28:0/2037051344 shutdown_connections 2024-04-26T20:58:25.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.205+0000 7f193da00640 1 --2- 172.21.15.28:0/2037051344 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f191406c8f0 0x7f191406edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.205+0000 7f193da00640 1 --2- 172.21.15.28:0/2037051344 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1938146670 0x7f19380b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.205+0000 7f193da00640 1 --2- 172.21.15.28:0/2037051344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1938146060 0x7f19381e6680 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.205+0000 7f193da00640 1 --2- 172.21.15.28:0/2037051344 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1938071e20 0x7f1938141580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.205+0000 7f193da00640 1 -- 172.21.15.28:0/2037051344 >> 172.21.15.28:0/2037051344 conn(0x7f19380d0a50 msgr2=0x7f1938160280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:25.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.205+0000 7f193da00640 1 -- 172.21.15.28:0/2037051344 shutdown_connections 2024-04-26T20:58:25.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.205+0000 7f193da00640 1 -- 172.21.15.28:0/2037051344 wait complete. 2024-04-26T20:58:25.226 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:58:07.143890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:58:08.193405+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:58:25.226 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:58:25.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.443+0000 7f43c8200640 1 Processor -- start 2024-04-26T20:58:25.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.443+0000 7f43c8200640 1 -- start start 2024-04-26T20:58:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.445+0000 7f43c8200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f43c006fc10 0x7f43c0073f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.445+0000 7f43c8200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f43c0073580 0x7f43c00ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.445+0000 7f43c8200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43c00ced20 con 0x7f43c00ce8d0 2024-04-26T20:58:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.445+0000 7f43c8200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f43c0074490 con 0x7f43c0073580 2024-04-26T20:58:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.445+0000 7f43c8200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f43c015e310 con 0x7f43c006fc10 2024-04-26T20:58:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.446+0000 7f43c4e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f43c0073580 0x7f43c00ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.446+0000 7f43c4e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f43c0073580 0x7f43c00ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36426/0 (socket says 172.21.15.28:36426) 2024-04-26T20:58:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.446+0000 7f43c4e00640 1 -- 172.21.15.28:0/1263501393 learned_addr learned my addr 172.21.15.28:0/1263501393 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:25.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.446+0000 7f43c5800640 1 --2- 172.21.15.28:0/1263501393 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f43c006fc10 0x7f43c0073f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:25.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.446+0000 7f43c4e00640 1 -- 172.21.15.28:0/1263501393 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f43c006fc10 msgr2=0x7f43c0073f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:25.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.446+0000 7f43c4e00640 1 --2- 172.21.15.28:0/1263501393 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f43c006fc10 0x7f43c0073f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.446+0000 7f43c4e00640 1 -- 172.21.15.28:0/1263501393 >> v1:172.21.15.28:6789/0 conn(0x7f43c00ce8d0 legacy=0x7f43c006e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:25.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.446+0000 7f43c4e00640 1 -- 172.21.15.28:0/1263501393 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c0072aa0 con 0x7f43c0073580 2024-04-26T20:58:25.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.447+0000 7f43c4e00640 1 --2- 172.21.15.28:0/1263501393 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43c0073580 0x7f43c00ce390 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009870 tx=0x7f43ac02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cc3cb5ee1c72c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.448+0000 7f43bfe00640 1 -- 172.21.15.28:0/1263501393 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac037070 con 0x7f43c0073580 2024-04-26T20:58:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.448+0000 7f43bfe00640 1 -- 172.21.15.28:0/1263501393 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43ac032440 con 0x7f43c0073580 2024-04-26T20:58:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.448+0000 7f43bfe00640 1 -- 172.21.15.28:0/1263501393 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac032810 con 0x7f43c0073580 2024-04-26T20:58:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.448+0000 7f43c8200640 1 -- 172.21.15.28:0/1263501393 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43c0073580 msgr2=0x7f43c00ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.448+0000 7f43c8200640 1 --2- 172.21.15.28:0/1263501393 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43c0073580 0x7f43c00ce390 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009870 tx=0x7f43ac02a1c0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.448+0000 7f43c8200640 1 -- 172.21.15.28:0/1263501393 shutdown_connections 2024-04-26T20:58:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.448+0000 7f43c8200640 1 --2- 172.21.15.28:0/1263501393 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43c0073580 0x7f43c00ce390 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.448+0000 7f43c8200640 1 --2- 172.21.15.28:0/1263501393 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f43c006fc10 0x7f43c0073f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.448+0000 7f43c8200640 1 -- 172.21.15.28:0/1263501393 >> 172.21.15.28:0/1263501393 conn(0x7f43c006c770 msgr2=0x7f43c006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.448+0000 7f43c8200640 1 -- 172.21.15.28:0/1263501393 shutdown_connections 2024-04-26T20:58:25.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.449+0000 7f43c8200640 1 -- 172.21.15.28:0/1263501393 wait complete. 2024-04-26T20:58:25.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.450+0000 7f43c8200640 1 Processor -- start 2024-04-26T20:58:25.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.451+0000 7f43c8200640 1 -- start start 2024-04-26T20:58:25.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.452+0000 7f43c8200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43c006fc10 0x7f43c0154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:25.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.452+0000 7f43c8200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f43c0073580 0x7f43c00ab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:25.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.452+0000 7f43c8200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c00ce8d0 0x7f43c0149510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:25.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.452+0000 7f43c8200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f43c015dfa0 con 0x7f43c006fc10 2024-04-26T20:58:25.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.452+0000 7f43c8200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f43c0073060 con 0x7f43c00ce8d0 2024-04-26T20:58:25.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.452+0000 7f43c8200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f43c0073a00 con 0x7f43c0073580 2024-04-26T20:58:25.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.452+0000 7f43c5800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43c006fc10 0x7f43c0154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:25.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.452+0000 7f43c5800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43c006fc10 0x7f43c0154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36432/0 (socket says 172.21.15.28:36432) 2024-04-26T20:58:25.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.452+0000 7f43c5800640 1 -- 172.21.15.28:0/802324999 learned_addr learned my addr 172.21.15.28:0/802324999 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:25.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.452+0000 7f43c5800640 1 -- 172.21.15.28:0/802324999 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f43c0073580 msgr2=0x7f43c00ab7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:25.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.452+0000 7f43c4e00640 1 --2- 172.21.15.28:0/802324999 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f43c0073580 0x7f43c00ab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:25.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.452+0000 7f43c5800640 1 --2- 172.21.15.28:0/802324999 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f43c0073580 0x7f43c00ab7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.452+0000 7f43c5800640 1 -- 172.21.15.28:0/802324999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c00ce8d0 msgr2=0x7f43c0149510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:25.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.452+0000 7f43c6200640 1 --2- 172.21.15.28:0/802324999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c00ce8d0 0x7f43c0149510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:25.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.453+0000 7f43c5800640 1 --2- 172.21.15.28:0/802324999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c00ce8d0 0x7f43c0149510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.453+0000 7f43c5800640 1 -- 172.21.15.28:0/802324999 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c0149a50 con 0x7f43c006fc10 2024-04-26T20:58:25.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.453+0000 7f43c6200640 1 --2- 172.21.15.28:0/802324999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c00ce8d0 0x7f43c0149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:25.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.453+0000 7f43c5800640 1 --2- 172.21.15.28:0/802324999 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43c006fc10 0x7f43c0154050 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f43b400d5f0 tx=0x7f43b400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:25.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.453+0000 7f43bd600640 1 -- 172.21.15.28:0/802324999 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b40125f0 con 0x7f43c006fc10 2024-04-26T20:58:25.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.453+0000 7f43bd600640 1 -- 172.21.15.28:0/802324999 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b4019070 con 0x7f43c006fc10 2024-04-26T20:58:25.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.453+0000 7f43bd600640 1 -- 172.21.15.28:0/802324999 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b40129d0 con 0x7f43c006fc10 2024-04-26T20:58:25.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.453+0000 7f43c8200640 1 -- 172.21.15.28:0/802324999 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c0152fb0 con 0x7f43c006fc10 2024-04-26T20:58:25.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.453+0000 7f43c8200640 1 -- 172.21.15.28:0/802324999 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c0153570 con 0x7f43c006fc10 2024-04-26T20:58:25.455 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.455+0000 7f43c8200640 1 -- 172.21.15.28:0/802324999 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c006a360 con 0x7f43c006fc10 2024-04-26T20:58:25.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.455+0000 7f43bd600640 1 -- 172.21.15.28:0/802324999 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f43b4012b70 con 0x7f43c006fc10 2024-04-26T20:58:25.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.456+0000 7f43bd600640 1 --2- 172.21.15.28:0/802324999 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f43a006c920 0x7f43a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:25.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.456+0000 7f43bd600640 1 -- 172.21.15.28:0/802324999 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f43b408bf30 con 0x7f43c006fc10 2024-04-26T20:58:25.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.457+0000 7f43c4e00640 1 --2- 172.21.15.28:0/802324999 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f43a006c920 0x7f43a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:25.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.457+0000 7f43c4e00640 1 --2- 172.21.15.28:0/802324999 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f43a006c920 0x7f43a006ede0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f43ac034000 tx=0x7f43ac02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:25.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.461+0000 7f43bd600640 1 -- 172.21.15.28:0/802324999 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f43b405a360 con 0x7f43c006fc10 2024-04-26T20:58:25.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.673+0000 7f43c8200640 1 -- 172.21.15.28:0/802324999 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f43c006d360 con 0x7f43c006fc10 2024-04-26T20:58:25.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.674+0000 7f43bd600640 1 -- 172.21.15.28:0/802324999 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) ==== 93+0+10 (secure 0 0 0) 0x7f43b4059c60 con 0x7f43c006fc10 2024-04-26T20:58:25.674 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:58:25.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.676+0000 7f43c8200640 1 -- 172.21.15.28:0/802324999 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f43a006c920 msgr2=0x7f43a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:25.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.676+0000 7f43c8200640 1 --2- 172.21.15.28:0/802324999 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f43a006c920 0x7f43a006ede0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f43ac034000 tx=0x7f43ac02e000 comp rx=0 tx=0).stop 2024-04-26T20:58:25.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.676+0000 7f43c8200640 1 -- 172.21.15.28:0/802324999 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43c006fc10 msgr2=0x7f43c0154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:25.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.676+0000 7f43c8200640 1 --2- 172.21.15.28:0/802324999 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43c006fc10 0x7f43c0154050 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f43b400d5f0 tx=0x7f43b400dab0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.677+0000 7f43c8200640 1 -- 172.21.15.28:0/802324999 shutdown_connections 2024-04-26T20:58:25.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.677+0000 7f43c8200640 1 --2- 172.21.15.28:0/802324999 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f43a006c920 0x7f43a006ede0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.677+0000 7f43c8200640 1 --2- 172.21.15.28:0/802324999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f43c00ce8d0 0x7f43c0149510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.677+0000 7f43c8200640 1 --2- 172.21.15.28:0/802324999 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f43c0073580 0x7f43c00ab7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.677+0000 7f43c8200640 1 --2- 172.21.15.28:0/802324999 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43c006fc10 0x7f43c0154050 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.677+0000 7f43c8200640 1 -- 172.21.15.28:0/802324999 >> 172.21.15.28:0/802324999 conn(0x7f43c006c770 msgr2=0x7f43c00b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:25.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.677+0000 7f43c8200640 1 -- 172.21.15.28:0/802324999 shutdown_connections 2024-04-26T20:58:25.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.677+0000 7f43c8200640 1 -- 172.21.15.28:0/802324999 wait complete. 2024-04-26T20:58:25.697 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T20:58:25.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.909+0000 7fcaf6600640 1 Processor -- start 2024-04-26T20:58:25.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.909+0000 7fcaf6600640 1 -- start start 2024-04-26T20:58:25.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.911+0000 7fcaf6600640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fcaf0073f50 0x7fcaf00ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:25.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.911+0000 7fcaf6600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcaf00ce8d0 0x7fcaf006e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:25.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.911+0000 7fcaf6600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaf0074400 con 0x7fcaf006fc10 2024-04-26T20:58:25.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.911+0000 7fcaf6600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fcaf00ced00 con 0x7fcaf00ce8d0 2024-04-26T20:58:25.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.911+0000 7fcaf6600640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fcaf015e310 con 0x7fcaf0073f50 2024-04-26T20:58:25.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.911+0000 7fcaefe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcaf00ce8d0 0x7fcaf006e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:25.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.911+0000 7fcaefe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcaf00ce8d0 0x7fcaf006e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36444/0 (socket says 172.21.15.28:36444) 2024-04-26T20:58:25.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.911+0000 7fcaefe00640 1 -- 172.21.15.28:0/501708660 learned_addr learned my addr 172.21.15.28:0/501708660 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:25.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.912+0000 7fcaeea00640 1 --2- 172.21.15.28:0/501708660 >> v2:172.21.15.103:3300/0 conn(0x7fcaf0073f50 0x7fcaf00ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:25.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.912+0000 7fcaefe00640 1 -- 172.21.15.28:0/501708660 >> v2:172.21.15.103:3300/0 conn(0x7fcaf0073f50 msgr2=0x7fcaf00ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:25.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.912+0000 7fcaefe00640 1 --2- 172.21.15.28:0/501708660 >> v2:172.21.15.103:3300/0 conn(0x7fcaf0073f50 0x7fcaf00ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.912+0000 7fcaefe00640 1 -- 172.21.15.28:0/501708660 >> v1:172.21.15.28:6789/0 conn(0x7fcaf006fc10 legacy=0x7fcaf0073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:25.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.912+0000 7fcaefe00640 1 -- 172.21.15.28:0/501708660 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf0072aa0 con 0x7fcaf00ce8d0 2024-04-26T20:58:25.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.913+0000 7fcaefe00640 1 --2- 172.21.15.28:0/501708660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcaf00ce8d0 0x7fcaf006e610 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0070460 tx=0x7fcae802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f5d1f78f80ae2b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:25.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.913+0000 7fcaee000640 1 -- 172.21.15.28:0/501708660 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae8036070 con 0x7fcaf00ce8d0 2024-04-26T20:58:25.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.913+0000 7fcaee000640 1 -- 172.21.15.28:0/501708660 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae8031600 con 0x7fcaf00ce8d0 2024-04-26T20:58:25.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.913+0000 7fcaee000640 1 -- 172.21.15.28:0/501708660 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae80319d0 con 0x7fcaf00ce8d0 2024-04-26T20:58:25.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.914+0000 7fcaf6600640 1 -- 172.21.15.28:0/501708660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcaf00ce8d0 msgr2=0x7fcaf006e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:25.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.914+0000 7fcaf6600640 1 --2- 172.21.15.28:0/501708660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcaf00ce8d0 0x7fcaf006e610 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0070460 tx=0x7fcae802a6a0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.914+0000 7fcaf6600640 1 -- 172.21.15.28:0/501708660 shutdown_connections 2024-04-26T20:58:25.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.914+0000 7fcaf6600640 1 --2- 172.21.15.28:0/501708660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcaf00ce8d0 0x7fcaf006e610 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.914+0000 7fcaf6600640 1 --2- 172.21.15.28:0/501708660 >> v2:172.21.15.103:3300/0 conn(0x7fcaf0073f50 0x7fcaf00ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.914+0000 7fcaf6600640 1 -- 172.21.15.28:0/501708660 >> 172.21.15.28:0/501708660 conn(0x7fcaf006c770 msgr2=0x7fcaf006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:25.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.914+0000 7fcaf6600640 1 -- 172.21.15.28:0/501708660 shutdown_connections 2024-04-26T20:58:25.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.914+0000 7fcaf6600640 1 -- 172.21.15.28:0/501708660 wait complete. 2024-04-26T20:58:25.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.916+0000 7fcaf6600640 1 Processor -- start 2024-04-26T20:58:25.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.916+0000 7fcaf6600640 1 -- start start 2024-04-26T20:58:25.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.917+0000 7fcaf6600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcaf006fc10 0x7fcaf01f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:25.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.917+0000 7fcaf6600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcaf0073f50 0x7fcaf0153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:25.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.917+0000 7fcaf6600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaf00ce8d0 0x7fcaf0153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:25.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.917+0000 7fcaf6600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf015dfa0 con 0x7fcaf006fc10 2024-04-26T20:58:25.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.917+0000 7fcaf6600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf0073060 con 0x7fcaf00ce8d0 2024-04-26T20:58:25.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.917+0000 7fcaf6600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf00d07d0 con 0x7fcaf0073f50 2024-04-26T20:58:25.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.917+0000 7fcaef400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcaf006fc10 0x7fcaf01f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:25.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.918+0000 7fcaef400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcaf006fc10 0x7fcaf01f7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36450/0 (socket says 172.21.15.28:36450) 2024-04-26T20:58:25.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.918+0000 7fcaef400640 1 -- 172.21.15.28:0/281932730 learned_addr learned my addr 172.21.15.28:0/281932730 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.918+0000 7fcaefe00640 1 --2- 172.21.15.28:0/281932730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaf00ce8d0 0x7fcaf0153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.918+0000 7fcaeea00640 1 --2- 172.21.15.28:0/281932730 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcaf0073f50 0x7fcaf0153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.918+0000 7fcaef400640 1 -- 172.21.15.28:0/281932730 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcaf0073f50 msgr2=0x7fcaf0153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.918+0000 7fcaef400640 1 --2- 172.21.15.28:0/281932730 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcaf0073f50 0x7fcaf0153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.918+0000 7fcaef400640 1 -- 172.21.15.28:0/281932730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaf00ce8d0 msgr2=0x7fcaf0153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.918+0000 7fcaef400640 1 --2- 172.21.15.28:0/281932730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaf00ce8d0 0x7fcaf0153790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.918+0000 7fcaef400640 1 -- 172.21.15.28:0/281932730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf00b5060 con 0x7fcaf006fc10 2024-04-26T20:58:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.918+0000 7fcaefe00640 1 --2- 172.21.15.28:0/281932730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaf00ce8d0 0x7fcaf0153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.918+0000 7fcaeea00640 1 --2- 172.21.15.28:0/281932730 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcaf0073f50 0x7fcaf0153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.918+0000 7fcaef400640 1 --2- 172.21.15.28:0/281932730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcaf006fc10 0x7fcaf01f7e00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcae000c3c0 tx=0x7fcae000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.919+0000 7fcae7400640 1 -- 172.21.15.28:0/281932730 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae00052d0 con 0x7fcaf006fc10 2024-04-26T20:58:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.919+0000 7fcae7400640 1 -- 172.21.15.28:0/281932730 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae0005490 con 0x7fcaf006fc10 2024-04-26T20:58:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.919+0000 7fcae7400640 1 -- 172.21.15.28:0/281932730 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0007260 con 0x7fcaf006fc10 2024-04-26T20:58:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.919+0000 7fcaf6600640 1 -- 172.21.15.28:0/281932730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf00b5340 con 0x7fcaf006fc10 2024-04-26T20:58:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.919+0000 7fcaf6600640 1 -- 172.21.15.28:0/281932730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf00b5900 con 0x7fcaf006fc10 2024-04-26T20:58:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.920+0000 7fcaf6600640 1 -- 172.21.15.28:0/281932730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf006a360 con 0x7fcaf006fc10 2024-04-26T20:58:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.921+0000 7fcae7400640 1 -- 172.21.15.28:0/281932730 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcae0007400 con 0x7fcaf006fc10 2024-04-26T20:58:25.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.922+0000 7fcae7400640 1 --2- 172.21.15.28:0/281932730 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcad006c8f0 0x7fcad006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:25.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.922+0000 7fcae7400640 1 -- 172.21.15.28:0/281932730 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fcae008b180 con 0x7fcaf006fc10 2024-04-26T20:58:25.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.922+0000 7fcaeea00640 1 --2- 172.21.15.28:0/281932730 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcad006c8f0 0x7fcad006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:25.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.923+0000 7fcaeea00640 1 --2- 172.21.15.28:0/281932730 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcad006c8f0 0x7fcad006edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcadc006fd0 tx=0x7fcadc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:25.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:25.926+0000 7fcae7400640 1 -- 172.21.15.28:0/281932730 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcae0059480 con 0x7fcaf006fc10 2024-04-26T20:58:26.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.131+0000 7fcaf6600640 1 -- 172.21.15.28:0/281932730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcaf00b2580 con 0x7fcaf006fc10 2024-04-26T20:58:26.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.131+0000 7fcae7400640 1 -- 172.21.15.28:0/281932730 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) ==== 104+0+11 (secure 0 0 0) 0x7fcae0058d80 con 0x7fcaf006fc10 2024-04-26T20:58:26.132 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-04-26T20:58:26.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.134+0000 7fcaf6600640 1 -- 172.21.15.28:0/281932730 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcad006c8f0 msgr2=0x7fcad006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:26.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.134+0000 7fcaf6600640 1 --2- 172.21.15.28:0/281932730 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcad006c8f0 0x7fcad006edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcadc006fd0 tx=0x7fcadc008040 comp rx=0 tx=0).stop 2024-04-26T20:58:26.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.134+0000 7fcaf6600640 1 -- 172.21.15.28:0/281932730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcaf006fc10 msgr2=0x7fcaf01f7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:26.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.134+0000 7fcaf6600640 1 --2- 172.21.15.28:0/281932730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcaf006fc10 0x7fcaf01f7e00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcae000c3c0 tx=0x7fcae000c880 comp rx=0 tx=0).stop 2024-04-26T20:58:26.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.134+0000 7fcaf6600640 1 -- 172.21.15.28:0/281932730 shutdown_connections 2024-04-26T20:58:26.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.134+0000 7fcaf6600640 1 --2- 172.21.15.28:0/281932730 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcad006c8f0 0x7fcad006edb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:26.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.134+0000 7fcaf6600640 1 --2- 172.21.15.28:0/281932730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaf00ce8d0 0x7fcaf0153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:26.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.134+0000 7fcaf6600640 1 --2- 172.21.15.28:0/281932730 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcaf0073f50 0x7fcaf0153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:26.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.134+0000 7fcaf6600640 1 --2- 172.21.15.28:0/281932730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcaf006fc10 0x7fcaf01f7e00 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:26.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.134+0000 7fcaf6600640 1 -- 172.21.15.28:0/281932730 >> 172.21.15.28:0/281932730 conn(0x7fcaf006c770 msgr2=0x7fcaf015e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:26.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.134+0000 7fcaf6600640 1 -- 172.21.15.28:0/281932730 shutdown_connections 2024-04-26T20:58:26.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.134+0000 7fcaf6600640 1 -- 172.21.15.28:0/281932730 wait complete. 2024-04-26T20:58:26.153 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T20:58:26.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.370+0000 7efdaae00640 1 Processor -- start 2024-04-26T20:58:26.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.370+0000 7efdaae00640 1 -- start start 2024-04-26T20:58:26.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.372+0000 7efdaae00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7efda40d0c70 0x7efda4161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:26.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.372+0000 7efdaae00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efda40b4a30 0x7efda406e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:26.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.372+0000 7efdaae00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efda4161860 con 0x7efda40d1050 2024-04-26T20:58:26.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.372+0000 7efdaae00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7efda40cf2f0 con 0x7efda40d0c70 2024-04-26T20:58:26.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.372+0000 7efdaae00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7efda40cf5d0 con 0x7efda40b4a30 2024-04-26T20:58:26.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.372+0000 7efda9a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7efda40d0c70 0x7efda4161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:26.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.372+0000 7efda9a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7efda40d0c70 0x7efda4161320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36456/0 (socket says 172.21.15.28:36456) 2024-04-26T20:58:26.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.372+0000 7efda9a00640 1 -- 172.21.15.28:0/2891739379 learned_addr learned my addr 172.21.15.28:0/2891739379 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:26.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.372+0000 7efdaa400640 1 --2- 172.21.15.28:0/2891739379 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efda40b4a30 0x7efda406e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:26.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.373+0000 7efda9a00640 1 -- 172.21.15.28:0/2891739379 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efda40b4a30 msgr2=0x7efda406e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:26.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.373+0000 7efda9a00640 1 --2- 172.21.15.28:0/2891739379 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efda40b4a30 0x7efda406e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:26.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.373+0000 7efda9a00640 1 -- 172.21.15.28:0/2891739379 >> v1:172.21.15.103:6789/0 conn(0x7efda40d1050 legacy=0x7efda406df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:26.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.373+0000 7efda9a00640 1 -- 172.21.15.28:0/2891739379 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda40cf8b0 con 0x7efda40d0c70 2024-04-26T20:58:26.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.374+0000 7efda9a00640 1 --2- 172.21.15.28:0/2891739379 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda40d0c70 0x7efda4161320 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efd98009870 tx=0x7efd9802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6608cbe7ff1e903 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:26.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.374+0000 7efda3e00640 1 -- 172.21.15.28:0/2891739379 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98037070 con 0x7efda40d0c70 2024-04-26T20:58:26.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.374+0000 7efda3e00640 1 -- 172.21.15.28:0/2891739379 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd98032440 con 0x7efda40d0c70 2024-04-26T20:58:26.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.374+0000 7efda3e00640 1 -- 172.21.15.28:0/2891739379 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd98032810 con 0x7efda40d0c70 2024-04-26T20:58:26.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.375+0000 7efdaae00640 1 -- 172.21.15.28:0/2891739379 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda40d0c70 msgr2=0x7efda4161320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:26.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.375+0000 7efdaae00640 1 --2- 172.21.15.28:0/2891739379 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda40d0c70 0x7efda4161320 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7efd98009870 tx=0x7efd9802a210 comp rx=0 tx=0).stop 2024-04-26T20:58:26.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.375+0000 7efdaae00640 1 -- 172.21.15.28:0/2891739379 shutdown_connections 2024-04-26T20:58:26.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.375+0000 7efdaae00640 1 --2- 172.21.15.28:0/2891739379 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efda40b4a30 0x7efda406e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:26.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.375+0000 7efdaae00640 1 --2- 172.21.15.28:0/2891739379 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda40d0c70 0x7efda4161320 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:26.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.375+0000 7efdaae00640 1 -- 172.21.15.28:0/2891739379 >> 172.21.15.28:0/2891739379 conn(0x7efda406c530 msgr2=0x7efda406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:26.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.375+0000 7efdaae00640 1 -- 172.21.15.28:0/2891739379 shutdown_connections 2024-04-26T20:58:26.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.375+0000 7efdaae00640 1 -- 172.21.15.28:0/2891739379 wait complete. 2024-04-26T20:58:26.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.377+0000 7efdaae00640 1 Processor -- start 2024-04-26T20:58:26.378 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.377+0000 7efdaae00640 1 -- start start 2024-04-26T20:58:26.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.378+0000 7efdaae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efda40b4a30 0x7efda40aaec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:26.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.378+0000 7efdaae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda40d0c70 0x7efda415cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:26.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.378+0000 7efdaae00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efda40d1050 0x7efda41e7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:26.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.378+0000 7efdaae00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efda4146840 con 0x7efda40d0c70 2024-04-26T20:58:26.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.378+0000 7efdaae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efda40cfe70 con 0x7efda40b4a30 2024-04-26T20:58:26.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.378+0000 7efdaae00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7efda40d14e0 con 0x7efda40d1050 2024-04-26T20:58:26.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.379+0000 7efda9000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda40d0c70 0x7efda415cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:26.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.379+0000 7efda9000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda40d0c70 0x7efda415cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36464/0 (socket says 172.21.15.28:36464) 2024-04-26T20:58:26.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.379+0000 7efda9000640 1 -- 172.21.15.28:0/1850490434 learned_addr learned my addr 172.21.15.28:0/1850490434 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:26.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.379+0000 7efda9a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efda40b4a30 0x7efda40aaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:26.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.379+0000 7efdaa400640 1 --2- 172.21.15.28:0/1850490434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efda40d1050 0x7efda41e7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:26.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.379+0000 7efda9000640 1 -- 172.21.15.28:0/1850490434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efda40d1050 msgr2=0x7efda41e7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:26.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.379+0000 7efda9000640 1 --2- 172.21.15.28:0/1850490434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efda40d1050 0x7efda41e7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:26.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.379+0000 7efda9000640 1 -- 172.21.15.28:0/1850490434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efda40b4a30 msgr2=0x7efda40aaec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:26.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.379+0000 7efda9000640 1 --2- 172.21.15.28:0/1850490434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efda40b4a30 0x7efda40aaec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:26.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.379+0000 7efda9000640 1 -- 172.21.15.28:0/1850490434 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efda40ab510 con 0x7efda40d0c70 2024-04-26T20:58:26.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.380+0000 7efda9000640 1 --2- 172.21.15.28:0/1850490434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda40d0c70 0x7efda415cde0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efd9000b3f0 tx=0x7efd9000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:26.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.380+0000 7efda1600640 1 -- 172.21.15.28:0/1850490434 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90015020 con 0x7efda40d0c70 2024-04-26T20:58:26.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.380+0000 7efda1600640 1 -- 172.21.15.28:0/1850490434 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd90011830 con 0x7efda40d0c70 2024-04-26T20:58:26.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.380+0000 7efda1600640 1 -- 172.21.15.28:0/1850490434 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd90011c20 con 0x7efda40d0c70 2024-04-26T20:58:26.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.380+0000 7efdaae00640 1 -- 172.21.15.28:0/1850490434 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efda41e8dd0 con 0x7efda40d0c70 2024-04-26T20:58:26.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.380+0000 7efdaae00640 1 -- 172.21.15.28:0/1850490434 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efda41e9390 con 0x7efda40d0c70 2024-04-26T20:58:26.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.381+0000 7efdaae00640 1 -- 172.21.15.28:0/1850490434 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efda406a2c0 con 0x7efda40d0c70 2024-04-26T20:58:26.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.382+0000 7efda1600640 1 -- 172.21.15.28:0/1850490434 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd90018020 con 0x7efda40d0c70 2024-04-26T20:58:26.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.383+0000 7efda1600640 1 --2- 172.21.15.28:0/1850490434 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efd8406c8f0 0x7efd8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:26.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.384+0000 7efda1600640 1 -- 172.21.15.28:0/1850490434 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7efd9008b790 con 0x7efda40d0c70 2024-04-26T20:58:26.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.384+0000 7efda9a00640 1 --2- 172.21.15.28:0/1850490434 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efd8406c8f0 0x7efd8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:26.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.385+0000 7efda9a00640 1 --2- 172.21.15.28:0/1850490434 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efd8406c8f0 0x7efd8406edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efd98034000 tx=0x7efd9802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:26.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.388+0000 7efda1600640 1 -- 172.21.15.28:0/1850490434 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efd9001f030 con 0x7efda40d0c70 2024-04-26T20:58:26.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.590+0000 7efdaae00640 1 -- 172.21.15.28:0/1850490434 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efda40aaca0 con 0x7efda40d0c70 2024-04-26T20:58:26.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.591+0000 7efda1600640 1 -- 172.21.15.28:0/1850490434 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) ==== 100+0+11 (secure 0 0 0) 0x7efd90059b20 con 0x7efda40d0c70 2024-04-26T20:58:26.593 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-04-26T20:58:26.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.594+0000 7efdaae00640 1 -- 172.21.15.28:0/1850490434 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efd8406c8f0 msgr2=0x7efd8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:26.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.594+0000 7efdaae00640 1 --2- 172.21.15.28:0/1850490434 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efd8406c8f0 0x7efd8406edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efd98034000 tx=0x7efd9802e000 comp rx=0 tx=0).stop 2024-04-26T20:58:26.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.594+0000 7efdaae00640 1 -- 172.21.15.28:0/1850490434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda40d0c70 msgr2=0x7efda415cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:26.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.594+0000 7efdaae00640 1 --2- 172.21.15.28:0/1850490434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda40d0c70 0x7efda415cde0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efd9000b3f0 tx=0x7efd9000b8b0 comp rx=0 tx=0).stop 2024-04-26T20:58:26.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.594+0000 7efdaae00640 1 -- 172.21.15.28:0/1850490434 shutdown_connections 2024-04-26T20:58:26.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.594+0000 7efdaae00640 1 --2- 172.21.15.28:0/1850490434 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efd8406c8f0 0x7efd8406edb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:26.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.594+0000 7efdaae00640 1 --2- 172.21.15.28:0/1850490434 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efda40d1050 0x7efda41e7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:26.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.594+0000 7efdaae00640 1 --2- 172.21.15.28:0/1850490434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efda40d0c70 0x7efda415cde0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:26.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.594+0000 7efdaae00640 1 --2- 172.21.15.28:0/1850490434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efda40b4a30 0x7efda40aaec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:26.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.594+0000 7efdaae00640 1 -- 172.21.15.28:0/1850490434 >> 172.21.15.28:0/1850490434 conn(0x7efda406c530 msgr2=0x7efda41609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:26.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.594+0000 7efdaae00640 1 -- 172.21.15.28:0/1850490434 shutdown_connections 2024-04-26T20:58:26.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.594+0000 7efdaae00640 1 -- 172.21.15.28:0/1850490434 wait complete. 2024-04-26T20:58:26.616 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:58:26.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.819+0000 7fca17e00640 1 Processor -- start 2024-04-26T20:58:26.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.819+0000 7fca17e00640 1 -- start start 2024-04-26T20:58:26.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.821+0000 7fca17e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fca18146060 0x7fca18071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:26.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.821+0000 7fca17e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fca181610c0 0x7fca180cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:26.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.821+0000 7fca17e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca18161530 con 0x7fca18146670 2024-04-26T20:58:26.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.821+0000 7fca17e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fca1806f510 con 0x7fca181610c0 2024-04-26T20:58:26.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.821+0000 7fca17e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fca18146440 con 0x7fca18146060 2024-04-26T20:58:26.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.821+0000 7fca16000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fca18146670 0x7fca181609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37106/0 (socket says 172.21.15.28:37106) 2024-04-26T20:58:26.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.821+0000 7fca16000640 1 -- 172.21.15.28:0/2091035571 learned_addr learned my addr 172.21.15.28:0/2091035571 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:26.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.821+0000 7fca17400640 1 --2- 172.21.15.28:0/2091035571 >> v2:172.21.15.103:3300/0 conn(0x7fca181610c0 0x7fca180cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:26.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.821+0000 7fca16a00640 1 --2- 172.21.15.28:0/2091035571 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fca18146060 0x7fca18071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:26.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.822+0000 7fca15600640 1 -- 172.21.15.28:0/2091035571 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1081982168 0 0) 0x7fca18161530 con 0x7fca18146670 2024-04-26T20:58:26.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.822+0000 7fca15600640 1 -- 172.21.15.28:0/2091035571 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9fc003520 con 0x7fca18146670 2024-04-26T20:58:26.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.822+0000 7fca15600640 1 -- 172.21.15.28:0/2091035571 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2892970862 0 0) 0x7fc9fc003520 con 0x7fca18146670 2024-04-26T20:58:26.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.822+0000 7fca15600640 1 -- 172.21.15.28:0/2091035571 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fca18146060 msgr2=0x7fca18071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:26.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.822+0000 7fca15600640 1 --2- 172.21.15.28:0/2091035571 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fca18146060 0x7fca18071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:26.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.822+0000 7fca15600640 1 -- 172.21.15.28:0/2091035571 >> v2:172.21.15.103:3300/0 conn(0x7fca181610c0 msgr2=0x7fca180cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:26.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.822+0000 7fca15600640 1 --2- 172.21.15.28:0/2091035571 >> v2:172.21.15.103:3300/0 conn(0x7fca181610c0 0x7fca180cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:26.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.822+0000 7fca15600640 1 -- 172.21.15.28:0/2091035571 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1806f010 con 0x7fca18146670 2024-04-26T20:58:26.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.822+0000 7fca15600640 1 -- 172.21.15.28:0/2091035571 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fca04001670 con 0x7fca18146670 2024-04-26T20:58:26.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.822+0000 7fca15600640 1 -- 172.21.15.28:0/2091035571 >> v1:172.21.15.28:6789/0 conn(0x7fca18146670 legacy=0x7fca181609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:26.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.822+0000 7fca15600640 1 --2- 172.21.15.28:0/2091035571 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9fc0039b0 0x7fc9fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:26.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.822+0000 7fca15600640 1 --2- 172.21.15.28:0/2091035571 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fca18146060 0x7fc9fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:26.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.822+0000 7fca15600640 1 --2- 172.21.15.28:0/2091035571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9fc004d40 0x7fc9fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:26.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.822+0000 7fca15600640 1 -- 172.21.15.28:0/2091035571 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fca1806f510 con 0x7fc9fc0039b0 2024-04-26T20:58:26.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.822+0000 7fca15600640 1 -- 172.21.15.28:0/2091035571 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca18146440 con 0x7fc9fc004d40 2024-04-26T20:58:26.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.822+0000 7fca15600640 1 -- 172.21.15.28:0/2091035571 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc9fc00b9c0 con 0x7fca18146060 2024-04-26T20:58:26.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.822+0000 7fca16a00640 1 --2- 172.21.15.28:0/2091035571 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9fc0039b0 0x7fc9fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:26.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.823+0000 7fca16000640 1 --2- 172.21.15.28:0/2091035571 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fca18146060 0x7fc9fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:26.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.823+0000 7fca17400640 1 --2- 172.21.15.28:0/2091035571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9fc004d40 0x7fc9fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:26.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.823+0000 7fca16a00640 1 -- 172.21.15.28:0/2091035571 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fca18146060 msgr2=0x7fc9fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:26.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.823+0000 7fca16a00640 1 --2- 172.21.15.28:0/2091035571 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fca18146060 0x7fc9fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:26.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.823+0000 7fca16a00640 1 -- 172.21.15.28:0/2091035571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9fc004d40 msgr2=0x7fc9fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:26.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.823+0000 7fca16a00640 1 --2- 172.21.15.28:0/2091035571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9fc004d40 0x7fc9fc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:26.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.823+0000 7fca16a00640 1 -- 172.21.15.28:0/2091035571 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9fc003520 con 0x7fc9fc0039b0 2024-04-26T20:58:26.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.823+0000 7fca16000640 1 --2- 172.21.15.28:0/2091035571 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fca18146060 0x7fc9fc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:26.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.824+0000 7fca16a00640 1 --2- 172.21.15.28:0/2091035571 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9fc0039b0 0x7fc9fc003dd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fca08009ac0 tx=0x7fc9fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16509695b082617 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:26.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.824+0000 7fca15600640 1 -- 172.21.15.28:0/2091035571 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08036070 con 0x7fc9fc0039b0 2024-04-26T20:58:26.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.824+0000 7fca15600640 1 -- 172.21.15.28:0/2091035571 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca08031400 con 0x7fc9fc0039b0 2024-04-26T20:58:26.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.824+0000 7fca15600640 1 -- 172.21.15.28:0/2091035571 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca08009690 con 0x7fc9fc0039b0 2024-04-26T20:58:26.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.825+0000 7fca17e00640 1 -- 172.21.15.28:0/2091035571 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9fc0039b0 msgr2=0x7fc9fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:26.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.825+0000 7fca17e00640 1 --2- 172.21.15.28:0/2091035571 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9fc0039b0 0x7fc9fc003dd0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fca08009ac0 tx=0x7fc9fc005830 comp rx=0 tx=0).stop 2024-04-26T20:58:26.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.825+0000 7fca17400640 1 -- 172.21.15.28:0/2091035571 reap_dead start 2024-04-26T20:58:26.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.825+0000 7fca17e00640 1 -- 172.21.15.28:0/2091035571 shutdown_connections 2024-04-26T20:58:26.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.825+0000 7fca17e00640 1 -- 172.21.15.28:0/2091035571 >> 172.21.15.28:0/2091035571 conn(0x7fca180d0a50 msgr2=0x7fca180d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:26.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.825+0000 7fca17e00640 1 -- 172.21.15.28:0/2091035571 shutdown_connections 2024-04-26T20:58:26.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.825+0000 7fca17e00640 1 -- 172.21.15.28:0/2091035571 wait complete. 2024-04-26T20:58:26.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.827+0000 7fca17e00640 1 Processor -- start 2024-04-26T20:58:26.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.827+0000 7fca17e00640 1 -- start start 2024-04-26T20:58:26.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.828+0000 7fca17e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc9fc0039b0 0x7fca1813acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:26.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.828+0000 7fca17e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca1813c070 0x7fca1813cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:26.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.828+0000 7fca17e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca1813c4f0 0x7fca1813d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:26.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.828+0000 7fca17e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fca1806ff60 con 0x7fca1813c4f0 2024-04-26T20:58:26.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.828+0000 7fca17e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fca1806fc80 con 0x7fca1813c070 2024-04-26T20:58:26.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.828+0000 7fca17e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fca18071e20 con 0x7fc9fc0039b0 2024-04-26T20:58:26.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.828+0000 7fca17400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca1813c4f0 0x7fca1813d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:26.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.828+0000 7fca17400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca1813c4f0 0x7fca1813d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36474/0 (socket says 172.21.15.28:36474) 2024-04-26T20:58:26.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.828+0000 7fca17400640 1 -- 172.21.15.28:0/3939439832 learned_addr learned my addr 172.21.15.28:0/3939439832 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:26.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.829+0000 7fca16a00640 1 --2- 172.21.15.28:0/3939439832 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc9fc0039b0 0x7fca1813acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:26.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.829+0000 7fca16000640 1 --2- 172.21.15.28:0/3939439832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca1813c070 0x7fca1813cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:26.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.829+0000 7fca17400640 1 -- 172.21.15.28:0/3939439832 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc9fc0039b0 msgr2=0x7fca1813acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:26.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.829+0000 7fca17400640 1 --2- 172.21.15.28:0/3939439832 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc9fc0039b0 0x7fca1813acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:26.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.829+0000 7fca17400640 1 -- 172.21.15.28:0/3939439832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca1813c070 msgr2=0x7fca1813cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:26.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.829+0000 7fca17400640 1 --2- 172.21.15.28:0/3939439832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca1813c070 0x7fca1813cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:26.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.829+0000 7fca17400640 1 -- 172.21.15.28:0/3939439832 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca180b4f00 con 0x7fca1813c4f0 2024-04-26T20:58:26.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.829+0000 7fca17400640 1 --2- 172.21.15.28:0/3939439832 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca1813c4f0 0x7fca1813d250 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fca18072360 tx=0x7fca1000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:26.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.829+0000 7fca0ea00640 1 -- 172.21.15.28:0/3939439832 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca10016420 con 0x7fca1813c4f0 2024-04-26T20:58:26.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.830+0000 7fca0ea00640 1 -- 172.21.15.28:0/3939439832 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca1001b070 con 0x7fca1813c4f0 2024-04-26T20:58:26.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.830+0000 7fca0ea00640 1 -- 172.21.15.28:0/3939439832 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca10016820 con 0x7fca1813c4f0 2024-04-26T20:58:26.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.830+0000 7fca17e00640 1 -- 172.21.15.28:0/3939439832 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca181435c0 con 0x7fca1813c4f0 2024-04-26T20:58:26.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.830+0000 7fca17e00640 1 -- 172.21.15.28:0/3939439832 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca18143b80 con 0x7fca1813c4f0 2024-04-26T20:58:26.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.831+0000 7fca17e00640 1 -- 172.21.15.28:0/3939439832 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca1806a360 con 0x7fca1813c4f0 2024-04-26T20:58:26.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.833+0000 7fca0ea00640 1 -- 172.21.15.28:0/3939439832 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca10005d10 con 0x7fca1813c4f0 2024-04-26T20:58:26.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.834+0000 7fca0ea00640 1 --2- 172.21.15.28:0/3939439832 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc9f806c920 0x7fc9f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:26.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.834+0000 7fca0ea00640 1 -- 172.21.15.28:0/3939439832 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fca1008c3d0 con 0x7fca1813c4f0 2024-04-26T20:58:26.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.834+0000 7fca16a00640 1 --2- 172.21.15.28:0/3939439832 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc9f806c920 0x7fc9f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:26.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.835+0000 7fca16a00640 1 --2- 172.21.15.28:0/3939439832 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc9f806c920 0x7fc9f806ede0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fca08033000 tx=0x7fca08008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:26.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:26.839+0000 7fca0ea00640 1 -- 172.21.15.28:0/3939439832 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fca1005a7f0 con 0x7fca1813c4f0 2024-04-26T20:58:27.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.076+0000 7fca17e00640 1 -- 172.21.15.28:0/3939439832 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca180b50d0 con 0x7fca1813c4f0 2024-04-26T20:58:27.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.077+0000 7fca0ea00640 1 -- 172.21.15.28:0/3939439832 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7fca1005a0f0 con 0x7fca1813c4f0 2024-04-26T20:58:27.078 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:27.079 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/1364678224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1364678224},{"type":"v1","addr":"172.21.15.103:6834","nonce":1364678224}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4081926466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":4081926466},{"type":"v1","addr":"172.21.15.28:6837","nonce":4081926466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5175,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/1370252546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":1370252546},{"type":"v1","addr":"172.21.15.28:6835","nonce":1370252546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:58:09.226353+0000","modified":"2024-04-26T20:58:12.319110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5184":{"gid":5184,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6835/1781724684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":1781724684},{"type":"v1","addr":"172.21.15.103:6835","nonce":1781724684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5184,"qdb_cluster":[5184]},"id":2}]} 2024-04-26T20:58:27.079 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-04-26T20:58:27.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.080+0000 7fca17e00640 1 -- 172.21.15.28:0/3939439832 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc9f806c920 msgr2=0x7fc9f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:27.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.080+0000 7fca17e00640 1 --2- 172.21.15.28:0/3939439832 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc9f806c920 0x7fc9f806ede0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fca08033000 tx=0x7fca08008030 comp rx=0 tx=0).stop 2024-04-26T20:58:27.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.080+0000 7fca17e00640 1 -- 172.21.15.28:0/3939439832 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca1813c4f0 msgr2=0x7fca1813d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:27.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.080+0000 7fca17e00640 1 --2- 172.21.15.28:0/3939439832 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca1813c4f0 0x7fca1813d250 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fca18072360 tx=0x7fca1000f520 comp rx=0 tx=0).stop 2024-04-26T20:58:27.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.080+0000 7fca17e00640 1 -- 172.21.15.28:0/3939439832 shutdown_connections 2024-04-26T20:58:27.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.080+0000 7fca17e00640 1 --2- 172.21.15.28:0/3939439832 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc9f806c920 0x7fc9f806ede0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:27.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.080+0000 7fca17e00640 1 --2- 172.21.15.28:0/3939439832 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fca1813c4f0 0x7fca1813d250 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:27.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.080+0000 7fca17e00640 1 --2- 172.21.15.28:0/3939439832 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fca1813c070 0x7fca1813cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:27.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.080+0000 7fca17e00640 1 --2- 172.21.15.28:0/3939439832 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc9fc0039b0 0x7fca1813acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:27.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.080+0000 7fca17e00640 1 -- 172.21.15.28:0/3939439832 >> 172.21.15.28:0/3939439832 conn(0x7fca180d0a50 msgr2=0x7fca180b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:27.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.080+0000 7fca17e00640 1 -- 172.21.15.28:0/3939439832 shutdown_connections 2024-04-26T20:58:27.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.080+0000 7fca17e00640 1 -- 172.21.15.28:0/3939439832 wait complete. 2024-04-26T20:58:27.101 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:58:27.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.304+0000 7fb4ff200640 1 Processor -- start 2024-04-26T20:58:27.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.304+0000 7fb4ff200640 1 -- start start 2024-04-26T20:58:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.305+0000 7fb4ff200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb5000d4700 0x7fb5000d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.306+0000 7fb4ff200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fb5000d5370 0x7fb5000b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.306+0000 7fb4ff200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5000d0710 con 0x7fb50015f1b0 2024-04-26T20:58:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.306+0000 7fb4ff200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb5000d57e0 con 0x7fb5000d4700 2024-04-26T20:58:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.306+0000 7fb4ff200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fb5000cff00 con 0x7fb5000d5370 2024-04-26T20:58:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.306+0000 7fb4f7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb5000d4700 0x7fb5000d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.306+0000 7fb4f7e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb5000d4700 0x7fb5000d4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36480/0 (socket says 172.21.15.28:36480) 2024-04-26T20:58:27.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.306+0000 7fb4f7e00640 1 -- 172.21.15.28:0/3967498695 learned_addr learned my addr 172.21.15.28:0/3967498695 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:27.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.306+0000 7fb4f7e00640 1 -- 172.21.15.28:0/3967498695 >> v2:172.21.15.103:3300/0 conn(0x7fb5000d5370 msgr2=0x7fb5000b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:27.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.306+0000 7fb4f7e00640 1 --2- 172.21.15.28:0/3967498695 >> v2:172.21.15.103:3300/0 conn(0x7fb5000d5370 0x7fb5000b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:27.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.306+0000 7fb4f7e00640 1 -- 172.21.15.28:0/3967498695 >> v1:172.21.15.103:6789/0 conn(0x7fb50015f1b0 legacy=0x7fb5000cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:27.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.306+0000 7fb4f7e00640 1 -- 172.21.15.28:0/3967498695 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5000d4b40 con 0x7fb5000d4700 2024-04-26T20:58:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.308+0000 7fb4f7e00640 1 --2- 172.21.15.28:0/3967498695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5000d4700 0x7fb5000d4e30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb5000b7a90 tx=0x7fb4e802a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29eb285cf3a1f02e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.308+0000 7fb4fd400640 1 -- 172.21.15.28:0/3967498695 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8037070 con 0x7fb5000d4700 2024-04-26T20:58:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.308+0000 7fb4fd400640 1 -- 172.21.15.28:0/3967498695 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4e8032440 con 0x7fb5000d4700 2024-04-26T20:58:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.308+0000 7fb4fd400640 1 -- 172.21.15.28:0/3967498695 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8032810 con 0x7fb5000d4700 2024-04-26T20:58:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.308+0000 7fb4ff200640 1 -- 172.21.15.28:0/3967498695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5000d4700 msgr2=0x7fb5000d4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.308+0000 7fb4ff200640 1 --2- 172.21.15.28:0/3967498695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5000d4700 0x7fb5000d4e30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb5000b7a90 tx=0x7fb4e802a290 comp rx=0 tx=0).stop 2024-04-26T20:58:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.309+0000 7fb4ff200640 1 -- 172.21.15.28:0/3967498695 shutdown_connections 2024-04-26T20:58:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.309+0000 7fb4ff200640 1 --2- 172.21.15.28:0/3967498695 >> v2:172.21.15.103:3300/0 conn(0x7fb5000d5370 0x7fb5000b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.309+0000 7fb4ff200640 1 --2- 172.21.15.28:0/3967498695 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5000d4700 0x7fb5000d4e30 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.309+0000 7fb4ff200640 1 -- 172.21.15.28:0/3967498695 >> 172.21.15.28:0/3967498695 conn(0x7fb50006c770 msgr2=0x7fb50006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.309+0000 7fb4ff200640 1 -- 172.21.15.28:0/3967498695 shutdown_connections 2024-04-26T20:58:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.309+0000 7fb4ff200640 1 -- 172.21.15.28:0/3967498695 wait complete. 2024-04-26T20:58:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.310+0000 7fb4ff200640 1 Processor -- start 2024-04-26T20:58:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.310+0000 7fb4ff200640 1 -- start start 2024-04-26T20:58:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.311+0000 7fb4ff200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5000d4700 0x7fb500159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.311+0000 7fb4ff200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb5000d5370 0x7fb50015f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.311+0000 7fb4ff200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50015f1b0 0x7fb50015fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.311+0000 7fb4ff200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb5000d0430 con 0x7fb5000d4700 2024-04-26T20:58:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.311+0000 7fb4ff200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb5000d4270 con 0x7fb50015f1b0 2024-04-26T20:58:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.311+0000 7fb4ff200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb5000d3f00 con 0x7fb5000d5370 2024-04-26T20:58:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.312+0000 7fb4fde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5000d4700 0x7fb500159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.312+0000 7fb4fde00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5000d4700 0x7fb500159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36490/0 (socket says 172.21.15.28:36490) 2024-04-26T20:58:27.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.312+0000 7fb4fde00640 1 -- 172.21.15.28:0/1231012463 learned_addr learned my addr 172.21.15.28:0/1231012463 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:27.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.312+0000 7fb4fe800640 1 --2- 172.21.15.28:0/1231012463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50015f1b0 0x7fb50015fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:27.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.312+0000 7fb4fde00640 1 -- 172.21.15.28:0/1231012463 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb5000d5370 msgr2=0x7fb50015f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:27.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.312+0000 7fb4fde00640 1 --2- 172.21.15.28:0/1231012463 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb5000d5370 0x7fb50015f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:27.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.312+0000 7fb4fde00640 1 -- 172.21.15.28:0/1231012463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50015f1b0 msgr2=0x7fb50015fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:27.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.312+0000 7fb4fde00640 1 --2- 172.21.15.28:0/1231012463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50015f1b0 0x7fb50015fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:27.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.312+0000 7fb4fde00640 1 -- 172.21.15.28:0/1231012463 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5000c9620 con 0x7fb5000d4700 2024-04-26T20:58:27.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.312+0000 7fb4fe800640 1 --2- 172.21.15.28:0/1231012463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50015f1b0 0x7fb50015fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:27.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.312+0000 7fb4fde00640 1 --2- 172.21.15.28:0/1231012463 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5000d4700 0x7fb500159340 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000b3f0 tx=0x7fb4f000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:27.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.313+0000 7fb4f6000640 1 -- 172.21.15.28:0/1231012463 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0015020 con 0x7fb5000d4700 2024-04-26T20:58:27.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.313+0000 7fb4f6000640 1 -- 172.21.15.28:0/1231012463 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4f0011830 con 0x7fb5000d4700 2024-04-26T20:58:27.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.313+0000 7fb4ff200640 1 -- 172.21.15.28:0/1231012463 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5000c98c0 con 0x7fb5000d4700 2024-04-26T20:58:27.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.313+0000 7fb4f6000640 1 -- 172.21.15.28:0/1231012463 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0011c20 con 0x7fb5000d4700 2024-04-26T20:58:27.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.313+0000 7fb4ff200640 1 -- 172.21.15.28:0/1231012463 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5000c9e80 con 0x7fb5000d4700 2024-04-26T20:58:27.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.314+0000 7fb4ff200640 1 -- 172.21.15.28:0/1231012463 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb50006a360 con 0x7fb5000d4700 2024-04-26T20:58:27.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.315+0000 7fb4f6000640 1 -- 172.21.15.28:0/1231012463 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb4f0018020 con 0x7fb5000d4700 2024-04-26T20:58:27.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.317+0000 7fb4f6000640 1 --2- 172.21.15.28:0/1231012463 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb4dc06c8f0 0x7fb4dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:27.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.317+0000 7fb4f6000640 1 -- 172.21.15.28:0/1231012463 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fb4f008b750 con 0x7fb5000d4700 2024-04-26T20:58:27.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.317+0000 7fb4f7e00640 1 --2- 172.21.15.28:0/1231012463 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb4dc06c8f0 0x7fb4dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:27.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.318+0000 7fb4f7e00640 1 --2- 172.21.15.28:0/1231012463 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb4dc06c8f0 0x7fb4dc06edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8034000 tx=0x7fb4e802c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:27.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.321+0000 7fb4f6000640 1 -- 172.21.15.28:0/1231012463 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb4f001f030 con 0x7fb5000d4700 2024-04-26T20:58:27.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.529+0000 7fb4ff200640 1 -- 172.21.15.28:0/1231012463 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5000d4f60 con 0x7fb5000d4700 2024-04-26T20:58:27.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.532+0000 7fb4f6000640 1 -- 172.21.15.28:0/1231012463 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16645 (secure 0 0 0) 0x7fb4f00172c0 con 0x7fb5000d4700 2024-04-26T20:58:27.534 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:27.534 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":34,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:58:10.759775+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:58:07.143890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:58:08.193405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:58:27.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.535+0000 7fb4ff200640 1 -- 172.21.15.28:0/1231012463 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb4dc06c8f0 msgr2=0x7fb4dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:27.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.535+0000 7fb4ff200640 1 --2- 172.21.15.28:0/1231012463 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb4dc06c8f0 0x7fb4dc06edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8034000 tx=0x7fb4e802c000 comp rx=0 tx=0).stop 2024-04-26T20:58:27.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.535+0000 7fb4ff200640 1 -- 172.21.15.28:0/1231012463 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5000d4700 msgr2=0x7fb500159340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:27.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.535+0000 7fb4ff200640 1 --2- 172.21.15.28:0/1231012463 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5000d4700 0x7fb500159340 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fb4f000b3f0 tx=0x7fb4f000b8b0 comp rx=0 tx=0).stop 2024-04-26T20:58:27.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.535+0000 7fb4ff200640 1 -- 172.21.15.28:0/1231012463 shutdown_connections 2024-04-26T20:58:27.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.535+0000 7fb4ff200640 1 --2- 172.21.15.28:0/1231012463 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb4dc06c8f0 0x7fb4dc06edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:27.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.535+0000 7fb4ff200640 1 --2- 172.21.15.28:0/1231012463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb50015f1b0 0x7fb50015fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:27.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.535+0000 7fb4ff200640 1 --2- 172.21.15.28:0/1231012463 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb5000d5370 0x7fb50015f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:27.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.535+0000 7fb4ff200640 1 --2- 172.21.15.28:0/1231012463 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb5000d4700 0x7fb500159340 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:27.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.535+0000 7fb4ff200640 1 -- 172.21.15.28:0/1231012463 >> 172.21.15.28:0/1231012463 conn(0x7fb50006c770 msgr2=0x7fb5000b80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:27.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.535+0000 7fb4ff200640 1 -- 172.21.15.28:0/1231012463 shutdown_connections 2024-04-26T20:58:27.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.535+0000 7fb4ff200640 1 -- 172.21.15.28:0/1231012463 wait complete. 2024-04-26T20:58:27.555 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T20:58:27.555 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:58:27.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.764+0000 7fe6dd800640 1 Processor -- start 2024-04-26T20:58:27.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.764+0000 7fe6dd800640 1 -- start start 2024-04-26T20:58:27.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.766+0000 7fe6dd800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6d8146060 0x7fe6d8071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:27.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.766+0000 7fe6dd800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6d8161530 con 0x7fe6d8146670 2024-04-26T20:58:27.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.766+0000 7fe6dd800640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6d8146440 con 0x7fe6d81610c0 2024-04-26T20:58:27.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.766+0000 7fe6dd800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d806f510 con 0x7fe6d8146060 2024-04-26T20:58:27.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.766+0000 7fe6d7400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe6d8146670 0x7fe6d81609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37116/0 (socket says 172.21.15.28:37116) 2024-04-26T20:58:27.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.766+0000 7fe6d7400640 1 -- 172.21.15.28:0/792263231 learned_addr learned my addr 172.21.15.28:0/792263231 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:27.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.766+0000 7fe6d7e00640 1 --2- 172.21.15.28:0/792263231 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6d8146060 0x7fe6d8071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:27.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.767+0000 7fe6d6a00640 1 -- 172.21.15.28:0/792263231 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1969528165 0 0) 0x7fe6d8161530 con 0x7fe6d8146670 2024-04-26T20:58:27.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.767+0000 7fe6d6a00640 1 -- 172.21.15.28:0/792263231 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe6bc003520 con 0x7fe6d8146670 2024-04-26T20:58:27.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.767+0000 7fe6d6a00640 1 -- 172.21.15.28:0/792263231 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3945421347 0 0) 0x7fe6bc003520 con 0x7fe6d8146670 2024-04-26T20:58:27.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.767+0000 7fe6d6a00640 1 -- 172.21.15.28:0/792263231 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6d8146060 msgr2=0x7fe6d8071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:27.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.767+0000 7fe6d6a00640 1 --2- 172.21.15.28:0/792263231 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6d8146060 0x7fe6d8071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:27.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.767+0000 7fe6d6a00640 1 -- 172.21.15.28:0/792263231 >> v1:172.21.15.103:6789/0 conn(0x7fe6d81610c0 legacy=0x7fe6d80cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:27.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.767+0000 7fe6d6a00640 1 -- 172.21.15.28:0/792263231 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d806f010 con 0x7fe6d8146670 2024-04-26T20:58:27.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.767+0000 7fe6d6a00640 1 -- 172.21.15.28:0/792263231 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fe6c4001670 con 0x7fe6d8146670 2024-04-26T20:58:27.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.767+0000 7fe6d6a00640 1 -- 172.21.15.28:0/792263231 >> v1:172.21.15.28:6789/0 conn(0x7fe6d8146670 legacy=0x7fe6d81609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:27.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.767+0000 7fe6d6a00640 1 --2- 172.21.15.28:0/792263231 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6bc0039e0 0x7fe6bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:27.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.767+0000 7fe6d6a00640 1 --2- 172.21.15.28:0/792263231 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6d8146060 0x7fe6bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.767+0000 7fe6d6a00640 1 --2- 172.21.15.28:0/792263231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6bc004d40 0x7fe6bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.767+0000 7fe6d6a00640 1 -- 172.21.15.28:0/792263231 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d806f510 con 0x7fe6bc0039e0 2024-04-26T20:58:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.767+0000 7fe6d6a00640 1 -- 172.21.15.28:0/792263231 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe6bc00ff00 con 0x7fe6bc004d40 2024-04-26T20:58:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.767+0000 7fe6d6a00640 1 -- 172.21.15.28:0/792263231 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe6bc010100 con 0x7fe6d8146060 2024-04-26T20:58:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.768+0000 7fe6d7e00640 1 --2- 172.21.15.28:0/792263231 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6bc0039e0 0x7fe6bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.768+0000 7fe6d7400640 1 --2- 172.21.15.28:0/792263231 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6d8146060 0x7fe6bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.768+0000 7fe6d7e00640 1 -- 172.21.15.28:0/792263231 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6d8146060 msgr2=0x7fe6bc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.768+0000 7fe6d7e00640 1 --2- 172.21.15.28:0/792263231 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6d8146060 0x7fe6bc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.768+0000 7fe6d7e00640 1 -- 172.21.15.28:0/792263231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6bc004d40 msgr2=0x7fe6bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.768+0000 7fe6dce00640 1 --2- 172.21.15.28:0/792263231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6bc004d40 0x7fe6bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.768+0000 7fe6d7e00640 1 --2- 172.21.15.28:0/792263231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6bc004d40 0x7fe6bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.768+0000 7fe6d7e00640 1 -- 172.21.15.28:0/792263231 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6bc003520 con 0x7fe6bc0039e0 2024-04-26T20:58:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.768+0000 7fe6dce00640 1 --2- 172.21.15.28:0/792263231 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6bc004d40 0x7fe6bc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.768+0000 7fe6d7400640 1 --2- 172.21.15.28:0/792263231 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6d8146060 0x7fe6bc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:27.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.769+0000 7fe6d7e00640 1 --2- 172.21.15.28:0/792263231 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6bc0039e0 0x7fe6bc003e00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009ac0 tx=0x7fe6bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43dafe9d2d993c95 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:27.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.769+0000 7fe6d6a00640 1 -- 172.21.15.28:0/792263231 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8036070 con 0x7fe6bc0039e0 2024-04-26T20:58:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.769+0000 7fe6d6a00640 1 -- 172.21.15.28:0/792263231 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6c8031400 con 0x7fe6bc0039e0 2024-04-26T20:58:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.769+0000 7fe6d6a00640 1 -- 172.21.15.28:0/792263231 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8009690 con 0x7fe6bc0039e0 2024-04-26T20:58:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.770+0000 7fe6dd800640 1 -- 172.21.15.28:0/792263231 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6bc0039e0 msgr2=0x7fe6bc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.770+0000 7fe6dd800640 1 --2- 172.21.15.28:0/792263231 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6bc0039e0 0x7fe6bc003e00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009ac0 tx=0x7fe6bc005830 comp rx=0 tx=0).stop 2024-04-26T20:58:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.770+0000 7fe6dce00640 1 -- 172.21.15.28:0/792263231 reap_dead start 2024-04-26T20:58:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.770+0000 7fe6dd800640 1 -- 172.21.15.28:0/792263231 shutdown_connections 2024-04-26T20:58:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.770+0000 7fe6dd800640 1 -- 172.21.15.28:0/792263231 >> 172.21.15.28:0/792263231 conn(0x7fe6d80d0a50 msgr2=0x7fe6d80d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.770+0000 7fe6dd800640 1 -- 172.21.15.28:0/792263231 shutdown_connections 2024-04-26T20:58:27.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.770+0000 7fe6dd800640 1 -- 172.21.15.28:0/792263231 wait complete. 2024-04-26T20:58:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.772+0000 7fe6dd800640 1 Processor -- start 2024-04-26T20:58:27.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.772+0000 7fe6dd800640 1 -- start start 2024-04-26T20:58:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.773+0000 7fe6dd800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6bc0039e0 0x7fe6d8140a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.773+0000 7fe6dd800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d8141170 0x7fe6d80b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.773+0000 7fe6dd800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6d8158f10 0x7fe6d8159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.773+0000 7fe6dd800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d806ff60 con 0x7fe6d8158f10 2024-04-26T20:58:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.773+0000 7fe6dd800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d806fc80 con 0x7fe6d8141170 2024-04-26T20:58:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.773+0000 7fe6dd800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d8071e20 con 0x7fe6bc0039e0 2024-04-26T20:58:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.773+0000 7fe6dce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6d8158f10 0x7fe6d8159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.774+0000 7fe6dce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6d8158f10 0x7fe6d8159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36514/0 (socket says 172.21.15.28:36514) 2024-04-26T20:58:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.774+0000 7fe6dce00640 1 -- 172.21.15.28:0/2983716159 learned_addr learned my addr 172.21.15.28:0/2983716159 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.774+0000 7fe6d7400640 1 --2- 172.21.15.28:0/2983716159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d8141170 0x7fe6d80b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:27.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.774+0000 7fe6d7e00640 1 --2- 172.21.15.28:0/2983716159 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6bc0039e0 0x7fe6d8140a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.774+0000 7fe6dce00640 1 -- 172.21.15.28:0/2983716159 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6bc0039e0 msgr2=0x7fe6d8140a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.774+0000 7fe6dce00640 1 --2- 172.21.15.28:0/2983716159 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6bc0039e0 0x7fe6d8140a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.774+0000 7fe6dce00640 1 -- 172.21.15.28:0/2983716159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d8141170 msgr2=0x7fe6d80b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.774+0000 7fe6dce00640 1 --2- 172.21.15.28:0/2983716159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d8141170 0x7fe6d80b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.774+0000 7fe6dce00640 1 -- 172.21.15.28:0/2983716159 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d8140f70 con 0x7fe6d8158f10 2024-04-26T20:58:27.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.774+0000 7fe6dce00640 1 --2- 172.21.15.28:0/2983716159 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6d8158f10 0x7fe6d8159390 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8072360 tx=0x7fe6d000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:27.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.775+0000 7fe6cfe00640 1 -- 172.21.15.28:0/2983716159 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0018070 con 0x7fe6d8158f10 2024-04-26T20:58:27.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.775+0000 7fe6cfe00640 1 -- 172.21.15.28:0/2983716159 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6d0003b90 con 0x7fe6d8158f10 2024-04-26T20:58:27.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.775+0000 7fe6cfe00640 1 -- 172.21.15.28:0/2983716159 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d0005070 con 0x7fe6d8158f10 2024-04-26T20:58:27.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.775+0000 7fe6dd800640 1 -- 172.21.15.28:0/2983716159 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d8158150 con 0x7fe6d8158f10 2024-04-26T20:58:27.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.775+0000 7fe6dd800640 1 -- 172.21.15.28:0/2983716159 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d8158710 con 0x7fe6d8158f10 2024-04-26T20:58:27.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.776+0000 7fe6dd800640 1 -- 172.21.15.28:0/2983716159 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d806a360 con 0x7fe6d8158f10 2024-04-26T20:58:27.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.777+0000 7fe6cfe00640 1 -- 172.21.15.28:0/2983716159 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe6d001b070 con 0x7fe6d8158f10 2024-04-26T20:58:27.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.778+0000 7fe6cfe00640 1 --2- 172.21.15.28:0/2983716159 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe6b806c8f0 0x7fe6b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:27.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.778+0000 7fe6cfe00640 1 -- 172.21.15.28:0/2983716159 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fe6d008b230 con 0x7fe6d8158f10 2024-04-26T20:58:27.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.778+0000 7fe6d7e00640 1 --2- 172.21.15.28:0/2983716159 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe6b806c8f0 0x7fe6b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:27.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.779+0000 7fe6d7e00640 1 --2- 172.21.15.28:0/2983716159 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe6b806c8f0 0x7fe6b806edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8033000 tx=0x7fe6c8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:27.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:27.783+0000 7fe6cfe00640 1 -- 172.21.15.28:0/2983716159 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe6d8140f70 con 0x7fe6d8158f10 2024-04-26T20:58:28.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.027+0000 7fe6dd800640 1 -- 172.21.15.28:0/2983716159 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6d815c2a0 con 0x7fe6d8158f10 2024-04-26T20:58:28.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.028+0000 7fe6cfe00640 1 -- 172.21.15.28:0/2983716159 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7fe6d00595c0 con 0x7fe6d8158f10 2024-04-26T20:58:28.030 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:28.030 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/1364678224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1364678224},{"type":"v1","addr":"172.21.15.103:6834","nonce":1364678224}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4081926466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":4081926466},{"type":"v1","addr":"172.21.15.28:6837","nonce":4081926466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5175,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/1370252546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":1370252546},{"type":"v1","addr":"172.21.15.28:6835","nonce":1370252546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:58:09.226353+0000","modified":"2024-04-26T20:58:12.319110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5184":{"gid":5184,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6835/1781724684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":1781724684},{"type":"v1","addr":"172.21.15.103:6835","nonce":1781724684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5184,"qdb_cluster":[5184]},"id":2}]} 2024-04-26T20:58:28.030 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-04-26T20:58:28.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.031+0000 7fe6dd800640 1 -- 172.21.15.28:0/2983716159 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe6b806c8f0 msgr2=0x7fe6b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:28.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.031+0000 7fe6dd800640 1 --2- 172.21.15.28:0/2983716159 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe6b806c8f0 0x7fe6b806edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8033000 tx=0x7fe6c8008030 comp rx=0 tx=0).stop 2024-04-26T20:58:28.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.031+0000 7fe6dd800640 1 -- 172.21.15.28:0/2983716159 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6d8158f10 msgr2=0x7fe6d8159390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:28.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.031+0000 7fe6dd800640 1 --2- 172.21.15.28:0/2983716159 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6d8158f10 0x7fe6d8159390 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8072360 tx=0x7fe6d000dc00 comp rx=0 tx=0).stop 2024-04-26T20:58:28.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.031+0000 7fe6dd800640 1 -- 172.21.15.28:0/2983716159 shutdown_connections 2024-04-26T20:58:28.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.031+0000 7fe6dd800640 1 --2- 172.21.15.28:0/2983716159 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe6b806c8f0 0x7fe6b806edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.031+0000 7fe6dd800640 1 --2- 172.21.15.28:0/2983716159 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe6d8158f10 0x7fe6d8159390 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.031+0000 7fe6dd800640 1 --2- 172.21.15.28:0/2983716159 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe6d8141170 0x7fe6d80b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.031+0000 7fe6dd800640 1 --2- 172.21.15.28:0/2983716159 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe6bc0039e0 0x7fe6d8140a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.031+0000 7fe6dd800640 1 -- 172.21.15.28:0/2983716159 >> 172.21.15.28:0/2983716159 conn(0x7fe6d80d0a50 msgr2=0x7fe6d80cf910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:28.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.031+0000 7fe6dd800640 1 -- 172.21.15.28:0/2983716159 shutdown_connections 2024-04-26T20:58:28.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.032+0000 7fe6dd800640 1 -- 172.21.15.28:0/2983716159 wait complete. 2024-04-26T20:58:28.052 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:58:28.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.262+0000 7fea07200640 1 Processor -- start 2024-04-26T20:58:28.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.263+0000 7fea07200640 1 -- start start 2024-04-26T20:58:28.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.264+0000 7fea07200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea080d1080 0x7fea0815d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:28.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.264+0000 7fea07200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fea0806d360 0x7fea0806d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:28.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.264+0000 7fea07200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea080d1460 con 0x7fea08161260 2024-04-26T20:58:28.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.264+0000 7fea07200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fea0815e240 con 0x7fea0806d360 2024-04-26T20:58:28.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.264+0000 7fea07200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fea081427a0 con 0x7fea080d1080 2024-04-26T20:58:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.265+0000 7fea06800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fea0806d360 0x7fea0806d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.265+0000 7fea06800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fea0806d360 0x7fea0806d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36520/0 (socket says 172.21.15.28:36520) 2024-04-26T20:58:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.265+0000 7fea06800640 1 -- 172.21.15.28:0/4254117377 learned_addr learned my addr 172.21.15.28:0/4254117377 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.265+0000 7fea05e00640 1 --2- 172.21.15.28:0/4254117377 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea080d1080 0x7fea0815d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.265+0000 7fea06800640 1 -- 172.21.15.28:0/4254117377 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea080d1080 msgr2=0x7fea0815d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.265+0000 7fea06800640 1 --2- 172.21.15.28:0/4254117377 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea080d1080 0x7fea0815d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.265+0000 7fea06800640 1 -- 172.21.15.28:0/4254117377 >> v1:172.21.15.103:6789/0 conn(0x7fea08161260 legacy=0x7fea08142c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.265+0000 7fea06800640 1 -- 172.21.15.28:0/4254117377 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0815d9a0 con 0x7fea0806d360 2024-04-26T20:58:28.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.266+0000 7fea05e00640 1 --2- 172.21.15.28:0/4254117377 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea080d1080 0x7fea0815d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:58:28.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.267+0000 7fea06800640 1 --2- 172.21.15.28:0/4254117377 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea0806d360 0x7fea0806d7a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fea00009870 tx=0x7fea0002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee740d36d052bda1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:28.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.267+0000 7fea04a00640 1 -- 172.21.15.28:0/4254117377 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00037070 con 0x7fea0806d360 2024-04-26T20:58:28.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.267+0000 7fea04a00640 1 -- 172.21.15.28:0/4254117377 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea00032440 con 0x7fea0806d360 2024-04-26T20:58:28.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.267+0000 7fea04a00640 1 -- 172.21.15.28:0/4254117377 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea00032810 con 0x7fea0806d360 2024-04-26T20:58:28.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.268+0000 7fea07200640 1 -- 172.21.15.28:0/4254117377 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea0806d360 msgr2=0x7fea0806d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:28.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.268+0000 7fea07200640 1 --2- 172.21.15.28:0/4254117377 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea0806d360 0x7fea0806d7a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fea00009870 tx=0x7fea0002a250 comp rx=0 tx=0).stop 2024-04-26T20:58:28.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.268+0000 7fea07200640 1 -- 172.21.15.28:0/4254117377 shutdown_connections 2024-04-26T20:58:28.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.268+0000 7fea07200640 1 --2- 172.21.15.28:0/4254117377 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea0806d360 0x7fea0806d7a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.268+0000 7fea07200640 1 --2- 172.21.15.28:0/4254117377 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea080d1080 0x7fea0815d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.268+0000 7fea07200640 1 -- 172.21.15.28:0/4254117377 >> 172.21.15.28:0/4254117377 conn(0x7fea08072f10 msgr2=0x7fea08073320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:28.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.268+0000 7fea07200640 1 -- 172.21.15.28:0/4254117377 shutdown_connections 2024-04-26T20:58:28.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.268+0000 7fea07200640 1 -- 172.21.15.28:0/4254117377 wait complete. 2024-04-26T20:58:28.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.270+0000 7fea07200640 1 Processor -- start 2024-04-26T20:58:28.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.270+0000 7fea07200640 1 -- start start 2024-04-26T20:58:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.271+0000 7fea07200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea0806d360 0x7fea08143680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.271+0000 7fea07200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea080d1080 0x7fea081f8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.271+0000 7fea07200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea08161260 0x7fea080b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.271+0000 7fea07200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fea0815df60 con 0x7fea0806d360 2024-04-26T20:58:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.271+0000 7fea07200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fea0806dce0 con 0x7fea080d1080 2024-04-26T20:58:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.271+0000 7fea07200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fea081616a0 con 0x7fea08161260 2024-04-26T20:58:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.271+0000 7fea05e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea0806d360 0x7fea08143680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.271+0000 7fea05e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea0806d360 0x7fea08143680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36534/0 (socket says 172.21.15.28:36534) 2024-04-26T20:58:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.271+0000 7fea05e00640 1 -- 172.21.15.28:0/2178272941 learned_addr learned my addr 172.21.15.28:0/2178272941 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.272+0000 7fea05e00640 1 -- 172.21.15.28:0/2178272941 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea08161260 msgr2=0x7fea080b0d80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T20:58:28.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.272+0000 7fea05e00640 1 --2- 172.21.15.28:0/2178272941 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea08161260 0x7fea080b0d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.272+0000 7fea05e00640 1 -- 172.21.15.28:0/2178272941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea080d1080 msgr2=0x7fea081f8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:28.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.272+0000 7fea05e00640 1 --2- 172.21.15.28:0/2178272941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea080d1080 0x7fea081f8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.272+0000 7fea05e00640 1 -- 172.21.15.28:0/2178272941 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea081506c0 con 0x7fea0806d360 2024-04-26T20:58:28.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.272+0000 7fea05e00640 1 --2- 172.21.15.28:0/2178272941 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea0806d360 0x7fea08143680 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009fc0 tx=0x7fe9f800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:28.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.272+0000 7fe9fe000640 1 -- 172.21.15.28:0/2178272941 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f801a070 con 0x7fea0806d360 2024-04-26T20:58:28.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.272+0000 7fe9fe000640 1 -- 172.21.15.28:0/2178272941 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f8003b90 con 0x7fea0806d360 2024-04-26T20:58:28.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.272+0000 7fea07200640 1 -- 172.21.15.28:0/2178272941 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea08151440 con 0x7fea0806d360 2024-04-26T20:58:28.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.272+0000 7fe9fe000640 1 -- 172.21.15.28:0/2178272941 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8004030 con 0x7fea0806d360 2024-04-26T20:58:28.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.272+0000 7fea07200640 1 -- 172.21.15.28:0/2178272941 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea08150960 con 0x7fea0806d360 2024-04-26T20:58:28.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.274+0000 7fea07200640 1 -- 172.21.15.28:0/2178272941 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea0806a360 con 0x7fea0806d360 2024-04-26T20:58:28.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.275+0000 7fe9fe000640 1 -- 172.21.15.28:0/2178272941 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9f801d070 con 0x7fea0806d360 2024-04-26T20:58:28.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.275+0000 7fe9fe000640 1 --2- 172.21.15.28:0/2178272941 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe9e406c8f0 0x7fe9e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:28.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.276+0000 7fe9fe000640 1 -- 172.21.15.28:0/2178272941 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fe9f808bd40 con 0x7fea0806d360 2024-04-26T20:58:28.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.276+0000 7fea05400640 1 --2- 172.21.15.28:0/2178272941 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe9e406c8f0 0x7fe9e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:28.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.277+0000 7fea05400640 1 --2- 172.21.15.28:0/2178272941 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe9e406c8f0 0x7fe9e406edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0000f60 tx=0x7fe9f00079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:28.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.280+0000 7fe9fe000640 1 -- 172.21.15.28:0/2178272941 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe9f8010070 con 0x7fea0806d360 2024-04-26T20:58:28.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.526+0000 7fea07200640 1 -- 172.21.15.28:0/2178272941 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea0806cab0 con 0x7fea0806d360 2024-04-26T20:58:28.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.527+0000 7fe9fe000640 1 -- 172.21.15.28:0/2178272941 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5187 (secure 0 0 0) 0x7fe9f8059b10 con 0x7fea0806d360 2024-04-26T20:58:28.528 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:28.528 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/1364678224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1364678224},{"type":"v1","addr":"172.21.15.103:6834","nonce":1364678224}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4081926466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":4081926466},{"type":"v1","addr":"172.21.15.28:6837","nonce":4081926466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5175,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/1370252546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":1370252546},{"type":"v1","addr":"172.21.15.28:6835","nonce":1370252546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:58:09.226353+0000","modified":"2024-04-26T20:58:12.319110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5184},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5184":{"gid":5184,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.103:6835/1781724684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":1781724684},{"type":"v1","addr":"172.21.15.103:6835","nonce":1781724684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5184,"qdb_cluster":[5184]},"id":2}]} 2024-04-26T20:58:28.529 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2024-04-26T20:58:28.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.530+0000 7fea07200640 1 -- 172.21.15.28:0/2178272941 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe9e406c8f0 msgr2=0x7fe9e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:28.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.530+0000 7fea07200640 1 --2- 172.21.15.28:0/2178272941 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe9e406c8f0 0x7fe9e406edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0000f60 tx=0x7fe9f00079c0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.530+0000 7fea07200640 1 -- 172.21.15.28:0/2178272941 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea0806d360 msgr2=0x7fea08143680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:28.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.530+0000 7fea07200640 1 --2- 172.21.15.28:0/2178272941 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea0806d360 0x7fea08143680 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009fc0 tx=0x7fe9f800d560 comp rx=0 tx=0).stop 2024-04-26T20:58:28.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.530+0000 7fea07200640 1 -- 172.21.15.28:0/2178272941 shutdown_connections 2024-04-26T20:58:28.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.530+0000 7fea07200640 1 --2- 172.21.15.28:0/2178272941 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe9e406c8f0 0x7fe9e406edb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.530+0000 7fea07200640 1 --2- 172.21.15.28:0/2178272941 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fea08161260 0x7fea080b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.530+0000 7fea07200640 1 --2- 172.21.15.28:0/2178272941 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea080d1080 0x7fea081f8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.530+0000 7fea07200640 1 --2- 172.21.15.28:0/2178272941 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea0806d360 0x7fea08143680 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.530+0000 7fea07200640 1 -- 172.21.15.28:0/2178272941 >> 172.21.15.28:0/2178272941 conn(0x7fea08072f10 msgr2=0x7fea080732f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:28.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.530+0000 7fea07200640 1 -- 172.21.15.28:0/2178272941 shutdown_connections 2024-04-26T20:58:28.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.530+0000 7fea07200640 1 -- 172.21.15.28:0/2178272941 wait complete. 2024-04-26T20:58:28.550 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:58:28.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.763+0000 7fac8b200640 1 Processor -- start 2024-04-26T20:58:28.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.764+0000 7fac8b200640 1 -- start start 2024-04-26T20:58:28.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.765+0000 7fac8b200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac8c15f070 0x7fac8c0d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:28.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.765+0000 7fac8b200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac8c0d4500 con 0x7fac8c0d1f10 2024-04-26T20:58:28.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.765+0000 7fac8b200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac8c15f450 con 0x7fac8c0ba710 2024-04-26T20:58:28.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.765+0000 7fac8b200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fac8c0d4a80 con 0x7fac8c15f070 2024-04-26T20:58:28.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.766+0000 7fac8a800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fac8c0d1f10 0x7fac8c0b7f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37118/0 (socket says 172.21.15.28:37118) 2024-04-26T20:58:28.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.766+0000 7fac8a800640 1 -- 172.21.15.28:0/29555826 learned_addr learned my addr 172.21.15.28:0/29555826 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:28.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.766+0000 7fac89e00640 1 --2- 172.21.15.28:0/29555826 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac8c15f070 0x7fac8c0d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:28.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.766+0000 7fac89400640 1 -- 172.21.15.28:0/29555826 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1140171367 0 0) 0x7fac8c0d4500 con 0x7fac8c0d1f10 2024-04-26T20:58:28.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.766+0000 7fac89400640 1 -- 172.21.15.28:0/29555826 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac70003520 con 0x7fac8c0d1f10 2024-04-26T20:58:28.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.767+0000 7fac89400640 1 -- 172.21.15.28:0/29555826 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3293100124 0 0) 0x7fac70003520 con 0x7fac8c0d1f10 2024-04-26T20:58:28.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.767+0000 7fac89400640 1 -- 172.21.15.28:0/29555826 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac8c15f070 msgr2=0x7fac8c0d36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:28.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.767+0000 7fac89400640 1 --2- 172.21.15.28:0/29555826 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac8c15f070 0x7fac8c0d36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.767+0000 7fac89400640 1 -- 172.21.15.28:0/29555826 >> v1:172.21.15.103:6789/0 conn(0x7fac8c0ba710 legacy=0x7fac8c0d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:28.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.767+0000 7fac89400640 1 -- 172.21.15.28:0/29555826 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8c0d01c0 con 0x7fac8c0d1f10 2024-04-26T20:58:28.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.767+0000 7fac89400640 1 -- 172.21.15.28:0/29555826 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fac84001a10 con 0x7fac8c0d1f10 2024-04-26T20:58:28.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.767+0000 7fac89400640 1 -- 172.21.15.28:0/29555826 >> v1:172.21.15.28:6789/0 conn(0x7fac8c0d1f10 legacy=0x7fac8c0b7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:28.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.767+0000 7fac89400640 1 --2- 172.21.15.28:0/29555826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac700039e0 0x7fac70003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:28.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.767+0000 7fac89400640 1 --2- 172.21.15.28:0/29555826 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac70004800 0x7fac70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:28.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.767+0000 7fac89400640 1 --2- 172.21.15.28:0/29555826 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac8c15f070 0x7fac70005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:28.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.767+0000 7fac89400640 1 -- 172.21.15.28:0/29555826 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c0d4a80 con 0x7fac70004800 2024-04-26T20:58:28.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.767+0000 7fac89400640 1 -- 172.21.15.28:0/29555826 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fac7000bb50 con 0x7fac700039e0 2024-04-26T20:58:28.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.767+0000 7fac89400640 1 -- 172.21.15.28:0/29555826 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fac7000bd50 con 0x7fac8c15f070 2024-04-26T20:58:28.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.767+0000 7fac83e00640 1 --2- 172.21.15.28:0/29555826 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac70004800 0x7fac70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:28.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.768+0000 7fac89e00640 1 --2- 172.21.15.28:0/29555826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac700039e0 0x7fac70003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:28.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.768+0000 7fac83e00640 1 -- 172.21.15.28:0/29555826 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac8c15f070 msgr2=0x7fac70005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:28.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.768+0000 7fac8a800640 1 --2- 172.21.15.28:0/29555826 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac8c15f070 0x7fac70005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:28.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.768+0000 7fac83e00640 1 --2- 172.21.15.28:0/29555826 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac8c15f070 0x7fac70005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.768+0000 7fac83e00640 1 -- 172.21.15.28:0/29555826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac700039e0 msgr2=0x7fac70003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:28.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.768+0000 7fac83e00640 1 --2- 172.21.15.28:0/29555826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac700039e0 0x7fac70003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.768+0000 7fac83e00640 1 -- 172.21.15.28:0/29555826 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac70003520 con 0x7fac70004800 2024-04-26T20:58:28.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.768+0000 7fac89e00640 1 --2- 172.21.15.28:0/29555826 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac700039e0 0x7fac70003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:28.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.768+0000 7fac8a800640 1 --2- 172.21.15.28:0/29555826 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac8c15f070 0x7fac70005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:28.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.769+0000 7fac83e00640 1 --2- 172.21.15.28:0/29555826 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac70004800 0x7fac70004c80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fac74001750 tx=0x7fac70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0b8c8f5847e6b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:28.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.769+0000 7fac89400640 1 -- 172.21.15.28:0/29555826 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74001370 con 0x7fac70004800 2024-04-26T20:58:28.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.770+0000 7fac89400640 1 -- 172.21.15.28:0/29555826 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac740052d0 con 0x7fac70004800 2024-04-26T20:58:28.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.770+0000 7fac89400640 1 -- 172.21.15.28:0/29555826 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7402abe0 con 0x7fac70004800 2024-04-26T20:58:28.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.770+0000 7fac8b200640 1 -- 172.21.15.28:0/29555826 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac70004800 msgr2=0x7fac70004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:28.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.770+0000 7fac8b200640 1 --2- 172.21.15.28:0/29555826 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac70004800 0x7fac70004c80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fac74001750 tx=0x7fac70005830 comp rx=0 tx=0).stop 2024-04-26T20:58:28.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.770+0000 7fac8a800640 1 -- 172.21.15.28:0/29555826 reap_dead start 2024-04-26T20:58:28.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.770+0000 7fac8b200640 1 -- 172.21.15.28:0/29555826 shutdown_connections 2024-04-26T20:58:28.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.770+0000 7fac8b200640 1 -- 172.21.15.28:0/29555826 >> 172.21.15.28:0/29555826 conn(0x7fac8c06c980 msgr2=0x7fac8c072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:28.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.770+0000 7fac8b200640 1 -- 172.21.15.28:0/29555826 shutdown_connections 2024-04-26T20:58:28.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.770+0000 7fac8b200640 1 -- 172.21.15.28:0/29555826 wait complete. 2024-04-26T20:58:28.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.772+0000 7fac8b200640 1 Processor -- start 2024-04-26T20:58:28.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.772+0000 7fac8b200640 1 -- start start 2024-04-26T20:58:28.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.773+0000 7fac8b200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac70004800 0x7fac8c14e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:28.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.773+0000 7fac8b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac8c0b5b70 0x7fac8c160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:28.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.773+0000 7fac8b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac8c0b5ff0 0x7fac8c161130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:28.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.773+0000 7fac8b200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c0cfde0 con 0x7fac8c0b5ff0 2024-04-26T20:58:28.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.773+0000 7fac8b200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c0d04a0 con 0x7fac8c0b5b70 2024-04-26T20:58:28.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.773+0000 7fac8b200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fac8c0cff60 con 0x7fac70004800 2024-04-26T20:58:28.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.773+0000 7fac8a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac8c0b5ff0 0x7fac8c161130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:28.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.774+0000 7fac8a800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac8c0b5ff0 0x7fac8c161130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36552/0 (socket says 172.21.15.28:36552) 2024-04-26T20:58:28.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.774+0000 7fac8a800640 1 -- 172.21.15.28:0/4018837665 learned_addr learned my addr 172.21.15.28:0/4018837665 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:28.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.774+0000 7fac89e00640 1 --2- 172.21.15.28:0/4018837665 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac70004800 0x7fac8c14e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:28.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.774+0000 7fac83e00640 1 --2- 172.21.15.28:0/4018837665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac8c0b5b70 0x7fac8c160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:28.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.774+0000 7fac8a800640 1 -- 172.21.15.28:0/4018837665 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac70004800 msgr2=0x7fac8c14e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:28.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.774+0000 7fac8a800640 1 --2- 172.21.15.28:0/4018837665 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac70004800 0x7fac8c14e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.774+0000 7fac8a800640 1 -- 172.21.15.28:0/4018837665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac8c0b5b70 msgr2=0x7fac8c160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:28.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.774+0000 7fac8a800640 1 --2- 172.21.15.28:0/4018837665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac8c0b5b70 0x7fac8c160bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.774+0000 7fac8a800640 1 -- 172.21.15.28:0/4018837665 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8c0b6550 con 0x7fac8c0b5ff0 2024-04-26T20:58:28.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.774+0000 7fac89e00640 1 --2- 172.21.15.28:0/4018837665 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac70004800 0x7fac8c14e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:28.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.774+0000 7fac83e00640 1 --2- 172.21.15.28:0/4018837665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac8c0b5b70 0x7fac8c160bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:28.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.774+0000 7fac8a800640 1 --2- 172.21.15.28:0/4018837665 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac8c0b5ff0 0x7fac8c161130 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fac70004780 tx=0x7fac8400b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:28.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.775+0000 7fac82000640 1 -- 172.21.15.28:0/4018837665 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8400b8d0 con 0x7fac8c0b5ff0 2024-04-26T20:58:28.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.775+0000 7fac82000640 1 -- 172.21.15.28:0/4018837665 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac8400ba90 con 0x7fac8c0b5ff0 2024-04-26T20:58:28.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.775+0000 7fac8b200640 1 -- 172.21.15.28:0/4018837665 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac8c0b6830 con 0x7fac8c0b5ff0 2024-04-26T20:58:28.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.775+0000 7fac82000640 1 -- 172.21.15.28:0/4018837665 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8400be80 con 0x7fac8c0b5ff0 2024-04-26T20:58:28.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.775+0000 7fac8b200640 1 -- 172.21.15.28:0/4018837665 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac8c0b6df0 con 0x7fac8c0b5ff0 2024-04-26T20:58:28.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.776+0000 7fac8b200640 1 -- 172.21.15.28:0/4018837665 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac8c06a360 con 0x7fac8c0b5ff0 2024-04-26T20:58:28.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.777+0000 7fac82000640 1 -- 172.21.15.28:0/4018837665 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac8401b020 con 0x7fac8c0b5ff0 2024-04-26T20:58:28.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.778+0000 7fac82000640 1 --2- 172.21.15.28:0/4018837665 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fac6806c8f0 0x7fac6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:28.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.778+0000 7fac82000640 1 -- 172.21.15.28:0/4018837665 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fac8408bca0 con 0x7fac8c0b5ff0 2024-04-26T20:58:28.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.779+0000 7fac89e00640 1 --2- 172.21.15.28:0/4018837665 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fac6806c8f0 0x7fac6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:28.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.779+0000 7fac89e00640 1 --2- 172.21.15.28:0/4018837665 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fac6806c8f0 0x7fac6806edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009aa0 tx=0x7fac7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:28.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.783+0000 7fac82000640 1 -- 172.21.15.28:0/4018837665 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fac8405a030 con 0x7fac8c0b5ff0 2024-04-26T20:58:28.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.990+0000 7fac8b200640 1 -- 172.21.15.28:0/4018837665 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac8c072a80 con 0x7fac8c0b5ff0 2024-04-26T20:58:28.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.993+0000 7fac82000640 1 -- 172.21.15.28:0/4018837665 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16645 (secure 0 0 0) 0x7fac84059930 con 0x7fac8c0b5ff0 2024-04-26T20:58:28.994 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:28.995 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":34,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:58:10.759775+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:58:07.143890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-26T20:58:08.193405+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:58:28.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.996+0000 7fac8b200640 1 -- 172.21.15.28:0/4018837665 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fac6806c8f0 msgr2=0x7fac6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:28.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.996+0000 7fac8b200640 1 --2- 172.21.15.28:0/4018837665 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fac6806c8f0 0x7fac6806edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009aa0 tx=0x7fac7c008040 comp rx=0 tx=0).stop 2024-04-26T20:58:28.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.996+0000 7fac8b200640 1 -- 172.21.15.28:0/4018837665 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac8c0b5ff0 msgr2=0x7fac8c161130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:28.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.996+0000 7fac8b200640 1 --2- 172.21.15.28:0/4018837665 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac8c0b5ff0 0x7fac8c161130 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fac70004780 tx=0x7fac8400b0d0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.996+0000 7fac8b200640 1 -- 172.21.15.28:0/4018837665 shutdown_connections 2024-04-26T20:58:28.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.996+0000 7fac8b200640 1 --2- 172.21.15.28:0/4018837665 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fac6806c8f0 0x7fac6806edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.996+0000 7fac8b200640 1 --2- 172.21.15.28:0/4018837665 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fac8c0b5ff0 0x7fac8c161130 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.996+0000 7fac8b200640 1 --2- 172.21.15.28:0/4018837665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fac8c0b5b70 0x7fac8c160bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.996+0000 7fac8b200640 1 --2- 172.21.15.28:0/4018837665 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fac70004800 0x7fac8c14e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:28.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.996+0000 7fac8b200640 1 -- 172.21.15.28:0/4018837665 >> 172.21.15.28:0/4018837665 conn(0x7fac8c06c980 msgr2=0x7fac8c15bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:28.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.996+0000 7fac8b200640 1 -- 172.21.15.28:0/4018837665 shutdown_connections 2024-04-26T20:58:28.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:28.996+0000 7fac8b200640 1 -- 172.21.15.28:0/4018837665 wait complete. 2024-04-26T20:58:29.019 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T20:58:29.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.238+0000 7ff135c00640 1 Processor -- start 2024-04-26T20:58:29.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.239+0000 7ff135c00640 1 -- start start 2024-04-26T20:58:29.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.240+0000 7ff135c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff1300d24f0 0x7ff1300d2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:29.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.240+0000 7ff135c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1300d36b0 con 0x7ff13015f3d0 2024-04-26T20:58:29.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.240+0000 7ff135c00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1300d3990 con 0x7ff13015eff0 2024-04-26T20:58:29.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.240+0000 7ff135c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ff1300d0400 con 0x7ff1300d24f0 2024-04-26T20:58:29.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.240+0000 7ff135200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff1300d24f0 0x7ff1300d2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:29.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.241+0000 7ff135200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff1300d24f0 0x7ff1300d2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36564/0 (socket says 172.21.15.28:36564) 2024-04-26T20:58:29.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.241+0000 7ff135200640 1 -- 172.21.15.28:0/2633064583 learned_addr learned my addr 172.21.15.28:0/2633064583 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:29.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.241+0000 7ff135200640 1 -- 172.21.15.28:0/2633064583 >> v1:172.21.15.103:6789/0 conn(0x7ff13015eff0 legacy=0x7ff1300d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:29.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.241+0000 7ff135200640 1 -- 172.21.15.28:0/2633064583 >> v1:172.21.15.28:6789/0 conn(0x7ff13015f3d0 legacy=0x7ff130166d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:29.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.241+0000 7ff135200640 1 -- 172.21.15.28:0/2633064583 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1300d3c70 con 0x7ff1300d24f0 2024-04-26T20:58:29.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.242+0000 7ff135200640 1 --2- 172.21.15.28:0/2633064583 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1300d24f0 0x7ff1300d2fb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff128009e50 tx=0x7ff12802a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7c7837038038ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:29.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.242+0000 7ff12ea00640 1 -- 172.21.15.28:0/2633064583 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128036070 con 0x7ff1300d24f0 2024-04-26T20:58:29.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.243+0000 7ff12ea00640 1 -- 172.21.15.28:0/2633064583 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff128031600 con 0x7ff1300d24f0 2024-04-26T20:58:29.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.243+0000 7ff12ea00640 1 -- 172.21.15.28:0/2633064583 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1280319d0 con 0x7ff1300d24f0 2024-04-26T20:58:29.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.243+0000 7ff135c00640 1 -- 172.21.15.28:0/2633064583 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1300d24f0 msgr2=0x7ff1300d2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:29.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.243+0000 7ff135c00640 1 --2- 172.21.15.28:0/2633064583 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1300d24f0 0x7ff1300d2fb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff128009e50 tx=0x7ff12802a7f0 comp rx=0 tx=0).stop 2024-04-26T20:58:29.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.243+0000 7ff135c00640 1 -- 172.21.15.28:0/2633064583 shutdown_connections 2024-04-26T20:58:29.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.243+0000 7ff135c00640 1 --2- 172.21.15.28:0/2633064583 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1300d24f0 0x7ff1300d2fb0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:29.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.243+0000 7ff135c00640 1 -- 172.21.15.28:0/2633064583 >> 172.21.15.28:0/2633064583 conn(0x7ff13006c770 msgr2=0x7ff13006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:29.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.243+0000 7ff135c00640 1 -- 172.21.15.28:0/2633064583 shutdown_connections 2024-04-26T20:58:29.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.243+0000 7ff135c00640 1 -- 172.21.15.28:0/2633064583 wait complete. 2024-04-26T20:58:29.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.245+0000 7ff135c00640 1 Processor -- start 2024-04-26T20:58:29.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.245+0000 7ff135c00640 1 -- start start 2024-04-26T20:58:29.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.246+0000 7ff135c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1300d24f0 0x7ff130162280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:29.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.246+0000 7ff135c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff13015eff0 0x7ff13014caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:29.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.246+0000 7ff135c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff13015f3d0 0x7ff13015cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:29.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.246+0000 7ff135c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff1300ba4a0 con 0x7ff13015eff0 2024-04-26T20:58:29.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.246+0000 7ff135c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff1300b8000 con 0x7ff1300d24f0 2024-04-26T20:58:29.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.246+0000 7ff135c00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff1300ba620 con 0x7ff13015f3d0 2024-04-26T20:58:29.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.247+0000 7ff12f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff13015eff0 0x7ff13014caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:29.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.247+0000 7ff12f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff13015eff0 0x7ff13014caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36566/0 (socket says 172.21.15.28:36566) 2024-04-26T20:58:29.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.247+0000 7ff12f400640 1 -- 172.21.15.28:0/2762495627 learned_addr learned my addr 172.21.15.28:0/2762495627 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:29.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.247+0000 7ff135200640 1 --2- 172.21.15.28:0/2762495627 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff13015f3d0 0x7ff13015cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:29.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.247+0000 7ff12fe00640 1 --2- 172.21.15.28:0/2762495627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1300d24f0 0x7ff130162280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:29.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.247+0000 7ff12f400640 1 -- 172.21.15.28:0/2762495627 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff13015f3d0 msgr2=0x7ff13015cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:29.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.247+0000 7ff12f400640 1 --2- 172.21.15.28:0/2762495627 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff13015f3d0 0x7ff13015cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:29.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.247+0000 7ff12f400640 1 -- 172.21.15.28:0/2762495627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1300d24f0 msgr2=0x7ff130162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:29.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.247+0000 7ff12f400640 1 --2- 172.21.15.28:0/2762495627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1300d24f0 0x7ff130162280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:29.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.247+0000 7ff12f400640 1 -- 172.21.15.28:0/2762495627 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1300c9cc0 con 0x7ff13015eff0 2024-04-26T20:58:29.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.247+0000 7ff12f400640 1 --2- 172.21.15.28:0/2762495627 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff13015eff0 0x7ff13014caa0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00c440 tx=0x7ff11c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:29.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.248+0000 7ff127e00640 1 -- 172.21.15.28:0/2762495627 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c0052d0 con 0x7ff13015eff0 2024-04-26T20:58:29.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.248+0000 7ff127e00640 1 -- 172.21.15.28:0/2762495627 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff11c005470 con 0x7ff13015eff0 2024-04-26T20:58:29.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.248+0000 7ff127e00640 1 -- 172.21.15.28:0/2762495627 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c007260 con 0x7ff13015eff0 2024-04-26T20:58:29.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.248+0000 7ff135c00640 1 -- 172.21.15.28:0/2762495627 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1300c9fa0 con 0x7ff13015eff0 2024-04-26T20:58:29.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.248+0000 7ff135c00640 1 -- 172.21.15.28:0/2762495627 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1300ca560 con 0x7ff13015eff0 2024-04-26T20:58:29.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.249+0000 7ff135c00640 1 -- 172.21.15.28:0/2762495627 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff13006a360 con 0x7ff13015eff0 2024-04-26T20:58:29.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.250+0000 7ff127e00640 1 -- 172.21.15.28:0/2762495627 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff11c007400 con 0x7ff13015eff0 2024-04-26T20:58:29.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.251+0000 7ff127e00640 1 --2- 172.21.15.28:0/2762495627 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff11006c8f0 0x7ff11006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:29.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.251+0000 7ff127e00640 1 -- 172.21.15.28:0/2762495627 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7ff11c08af20 con 0x7ff13015eff0 2024-04-26T20:58:29.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.251+0000 7ff12fe00640 1 --2- 172.21.15.28:0/2762495627 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff11006c8f0 0x7ff11006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:29.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.252+0000 7ff12fe00640 1 --2- 172.21.15.28:0/2762495627 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff11006c8f0 0x7ff11006edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff120004ea0 tx=0x7ff120009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:29.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.256+0000 7ff127e00640 1 -- 172.21.15.28:0/2762495627 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff11c0593e0 con 0x7ff13015eff0 2024-04-26T20:58:29.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.506+0000 7ff135c00640 1 -- 172.21.15.28:0/2762495627 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff1300ba960 con 0x7ff13015eff0 2024-04-26T20:58:29.508 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-04-26T20:58:29.507+0000 7f9cbd800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T20:58:29.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.512+0000 7ff127e00640 1 -- 172.21.15.28:0/2762495627 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7ff11c058ce0 con 0x7ff13015eff0 2024-04-26T20:58:29.513 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T20:58:29.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.515+0000 7ff135c00640 1 -- 172.21.15.28:0/2762495627 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff11006c8f0 msgr2=0x7ff11006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:29.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.515+0000 7ff135c00640 1 --2- 172.21.15.28:0/2762495627 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff11006c8f0 0x7ff11006edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff120004ea0 tx=0x7ff120009040 comp rx=0 tx=0).stop 2024-04-26T20:58:29.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.515+0000 7ff135c00640 1 -- 172.21.15.28:0/2762495627 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff13015eff0 msgr2=0x7ff13014caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:29.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.515+0000 7ff135c00640 1 --2- 172.21.15.28:0/2762495627 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff13015eff0 0x7ff13014caa0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00c440 tx=0x7ff11c00c900 comp rx=0 tx=0).stop 2024-04-26T20:58:29.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.516+0000 7ff135c00640 1 -- 172.21.15.28:0/2762495627 shutdown_connections 2024-04-26T20:58:29.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.516+0000 7ff135c00640 1 --2- 172.21.15.28:0/2762495627 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff11006c8f0 0x7ff11006edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:29.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.516+0000 7ff135c00640 1 --2- 172.21.15.28:0/2762495627 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff13015f3d0 0x7ff13015cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:29.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.516+0000 7ff135c00640 1 --2- 172.21.15.28:0/2762495627 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff13015eff0 0x7ff13014caa0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:29.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.516+0000 7ff135c00640 1 --2- 172.21.15.28:0/2762495627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1300d24f0 0x7ff130162280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:29.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.516+0000 7ff135c00640 1 -- 172.21.15.28:0/2762495627 >> 172.21.15.28:0/2762495627 conn(0x7ff13006c770 msgr2=0x7ff13015e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:29.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.516+0000 7ff135c00640 1 -- 172.21.15.28:0/2762495627 shutdown_connections 2024-04-26T20:58:29.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.517+0000 7ff135c00640 1 -- 172.21.15.28:0/2762495627 wait complete. 2024-04-26T20:58:29.539 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T20:58:29.549 INFO:tasks.ceph.mds.d.smithi103.stderr: -5335> 2024-04-26T20:58:12.648+0000 7fc512600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:58:29.624 INFO:tasks.ceph.mds.d.smithi103.stdout:starting mds.d at 2024-04-26T20:58:29.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.737+0000 7f15dda00640 1 Processor -- start 2024-04-26T20:58:29.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.738+0000 7f15dda00640 1 -- start start 2024-04-26T20:58:29.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.739+0000 7f15dda00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15d80d1de0 0x7f15d80d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:29.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.739+0000 7f15dda00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15d80d2700 con 0x7f15d815f360 2024-04-26T20:58:29.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.739+0000 7f15dda00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15d80ba350 con 0x7f15d80bac00 2024-04-26T20:58:29.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.739+0000 7f15dda00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f15d80b9ab0 con 0x7f15d80d1de0 2024-04-26T20:58:29.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.740+0000 7f15d7400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f15d815f360 0x7f15d80d3490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37130/0 (socket says 172.21.15.28:37130) 2024-04-26T20:58:29.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.740+0000 7f15d7400640 1 -- 172.21.15.28:0/653548510 learned_addr learned my addr 172.21.15.28:0/653548510 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:29.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.740+0000 7f15d7e00640 1 --2- 172.21.15.28:0/653548510 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15d80d1de0 0x7f15d80d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:29.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.740+0000 7f15d6a00640 1 -- 172.21.15.28:0/653548510 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3653352366 0 0) 0x7f15d80d2700 con 0x7f15d815f360 2024-04-26T20:58:29.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.740+0000 7f15d6a00640 1 -- 172.21.15.28:0/653548510 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15b8003520 con 0x7f15d815f360 2024-04-26T20:58:29.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.741+0000 7f15d6a00640 1 -- 172.21.15.28:0/653548510 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1824027367 0 0) 0x7f15b8003520 con 0x7f15d815f360 2024-04-26T20:58:29.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.741+0000 7f15d6a00640 1 -- 172.21.15.28:0/653548510 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15d80d1de0 msgr2=0x7f15d80d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:29.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.741+0000 7f15d6a00640 1 --2- 172.21.15.28:0/653548510 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15d80d1de0 0x7f15d80d21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:29.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.741+0000 7f15d6a00640 1 -- 172.21.15.28:0/653548510 >> v1:172.21.15.103:6789/0 conn(0x7f15d80bac00 legacy=0x7f15d80b8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:29.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.741+0000 7f15d6a00640 1 -- 172.21.15.28:0/653548510 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d80b9d90 con 0x7f15d815f360 2024-04-26T20:58:29.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.741+0000 7f15d6a00640 1 -- 172.21.15.28:0/653548510 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f15c00019f0 con 0x7f15d815f360 2024-04-26T20:58:29.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.741+0000 7f15d6a00640 1 -- 172.21.15.28:0/653548510 >> v1:172.21.15.28:6789/0 conn(0x7f15d815f360 legacy=0x7f15d80d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:29.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.741+0000 7f15d6a00640 1 --2- 172.21.15.28:0/653548510 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15d80d1de0 0x7f15b80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:29.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.741+0000 7f15d6a00640 1 --2- 172.21.15.28:0/653548510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15b8003f60 0x7f15b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:29.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.741+0000 7f15d6a00640 1 --2- 172.21.15.28:0/653548510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15b8004d40 0x7f15b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:29.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.741+0000 7f15d6a00640 1 -- 172.21.15.28:0/653548510 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f15d80b9ab0 con 0x7f15b8003f60 2024-04-26T20:58:29.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.741+0000 7f15d6a00640 1 -- 172.21.15.28:0/653548510 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f15b800baa0 con 0x7f15b8004d40 2024-04-26T20:58:29.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.741+0000 7f15d6a00640 1 -- 172.21.15.28:0/653548510 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f15b800bca0 con 0x7f15d80d1de0 2024-04-26T20:58:29.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.741+0000 7f15d7400640 1 --2- 172.21.15.28:0/653548510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15b8003f60 0x7f15b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:29.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.742+0000 7f15d7e00640 1 --2- 172.21.15.28:0/653548510 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15d80d1de0 0x7f15b80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:29.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.742+0000 7f15dd000640 1 --2- 172.21.15.28:0/653548510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15b8004d40 0x7f15b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:29.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.742+0000 7f15d7400640 1 -- 172.21.15.28:0/653548510 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15d80d1de0 msgr2=0x7f15b80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:29.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.742+0000 7f15d7400640 1 --2- 172.21.15.28:0/653548510 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15d80d1de0 0x7f15b80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:29.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.742+0000 7f15d7400640 1 -- 172.21.15.28:0/653548510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15b8004d40 msgr2=0x7f15b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:29.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.742+0000 7f15d7400640 1 --2- 172.21.15.28:0/653548510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15b8004d40 0x7f15b80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:29.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.742+0000 7f15d7400640 1 -- 172.21.15.28:0/653548510 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b8003520 con 0x7f15b8003f60 2024-04-26T20:58:29.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.742+0000 7f15d7e00640 1 --2- 172.21.15.28:0/653548510 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15d80d1de0 0x7f15b80039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:29.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.743+0000 7f15d7400640 1 --2- 172.21.15.28:0/653548510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15b8003f60 0x7f15b8004800 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f15c000a830 tx=0x7f15b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8d481278b9b054 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:29.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.743+0000 7f15d6a00640 1 -- 172.21.15.28:0/653548510 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c000a480 con 0x7f15b8003f60 2024-04-26T20:58:29.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.743+0000 7f15d6a00640 1 -- 172.21.15.28:0/653548510 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15c002f070 con 0x7f15b8003f60 2024-04-26T20:58:29.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.743+0000 7f15d6a00640 1 -- 172.21.15.28:0/653548510 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c002bd30 con 0x7f15b8003f60 2024-04-26T20:58:29.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.744+0000 7f15dda00640 1 -- 172.21.15.28:0/653548510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15b8003f60 msgr2=0x7f15b8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:29.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.744+0000 7f15dda00640 1 --2- 172.21.15.28:0/653548510 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15b8003f60 0x7f15b8004800 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f15c000a830 tx=0x7f15b8005830 comp rx=0 tx=0).stop 2024-04-26T20:58:29.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.744+0000 7f15dd000640 1 -- 172.21.15.28:0/653548510 reap_dead start 2024-04-26T20:58:29.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.744+0000 7f15dda00640 1 -- 172.21.15.28:0/653548510 shutdown_connections 2024-04-26T20:58:29.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.744+0000 7f15dda00640 1 -- 172.21.15.28:0/653548510 >> 172.21.15.28:0/653548510 conn(0x7f15d806c770 msgr2=0x7f15d806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:29.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.744+0000 7f15dda00640 1 -- 172.21.15.28:0/653548510 shutdown_connections 2024-04-26T20:58:29.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.744+0000 7f15dda00640 1 -- 172.21.15.28:0/653548510 wait complete. 2024-04-26T20:58:29.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.746+0000 7f15dda00640 1 Processor -- start 2024-04-26T20:58:29.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.746+0000 7f15dda00640 1 -- start start 2024-04-26T20:58:29.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.747+0000 7f15dda00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15b8003f60 0x7f15d80cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:29.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.747+0000 7f15dda00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15d8161410 0x7f15d80ac980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:29.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.747+0000 7f15dda00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15d815f770 0x7f15d815fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:29.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.747+0000 7f15dda00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f15d80b96e0 con 0x7f15b8003f60 2024-04-26T20:58:29.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.747+0000 7f15dda00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f15d80d2f70 con 0x7f15d8161410 2024-04-26T20:58:29.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.747+0000 7f15dda00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f15d80b9860 con 0x7f15d815f770 2024-04-26T20:58:29.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.747+0000 7f15d7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15b8003f60 0x7f15d80cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:29.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.747+0000 7f15d7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15b8003f60 0x7f15d80cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:36588/0 (socket says 172.21.15.28:36588) 2024-04-26T20:58:29.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.747+0000 7f15d7e00640 1 -- 172.21.15.28:0/2056050781 learned_addr learned my addr 172.21.15.28:0/2056050781 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:29.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.747+0000 7f15d7400640 1 --2- 172.21.15.28:0/2056050781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15d8161410 0x7f15d80ac980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:29.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.747+0000 7f15d7e00640 1 -- 172.21.15.28:0/2056050781 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15d815f770 msgr2=0x7f15d815fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:29.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.747+0000 7f15d7e00640 1 --2- 172.21.15.28:0/2056050781 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15d815f770 0x7f15d815fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:29.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.747+0000 7f15d7e00640 1 -- 172.21.15.28:0/2056050781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15d8161410 msgr2=0x7f15d80ac980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:29.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.747+0000 7f15d7e00640 1 --2- 172.21.15.28:0/2056050781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15d8161410 0x7f15d80ac980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:29.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.747+0000 7f15d7e00640 1 -- 172.21.15.28:0/2056050781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d8161960 con 0x7f15b8003f60 2024-04-26T20:58:29.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.748+0000 7f15d7e00640 1 --2- 172.21.15.28:0/2056050781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15b8003f60 0x7f15d80cd220 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f15c80108e0 tx=0x7f15c8010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:29.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.748+0000 7f15cfe00640 1 -- 172.21.15.28:0/2056050781 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c800f040 con 0x7f15b8003f60 2024-04-26T20:58:29.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.748+0000 7f15cfe00640 1 -- 172.21.15.28:0/2056050781 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15c800ec20 con 0x7f15b8003f60 2024-04-26T20:58:29.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.748+0000 7f15cfe00640 1 -- 172.21.15.28:0/2056050781 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c801c400 con 0x7f15b8003f60 2024-04-26T20:58:29.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.748+0000 7f15dda00640 1 -- 172.21.15.28:0/2056050781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d8161ca0 con 0x7f15b8003f60 2024-04-26T20:58:29.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.748+0000 7f15dda00640 1 -- 172.21.15.28:0/2056050781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d81f4d90 con 0x7f15b8003f60 2024-04-26T20:58:29.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.749+0000 7f15dda00640 1 -- 172.21.15.28:0/2056050781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d806a250 con 0x7f15b8003f60 2024-04-26T20:58:29.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.750+0000 7f15cfe00640 1 -- 172.21.15.28:0/2056050781 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f15c801c5a0 con 0x7f15b8003f60 2024-04-26T20:58:29.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.751+0000 7f15cfe00640 1 --2- 172.21.15.28:0/2056050781 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f15b406c8f0 0x7f15b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:29.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.751+0000 7f15cfe00640 1 -- 172.21.15.28:0/2056050781 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f15c808c4b0 con 0x7f15b8003f60 2024-04-26T20:58:29.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.752+0000 7f15d7400640 1 --2- 172.21.15.28:0/2056050781 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f15b406c8f0 0x7f15b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:29.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.752+0000 7f15d7400640 1 --2- 172.21.15.28:0/2056050781 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f15b406c8f0 0x7f15b406edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f15c0033000 tx=0x7f15c0009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:29.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:29.756+0000 7f15cfe00640 1 -- 172.21.15.28:0/2056050781 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f15d8161ca0 con 0x7f15b8003f60 2024-04-26T20:58:30.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.005+0000 7f15dda00640 1 -- 172.21.15.28:0/2056050781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f15d80d5b80 con 0x7f15b8003f60 2024-04-26T20:58:30.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.516+0000 7f15cfe00640 1 -- 172.21.15.28:0/2056050781 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f15c8018030 con 0x7f15b8003f60 2024-04-26T20:58:30.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.519+0000 7f15dda00640 1 -- 172.21.15.28:0/2056050781 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f15b406c8f0 msgr2=0x7f15b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:30.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.519+0000 7f15dda00640 1 --2- 172.21.15.28:0/2056050781 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f15b406c8f0 0x7f15b406edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f15c0033000 tx=0x7f15c0009030 comp rx=0 tx=0).stop 2024-04-26T20:58:30.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.519+0000 7f15dda00640 1 -- 172.21.15.28:0/2056050781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15b8003f60 msgr2=0x7f15d80cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:30.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.519+0000 7f15dda00640 1 --2- 172.21.15.28:0/2056050781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15b8003f60 0x7f15d80cd220 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f15c80108e0 tx=0x7f15c8010da0 comp rx=0 tx=0).stop 2024-04-26T20:58:30.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.520+0000 7f15dda00640 1 -- 172.21.15.28:0/2056050781 shutdown_connections 2024-04-26T20:58:30.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.520+0000 7f15dda00640 1 --2- 172.21.15.28:0/2056050781 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f15b406c8f0 0x7f15b406edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:30.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.520+0000 7f15dda00640 1 --2- 172.21.15.28:0/2056050781 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f15d815f770 0x7f15d815fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:30.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.520+0000 7f15dda00640 1 --2- 172.21.15.28:0/2056050781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f15d8161410 0x7f15d80ac980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:30.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.520+0000 7f15dda00640 1 --2- 172.21.15.28:0/2056050781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15b8003f60 0x7f15d80cd220 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:30.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.520+0000 7f15dda00640 1 -- 172.21.15.28:0/2056050781 >> 172.21.15.28:0/2056050781 conn(0x7f15d806c770 msgr2=0x7f15d80d39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:30.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.520+0000 7f15dda00640 1 -- 172.21.15.28:0/2056050781 shutdown_connections 2024-04-26T20:58:30.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.520+0000 7f15dda00640 1 -- 172.21.15.28:0/2056050781 wait complete. 2024-04-26T20:58:30.545 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T20:58:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.766+0000 7fa44b200640 1 Processor -- start 2024-04-26T20:58:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.766+0000 7fa44b200640 1 -- start start 2024-04-26T20:58:30.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.768+0000 7fa44b200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa44c0d2540 0x7fa44c0baa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:30.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.768+0000 7fa44b200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa44c0d32b0 0x7fa44c0b8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:30.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.768+0000 7fa44b200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa44c0d3a20 con 0x7fa44c15f0d0 2024-04-26T20:58:30.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.768+0000 7fa44b200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fa44c15f4b0 con 0x7fa44c0d2540 2024-04-26T20:58:30.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.768+0000 7fa44b200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa44c0d3d00 con 0x7fa44c0d32b0 2024-04-26T20:58:30.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.769+0000 7fa44a800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa44c0d32b0 0x7fa44c0b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:30.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.769+0000 7fa449e00640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7fa44c15f0d0 0x7fa44c0d1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.28:43756/0 (socket says 172.21.15.28:43756) 2024-04-26T20:58:30.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.769+0000 7fa449e00640 1 -- 172.21.15.28:0/990179415 learned_addr learned my addr 172.21.15.28:0/990179415 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:30.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.769+0000 7fa443e00640 1 --2- 172.21.15.28:0/990179415 >> v2:172.21.15.103:3300/0 conn(0x7fa44c0d2540 0x7fa44c0baa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:30.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.770+0000 7fa449400640 1 -- 172.21.15.28:0/990179415 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2951398157 0 0) 0x7fa44c0d3a20 con 0x7fa44c15f0d0 2024-04-26T20:58:30.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.770+0000 7fa449400640 1 -- 172.21.15.28:0/990179415 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa430003520 con 0x7fa44c15f0d0 2024-04-26T20:58:30.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.770+0000 7fa443e00640 1 -- 172.21.15.28:0/990179415 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa44c0d32b0 msgr2=0x7fa44c0b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:30.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.770+0000 7fa443e00640 1 --2- 172.21.15.28:0/990179415 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa44c0d32b0 0x7fa44c0b8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:30.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.770+0000 7fa443e00640 1 -- 172.21.15.28:0/990179415 >> v1:172.21.15.103:6789/0 conn(0x7fa44c15f0d0 legacy=0x7fa44c0d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:30.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.770+0000 7fa443e00640 1 -- 172.21.15.28:0/990179415 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44c0d3fe0 con 0x7fa44c0d2540 2024-04-26T20:58:30.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.772+0000 7fa443e00640 1 --2- 172.21.15.28:0/990179415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa44c0d2540 0x7fa44c0baa20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa4340097a0 tx=0x7fa43402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a90c35b21d04560b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:30.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.772+0000 7fa449400640 1 -- 172.21.15.28:0/990179415 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434037070 con 0x7fa44c0d2540 2024-04-26T20:58:30.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.772+0000 7fa449400640 1 -- 172.21.15.28:0/990179415 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa434032440 con 0x7fa44c0d2540 2024-04-26T20:58:30.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.772+0000 7fa449400640 1 -- 172.21.15.28:0/990179415 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434032810 con 0x7fa44c0d2540 2024-04-26T20:58:30.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.773+0000 7fa44b200640 1 -- 172.21.15.28:0/990179415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa44c0d2540 msgr2=0x7fa44c0baa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:30.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.773+0000 7fa44b200640 1 --2- 172.21.15.28:0/990179415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa44c0d2540 0x7fa44c0baa20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa4340097a0 tx=0x7fa43402a180 comp rx=0 tx=0).stop 2024-04-26T20:58:30.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.773+0000 7fa44b200640 1 -- 172.21.15.28:0/990179415 shutdown_connections 2024-04-26T20:58:30.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.773+0000 7fa44b200640 1 --2- 172.21.15.28:0/990179415 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa44c0d32b0 0x7fa44c0b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:30.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.773+0000 7fa44b200640 1 --2- 172.21.15.28:0/990179415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa44c0d2540 0x7fa44c0baa20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:30.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.773+0000 7fa44b200640 1 -- 172.21.15.28:0/990179415 >> 172.21.15.28:0/990179415 conn(0x7fa44c06c770 msgr2=0x7fa44c06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:30.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.773+0000 7fa44b200640 1 -- 172.21.15.28:0/990179415 shutdown_connections 2024-04-26T20:58:30.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.773+0000 7fa44b200640 1 -- 172.21.15.28:0/990179415 wait complete. 2024-04-26T20:58:30.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.775+0000 7fa44b200640 1 Processor -- start 2024-04-26T20:58:30.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.775+0000 7fa44b200640 1 -- start start 2024-04-26T20:58:30.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.776+0000 7fa44b200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa44c0d2540 0x7fa44c15bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:30.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.776+0000 7fa44b200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa44c0d32b0 0x7fa44c14e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:30.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.776+0000 7fa44b200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa44c15f0d0 0x7fa44c156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:30.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.776+0000 7fa44b200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c0d3700 con 0x7fa44c0d32b0 2024-04-26T20:58:30.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.776+0000 7fa44b200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa44c0d45a0 con 0x7fa44c15f0d0 2024-04-26T20:58:30.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.776+0000 7fa44b200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa44c0d4ac0 con 0x7fa44c0d2540 2024-04-26T20:58:30.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.777+0000 7fa443e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa44c0d32b0 0x7fa44c14e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:30.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.777+0000 7fa443e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa44c0d32b0 0x7fa44c14e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44702/0 (socket says 172.21.15.28:44702) 2024-04-26T20:58:30.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.777+0000 7fa443e00640 1 -- 172.21.15.28:0/3601421781 learned_addr learned my addr 172.21.15.28:0/3601421781 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:30.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.777+0000 7fa44a800640 1 --2- 172.21.15.28:0/3601421781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa44c15f0d0 0x7fa44c156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:30.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.777+0000 7fa449e00640 1 --2- 172.21.15.28:0/3601421781 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa44c0d2540 0x7fa44c15bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.777+0000 7fa443e00640 1 -- 172.21.15.28:0/3601421781 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa44c0d2540 msgr2=0x7fa44c15bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.777+0000 7fa443e00640 1 --2- 172.21.15.28:0/3601421781 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa44c0d2540 0x7fa44c15bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.777+0000 7fa443e00640 1 -- 172.21.15.28:0/3601421781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa44c15f0d0 msgr2=0x7fa44c156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.777+0000 7fa443e00640 1 --2- 172.21.15.28:0/3601421781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa44c15f0d0 0x7fa44c156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.777+0000 7fa443e00640 1 -- 172.21.15.28:0/3601421781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa44c0cc050 con 0x7fa44c0d32b0 2024-04-26T20:58:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.778+0000 7fa443e00640 1 --2- 172.21.15.28:0/3601421781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa44c0d32b0 0x7fa44c14e2a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa434009890 tx=0x7fa4340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.778+0000 7fa442000640 1 -- 172.21.15.28:0/3601421781 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa434037050 con 0x7fa44c0d32b0 2024-04-26T20:58:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.778+0000 7fa442000640 1 -- 172.21.15.28:0/3601421781 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa434043070 con 0x7fa44c0d32b0 2024-04-26T20:58:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.778+0000 7fa442000640 1 -- 172.21.15.28:0/3601421781 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43403cb30 con 0x7fa44c0d32b0 2024-04-26T20:58:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.778+0000 7fa44b200640 1 -- 172.21.15.28:0/3601421781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa44c0cc2f0 con 0x7fa44c0d32b0 2024-04-26T20:58:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.778+0000 7fa44b200640 1 -- 172.21.15.28:0/3601421781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa44c0cc8b0 con 0x7fa44c0d32b0 2024-04-26T20:58:30.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.779+0000 7fa44b200640 1 -- 172.21.15.28:0/3601421781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa44c06a1e0 con 0x7fa44c0d32b0 2024-04-26T20:58:30.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.780+0000 7fa442000640 1 -- 172.21.15.28:0/3601421781 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa43403ccd0 con 0x7fa44c0d32b0 2024-04-26T20:58:30.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.781+0000 7fa442000640 1 --2- 172.21.15.28:0/3601421781 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa42806c8f0 0x7fa42806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:30.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.781+0000 7fa442000640 1 -- 172.21.15.28:0/3601421781 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7fa4340ac910 con 0x7fa44c0d32b0 2024-04-26T20:58:30.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.782+0000 7fa449e00640 1 --2- 172.21.15.28:0/3601421781 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa42806c8f0 0x7fa42806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:30.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.782+0000 7fa449e00640 1 --2- 172.21.15.28:0/3601421781 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa42806c8f0 0x7fa42806edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0011e0 tx=0x7fa43c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:30.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.787+0000 7fa442000640 1 -- 172.21.15.28:0/3601421781 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa434046360 con 0x7fa44c0d32b0 2024-04-26T20:58:30.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:30.996+0000 7fa44b200640 1 -- 172.21.15.28:0/3601421781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa44c0b9fa0 con 0x7fa44c0d32b0 2024-04-26T20:58:31.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:31.931+0000 7fa442000640 1 -- 172.21.15.28:0/3601421781 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) ==== 181+0+0 (secure 0 0 0) 0x7fa43407ac40 con 0x7fa44c0d32b0 2024-04-26T20:58:31.933 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' removed 2024-04-26T20:58:31.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:31.936+0000 7fa44b200640 1 -- 172.21.15.28:0/3601421781 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa42806c8f0 msgr2=0x7fa42806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:31.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:31.936+0000 7fa44b200640 1 --2- 172.21.15.28:0/3601421781 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa42806c8f0 0x7fa42806edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0011e0 tx=0x7fa43c009040 comp rx=0 tx=0).stop 2024-04-26T20:58:31.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:31.936+0000 7fa44b200640 1 -- 172.21.15.28:0/3601421781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa44c0d32b0 msgr2=0x7fa44c14e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:31.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:31.936+0000 7fa44b200640 1 --2- 172.21.15.28:0/3601421781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa44c0d32b0 0x7fa44c14e2a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa434009890 tx=0x7fa4340046c0 comp rx=0 tx=0).stop 2024-04-26T20:58:31.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:31.937+0000 7fa44b200640 1 -- 172.21.15.28:0/3601421781 shutdown_connections 2024-04-26T20:58:31.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:31.937+0000 7fa44b200640 1 --2- 172.21.15.28:0/3601421781 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa42806c8f0 0x7fa42806edb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:31.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:31.937+0000 7fa44b200640 1 --2- 172.21.15.28:0/3601421781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa44c15f0d0 0x7fa44c156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:31.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:31.937+0000 7fa44b200640 1 --2- 172.21.15.28:0/3601421781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa44c0d32b0 0x7fa44c14e2a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:31.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:31.937+0000 7fa44b200640 1 --2- 172.21.15.28:0/3601421781 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa44c0d2540 0x7fa44c15bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:31.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:31.937+0000 7fa44b200640 1 -- 172.21.15.28:0/3601421781 >> 172.21.15.28:0/3601421781 conn(0x7fa44c06c770 msgr2=0x7fa44c0b84b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:31.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:31.938+0000 7fa44b200640 1 -- 172.21.15.28:0/3601421781 shutdown_connections 2024-04-26T20:58:31.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:31.938+0000 7fa44b200640 1 -- 172.21.15.28:0/3601421781 wait complete. 2024-04-26T20:58:31.962 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T20:58:32.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.162+0000 7fe7b2400640 1 Processor -- start 2024-04-26T20:58:32.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.162+0000 7fe7b2400640 1 -- start start 2024-04-26T20:58:32.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.164+0000 7fe7b2400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7ac1434c0 0x7fe7ac160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:32.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.164+0000 7fe7b2400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7ac06f6d0 con 0x7fe7ac161070 2024-04-26T20:58:32.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.164+0000 7fe7b2400640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7ac0ceb10 con 0x7fe7ac06f0b0 2024-04-26T20:58:32.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.164+0000 7fe7b2400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe7ac143ce0 con 0x7fe7ac1434c0 2024-04-26T20:58:32.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.164+0000 7fe7abe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe7ac161070 0x7fe7ac06e9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:44588/0 (socket says 172.21.15.28:44588) 2024-04-26T20:58:32.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.164+0000 7fe7abe00640 1 -- 172.21.15.28:0/2486379219 learned_addr learned my addr 172.21.15.28:0/2486379219 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:32.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.165+0000 7fe7b1000640 1 --2- 172.21.15.28:0/2486379219 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7ac1434c0 0x7fe7ac160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:32.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.165+0000 7fe7ab400640 1 -- 172.21.15.28:0/2486379219 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 698336909 0 0) 0x7fe7ac06f6d0 con 0x7fe7ac161070 2024-04-26T20:58:32.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.165+0000 7fe7ab400640 1 -- 172.21.15.28:0/2486379219 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe794003520 con 0x7fe7ac161070 2024-04-26T20:58:32.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.165+0000 7fe7ab400640 1 -- 172.21.15.28:0/2486379219 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2083804055 0 0) 0x7fe794003520 con 0x7fe7ac161070 2024-04-26T20:58:32.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.165+0000 7fe7ab400640 1 -- 172.21.15.28:0/2486379219 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7ac1434c0 msgr2=0x7fe7ac160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:32.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.165+0000 7fe7ab400640 1 --2- 172.21.15.28:0/2486379219 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7ac1434c0 0x7fe7ac160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:32.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.165+0000 7fe7ab400640 1 -- 172.21.15.28:0/2486379219 >> v1:172.21.15.103:6789/0 conn(0x7fe7ac06f0b0 legacy=0x7fe7ac0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:32.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.165+0000 7fe7ab400640 1 -- 172.21.15.28:0/2486379219 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ac0cedf0 con 0x7fe7ac161070 2024-04-26T20:58:32.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.165+0000 7fe7ab400640 1 -- 172.21.15.28:0/2486379219 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fe798001670 con 0x7fe7ac161070 2024-04-26T20:58:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.165+0000 7fe7ab400640 1 -- 172.21.15.28:0/2486379219 >> v1:172.21.15.28:6789/0 conn(0x7fe7ac161070 legacy=0x7fe7ac06e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.166+0000 7fe7ab400640 1 --2- 172.21.15.28:0/2486379219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7940039e0 0x7fe794003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.166+0000 7fe7ab400640 1 --2- 172.21.15.28:0/2486379219 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe794004800 0x7fe794004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.166+0000 7fe7ab400640 1 --2- 172.21.15.28:0/2486379219 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7ac1434c0 0x7fe7940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.166+0000 7fe7ab400640 1 -- 172.21.15.28:0/2486379219 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe79400bb00 con 0x7fe794004800 2024-04-26T20:58:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.166+0000 7fe7ab400640 1 -- 172.21.15.28:0/2486379219 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe79400bc80 con 0x7fe7940039e0 2024-04-26T20:58:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.166+0000 7fe7ab400640 1 -- 172.21.15.28:0/2486379219 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe79400be80 con 0x7fe7ac1434c0 2024-04-26T20:58:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.166+0000 7fe7abe00640 1 --2- 172.21.15.28:0/2486379219 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe794004800 0x7fe794004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.166+0000 7fe7b1000640 1 --2- 172.21.15.28:0/2486379219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7940039e0 0x7fe794003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.166+0000 7fe7b1a00640 1 --2- 172.21.15.28:0/2486379219 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7ac1434c0 0x7fe7940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:32.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.166+0000 7fe7abe00640 1 -- 172.21.15.28:0/2486379219 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7ac1434c0 msgr2=0x7fe7940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:32.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.166+0000 7fe7abe00640 1 --2- 172.21.15.28:0/2486379219 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7ac1434c0 0x7fe7940051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:32.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.166+0000 7fe7abe00640 1 -- 172.21.15.28:0/2486379219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7940039e0 msgr2=0x7fe794003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:32.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.166+0000 7fe7abe00640 1 --2- 172.21.15.28:0/2486379219 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe7940039e0 0x7fe794003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:32.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.166+0000 7fe7abe00640 1 -- 172.21.15.28:0/2486379219 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe794003520 con 0x7fe794004800 2024-04-26T20:58:32.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.167+0000 7fe7abe00640 1 --2- 172.21.15.28:0/2486379219 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe794004800 0x7fe794004c80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe79800c210 tx=0x7fe794005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3004364e0926bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:32.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.168+0000 7fe7ab400640 1 -- 172.21.15.28:0/2486379219 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe798007cf0 con 0x7fe794004800 2024-04-26T20:58:32.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.168+0000 7fe7ab400640 1 -- 172.21.15.28:0/2486379219 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe798005ad0 con 0x7fe794004800 2024-04-26T20:58:32.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.168+0000 7fe7ab400640 1 -- 172.21.15.28:0/2486379219 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe798007070 con 0x7fe794004800 2024-04-26T20:58:32.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.168+0000 7fe7b2400640 1 -- 172.21.15.28:0/2486379219 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe794004800 msgr2=0x7fe794004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:32.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.168+0000 7fe7b2400640 1 --2- 172.21.15.28:0/2486379219 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe794004800 0x7fe794004c80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe79800c210 tx=0x7fe794005830 comp rx=0 tx=0).stop 2024-04-26T20:58:32.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.168+0000 7fe7b1a00640 1 -- 172.21.15.28:0/2486379219 reap_dead start 2024-04-26T20:58:32.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.168+0000 7fe7b2400640 1 -- 172.21.15.28:0/2486379219 shutdown_connections 2024-04-26T20:58:32.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.168+0000 7fe7b2400640 1 -- 172.21.15.28:0/2486379219 >> 172.21.15.28:0/2486379219 conn(0x7fe7ac0b2820 msgr2=0x7fe7ac0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:32.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.169+0000 7fe7b2400640 1 -- 172.21.15.28:0/2486379219 shutdown_connections 2024-04-26T20:58:32.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.169+0000 7fe7b2400640 1 -- 172.21.15.28:0/2486379219 wait complete. 2024-04-26T20:58:32.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.170+0000 7fe7b2400640 1 Processor -- start 2024-04-26T20:58:32.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.170+0000 7fe7b2400640 1 -- start start 2024-04-26T20:58:32.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.171+0000 7fe7b2400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe794004800 0x7fe7ac15bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:32.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.171+0000 7fe7b2400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7ac15c450 0x7fe7ac15e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:32.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.171+0000 7fe7b2400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7ac13da10 0x7fe7ac13de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:32.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.171+0000 7fe7b2400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac1438a0 con 0x7fe7ac15c450 2024-04-26T20:58:32.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.171+0000 7fe7b2400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac1614b0 con 0x7fe794004800 2024-04-26T20:58:32.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.171+0000 7fe7b2400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe7ac143a20 con 0x7fe7ac13da10 2024-04-26T20:58:32.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.172+0000 7fe7abe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7ac15c450 0x7fe7ac15e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:32.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.172+0000 7fe7abe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7ac15c450 0x7fe7ac15e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44708/0 (socket says 172.21.15.28:44708) 2024-04-26T20:58:32.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.172+0000 7fe7abe00640 1 -- 172.21.15.28:0/773780101 learned_addr learned my addr 172.21.15.28:0/773780101 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:32.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.172+0000 7fe7b1a00640 1 --2- 172.21.15.28:0/773780101 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7ac13da10 0x7fe7ac13de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:32.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.172+0000 7fe7b1000640 1 --2- 172.21.15.28:0/773780101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe794004800 0x7fe7ac15bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:32.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.172+0000 7fe7abe00640 1 -- 172.21.15.28:0/773780101 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7ac13da10 msgr2=0x7fe7ac13de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:32.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.172+0000 7fe7abe00640 1 --2- 172.21.15.28:0/773780101 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7ac13da10 0x7fe7ac13de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:32.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.172+0000 7fe7abe00640 1 -- 172.21.15.28:0/773780101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe794004800 msgr2=0x7fe7ac15bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:32.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.172+0000 7fe7abe00640 1 --2- 172.21.15.28:0/773780101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe794004800 0x7fe7ac15bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:32.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.172+0000 7fe7abe00640 1 -- 172.21.15.28:0/773780101 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ac160060 con 0x7fe7ac15c450 2024-04-26T20:58:32.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.173+0000 7fe7abe00640 1 --2- 172.21.15.28:0/773780101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7ac15c450 0x7fe7ac15e9b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe798001970 tx=0x7fe7ac06f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:32.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.173+0000 7fe7a8c00640 1 -- 172.21.15.28:0/773780101 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe798001630 con 0x7fe7ac15c450 2024-04-26T20:58:32.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.173+0000 7fe7a8c00640 1 -- 172.21.15.28:0/773780101 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe798038040 con 0x7fe7ac15c450 2024-04-26T20:58:32.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.173+0000 7fe7a8c00640 1 -- 172.21.15.28:0/773780101 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe798007290 con 0x7fe7ac15c450 2024-04-26T20:58:32.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.173+0000 7fe7b2400640 1 -- 172.21.15.28:0/773780101 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ac160340 con 0x7fe7ac15c450 2024-04-26T20:58:32.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.173+0000 7fe7b2400640 1 -- 172.21.15.28:0/773780101 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ac15eef0 con 0x7fe7ac15c450 2024-04-26T20:58:32.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.174+0000 7fe7b2400640 1 -- 172.21.15.28:0/773780101 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7ac06a3d0 con 0x7fe7ac15c450 2024-04-26T20:58:32.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.175+0000 7fe7a8c00640 1 -- 172.21.15.28:0/773780101 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe798007430 con 0x7fe7ac15c450 2024-04-26T20:58:32.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.176+0000 7fe7a8c00640 1 --2- 172.21.15.28:0/773780101 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe78c06c920 0x7fe78c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:32.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.176+0000 7fe7a8c00640 1 -- 172.21.15.28:0/773780101 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5357+0+0 (secure 0 0 0) 0x7fe7980abca0 con 0x7fe7ac15c450 2024-04-26T20:58:32.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.177+0000 7fe7b1000640 1 --2- 172.21.15.28:0/773780101 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe78c06c920 0x7fe78c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:32.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.177+0000 7fe7b1000640 1 --2- 172.21.15.28:0/773780101 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe78c06c920 0x7fe78c06ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009a10 tx=0x7fe7a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:32.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.181+0000 7fe7a8c00640 1 -- 172.21.15.28:0/773780101 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe798005aa0 con 0x7fe7ac15c450 2024-04-26T20:58:32.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.386+0000 7fe7b2400640 1 -- 172.21.15.28:0/773780101 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe7ac0ce320 con 0x7fe7ac15c450 2024-04-26T20:58:32.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.944+0000 7fe7a8c00640 1 -- 172.21.15.28:0/773780101 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) ==== 169+0+0 (secure 0 0 0) 0x7fe79807a230 con 0x7fe7ac15c450 2024-04-26T20:58:32.946 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' removed 2024-04-26T20:58:32.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.950+0000 7fe7b2400640 1 -- 172.21.15.28:0/773780101 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe78c06c920 msgr2=0x7fe78c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:32.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.950+0000 7fe7b2400640 1 --2- 172.21.15.28:0/773780101 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe78c06c920 0x7fe78c06ede0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009a10 tx=0x7fe7a0008040 comp rx=0 tx=0).stop 2024-04-26T20:58:32.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.951+0000 7fe7b2400640 1 -- 172.21.15.28:0/773780101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7ac15c450 msgr2=0x7fe7ac15e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:32.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.951+0000 7fe7b2400640 1 --2- 172.21.15.28:0/773780101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7ac15c450 0x7fe7ac15e9b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe798001970 tx=0x7fe7ac06f8a0 comp rx=0 tx=0).stop 2024-04-26T20:58:32.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.951+0000 7fe7b2400640 1 -- 172.21.15.28:0/773780101 shutdown_connections 2024-04-26T20:58:32.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.951+0000 7fe7b2400640 1 --2- 172.21.15.28:0/773780101 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe78c06c920 0x7fe78c06ede0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:32.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.951+0000 7fe7b2400640 1 --2- 172.21.15.28:0/773780101 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe7ac13da10 0x7fe7ac13de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:32.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.951+0000 7fe7b2400640 1 --2- 172.21.15.28:0/773780101 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7ac15c450 0x7fe7ac15e9b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:32.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.951+0000 7fe7b2400640 1 --2- 172.21.15.28:0/773780101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe794004800 0x7fe7ac15bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:32.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.951+0000 7fe7b2400640 1 -- 172.21.15.28:0/773780101 >> 172.21.15.28:0/773780101 conn(0x7fe7ac0b2820 msgr2=0x7fe7ac0d0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:32.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.952+0000 7fe7b2400640 1 -- 172.21.15.28:0/773780101 shutdown_connections 2024-04-26T20:58:32.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:32.952+0000 7fe7b2400640 1 -- 172.21.15.28:0/773780101 wait complete. 2024-04-26T20:58:32.983 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T20:58:33.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.164+0000 7fed47a00640 1 Processor -- start 2024-04-26T20:58:33.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.164+0000 7fed47a00640 1 -- start start 2024-04-26T20:58:33.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.166+0000 7fed47a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fed40145900 0x7fed40163870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:33.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.166+0000 7fed47a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fed4006f9e0 0x7fed400b4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:33.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.166+0000 7fed47a00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed40073ae0 con 0x7fed40073350 2024-04-26T20:58:33.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.166+0000 7fed47a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fed40073dc0 con 0x7fed40145900 2024-04-26T20:58:33.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.166+0000 7fed47a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fed400740a0 con 0x7fed4006f9e0 2024-04-26T20:58:33.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.167+0000 7fed45000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fed40145900 0x7fed40163870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:33.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.167+0000 7fed45000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fed40145900 0x7fed40163870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44718/0 (socket says 172.21.15.28:44718) 2024-04-26T20:58:33.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.167+0000 7fed45000640 1 -- 172.21.15.28:0/4278742251 learned_addr learned my addr 172.21.15.28:0/4278742251 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:33.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.167+0000 7fed45a00640 1 --2- 172.21.15.28:0/4278742251 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fed4006f9e0 0x7fed400b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:33.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.167+0000 7fed45000640 1 -- 172.21.15.28:0/4278742251 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fed4006f9e0 msgr2=0x7fed400b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:33.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.167+0000 7fed45000640 1 --2- 172.21.15.28:0/4278742251 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fed4006f9e0 0x7fed400b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:33.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.167+0000 7fed45000640 1 -- 172.21.15.28:0/4278742251 >> v1:172.21.15.103:6789/0 conn(0x7fed40073350 legacy=0x7fed4006f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:33.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.167+0000 7fed45000640 1 -- 172.21.15.28:0/4278742251 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed40074380 con 0x7fed40145900 2024-04-26T20:58:33.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.167+0000 7fed45a00640 1 --2- 172.21.15.28:0/4278742251 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fed4006f9e0 0x7fed400b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:33.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.168+0000 7fed45000640 1 --2- 172.21.15.28:0/4278742251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed40145900 0x7fed40163870 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fed28009870 tx=0x7fed2802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7149b7717863218 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:33.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.169+0000 7fed3fe00640 1 -- 172.21.15.28:0/4278742251 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28037070 con 0x7fed40145900 2024-04-26T20:58:33.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.169+0000 7fed3fe00640 1 -- 172.21.15.28:0/4278742251 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed28032440 con 0x7fed40145900 2024-04-26T20:58:33.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.169+0000 7fed3fe00640 1 -- 172.21.15.28:0/4278742251 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28032810 con 0x7fed40145900 2024-04-26T20:58:33.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.169+0000 7fed47a00640 1 -- 172.21.15.28:0/4278742251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed40145900 msgr2=0x7fed40163870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:33.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.169+0000 7fed47a00640 1 --2- 172.21.15.28:0/4278742251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed40145900 0x7fed40163870 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fed28009870 tx=0x7fed2802a210 comp rx=0 tx=0).stop 2024-04-26T20:58:33.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.169+0000 7fed47a00640 1 -- 172.21.15.28:0/4278742251 shutdown_connections 2024-04-26T20:58:33.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.169+0000 7fed47a00640 1 --2- 172.21.15.28:0/4278742251 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fed4006f9e0 0x7fed400b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:33.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.169+0000 7fed47a00640 1 --2- 172.21.15.28:0/4278742251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed40145900 0x7fed40163870 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:33.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.169+0000 7fed47a00640 1 -- 172.21.15.28:0/4278742251 >> 172.21.15.28:0/4278742251 conn(0x7fed400b2b60 msgr2=0x7fed400b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:33.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.169+0000 7fed47a00640 1 -- 172.21.15.28:0/4278742251 shutdown_connections 2024-04-26T20:58:33.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.169+0000 7fed47a00640 1 -- 172.21.15.28:0/4278742251 wait complete. 2024-04-26T20:58:33.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.171+0000 7fed47a00640 1 Processor -- start 2024-04-26T20:58:33.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.171+0000 7fed47a00640 1 -- start start 2024-04-26T20:58:33.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.172+0000 7fed47a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed4006f9e0 0x7fed40141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:33.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.172+0000 7fed47a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40073350 0x7fed401f8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:33.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.172+0000 7fed47a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fed40145900 0x7fed401f9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:33.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.172+0000 7fed47a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fed400b4900 con 0x7fed4006f9e0 2024-04-26T20:58:33.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.172+0000 7fed47a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed400b4780 con 0x7fed40073350 2024-04-26T20:58:33.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.172+0000 7fed47a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fed400b4a80 con 0x7fed40145900 2024-04-26T20:58:33.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.173+0000 7fed45000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed4006f9e0 0x7fed40141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:33.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.173+0000 7fed45000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed4006f9e0 0x7fed40141fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44728/0 (socket says 172.21.15.28:44728) 2024-04-26T20:58:33.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.173+0000 7fed45000640 1 -- 172.21.15.28:0/2435107822 learned_addr learned my addr 172.21.15.28:0/2435107822 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:33.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.173+0000 7fed37e00640 1 --2- 172.21.15.28:0/2435107822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40073350 0x7fed401f8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:33.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.173+0000 7fed45000640 1 -- 172.21.15.28:0/2435107822 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fed40145900 msgr2=0x7fed401f9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:33.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.173+0000 7fed45000640 1 --2- 172.21.15.28:0/2435107822 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fed40145900 0x7fed401f9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:33.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.173+0000 7fed45000640 1 -- 172.21.15.28:0/2435107822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40073350 msgr2=0x7fed401f8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:33.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.173+0000 7fed45000640 1 --2- 172.21.15.28:0/2435107822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40073350 0x7fed401f8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:33.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.173+0000 7fed45000640 1 -- 172.21.15.28:0/2435107822 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed401f6690 con 0x7fed4006f9e0 2024-04-26T20:58:33.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.173+0000 7fed45000640 1 --2- 172.21.15.28:0/2435107822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed4006f9e0 0x7fed40141fe0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fed28009870 tx=0x7fed280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:33.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.174+0000 7fed3d600640 1 -- 172.21.15.28:0/2435107822 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28041020 con 0x7fed4006f9e0 2024-04-26T20:58:33.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.174+0000 7fed3d600640 1 -- 172.21.15.28:0/2435107822 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed28030970 con 0x7fed4006f9e0 2024-04-26T20:58:33.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.174+0000 7fed3d600640 1 -- 172.21.15.28:0/2435107822 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed28037040 con 0x7fed4006f9e0 2024-04-26T20:58:33.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.174+0000 7fed47a00640 1 -- 172.21.15.28:0/2435107822 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed400cfc40 con 0x7fed4006f9e0 2024-04-26T20:58:33.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.174+0000 7fed47a00640 1 -- 172.21.15.28:0/2435107822 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed400d0200 con 0x7fed4006f9e0 2024-04-26T20:58:33.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.175+0000 7fed47a00640 1 -- 172.21.15.28:0/2435107822 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed40076520 con 0x7fed4006f9e0 2024-04-26T20:58:33.177 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.177+0000 7fed3d600640 1 -- 172.21.15.28:0/2435107822 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed28044020 con 0x7fed4006f9e0 2024-04-26T20:58:33.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.178+0000 7fed3d600640 1 --2- 172.21.15.28:0/2435107822 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fed1c06c8f0 0x7fed1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:33.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.178+0000 7fed3d600640 1 -- 172.21.15.28:0/2435107822 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fed280ac240 con 0x7fed4006f9e0 2024-04-26T20:58:33.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.178+0000 7fed37e00640 1 --2- 172.21.15.28:0/2435107822 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fed1c06c8f0 0x7fed1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:33.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.179+0000 7fed37e00640 1 --2- 172.21.15.28:0/2435107822 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fed1c06c8f0 0x7fed1c06edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fed400b5930 tx=0x7fed30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:33.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.183+0000 7fed3d600640 1 -- 172.21.15.28:0/2435107822 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fed2804b030 con 0x7fed4006f9e0 2024-04-26T20:58:33.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.409+0000 7fed47a00640 1 -- 172.21.15.28:0/2435107822 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fed401f6860 con 0x7fed4006f9e0 2024-04-26T20:58:33.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.416+0000 7fed3d600640 1 -- 172.21.15.28:0/2435107822 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7fed2807a970 con 0x7fed4006f9e0 2024-04-26T20:58:33.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.418+0000 7fed47a00640 1 -- 172.21.15.28:0/2435107822 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fed1c06c8f0 msgr2=0x7fed1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:33.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.418+0000 7fed47a00640 1 --2- 172.21.15.28:0/2435107822 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fed1c06c8f0 0x7fed1c06edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fed400b5930 tx=0x7fed30009040 comp rx=0 tx=0).stop 2024-04-26T20:58:33.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.418+0000 7fed47a00640 1 -- 172.21.15.28:0/2435107822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed4006f9e0 msgr2=0x7fed40141fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:33.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.418+0000 7fed47a00640 1 --2- 172.21.15.28:0/2435107822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed4006f9e0 0x7fed40141fe0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fed28009870 tx=0x7fed280046c0 comp rx=0 tx=0).stop 2024-04-26T20:58:33.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.419+0000 7fed47a00640 1 -- 172.21.15.28:0/2435107822 shutdown_connections 2024-04-26T20:58:33.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.419+0000 7fed47a00640 1 --2- 172.21.15.28:0/2435107822 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fed1c06c8f0 0x7fed1c06edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:33.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.419+0000 7fed47a00640 1 --2- 172.21.15.28:0/2435107822 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fed40145900 0x7fed401f9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:33.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.419+0000 7fed47a00640 1 --2- 172.21.15.28:0/2435107822 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed40073350 0x7fed401f8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:33.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.419+0000 7fed47a00640 1 --2- 172.21.15.28:0/2435107822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed4006f9e0 0x7fed40141fe0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:33.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.419+0000 7fed47a00640 1 -- 172.21.15.28:0/2435107822 >> 172.21.15.28:0/2435107822 conn(0x7fed400b2b60 msgr2=0x7fed4015fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:33.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.419+0000 7fed47a00640 1 -- 172.21.15.28:0/2435107822 shutdown_connections 2024-04-26T20:58:33.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.419+0000 7fed47a00640 1 -- 172.21.15.28:0/2435107822 wait complete. 2024-04-26T20:58:33.439 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_dump_tree.TestDumpTree.test_basic' 2024-04-26T20:58:33.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.643+0000 7f96d1200640 1 Processor -- start 2024-04-26T20:58:33.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.644+0000 7f96d1200640 1 -- start start 2024-04-26T20:58:33.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.645+0000 7f96d1200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f96cc15f2e0 0x7f96cc06c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:33.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.645+0000 7f96d1200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f96cc0d38f0 0x7f96cc06c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:33.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.645+0000 7f96d1200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f96cc0d2a00 0x7f96cc0b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:33.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.645+0000 7f96d1200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f96cc0d3db0 con 0x7f96cc0d38f0 2024-04-26T20:58:33.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.645+0000 7f96d1200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f96cc0d4090 con 0x7f96cc0d2a00 2024-04-26T20:58:33.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.645+0000 7f96d1200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f96cc0d4370 con 0x7f96cc15f2e0 2024-04-26T20:58:33.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.646+0000 7f96caa00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f96cc0d38f0 0x7f96cc06c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:33.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.646+0000 7f96caa00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f96cc0d38f0 0x7f96cc06c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44740/0 (socket says 172.21.15.28:44740) 2024-04-26T20:58:33.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.646+0000 7f96caa00640 1 -- 172.21.15.28:0/920560433 learned_addr learned my addr 172.21.15.28:0/920560433 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:33.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.646+0000 7f96cbe00640 1 --2- 172.21.15.28:0/920560433 >> v2:172.21.15.103:3300/0 conn(0x7f96cc0d2a00 0x7f96cc0b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:33.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.646+0000 7f96cb400640 1 --2- 172.21.15.28:0/920560433 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f96cc15f2e0 0x7f96cc06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:33.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.646+0000 7f96caa00640 1 -- 172.21.15.28:0/920560433 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f96cc15f2e0 msgr2=0x7f96cc06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:33.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.646+0000 7f96caa00640 1 --2- 172.21.15.28:0/920560433 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f96cc15f2e0 0x7f96cc06c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:33.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.646+0000 7f96caa00640 1 -- 172.21.15.28:0/920560433 >> v2:172.21.15.103:3300/0 conn(0x7f96cc0d2a00 msgr2=0x7f96cc0b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:33.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.646+0000 7f96caa00640 1 --2- 172.21.15.28:0/920560433 >> v2:172.21.15.103:3300/0 conn(0x7f96cc0d2a00 0x7f96cc0b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:33.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.646+0000 7f96caa00640 1 -- 172.21.15.28:0/920560433 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96cc0d4650 con 0x7f96cc0d38f0 2024-04-26T20:58:33.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.646+0000 7f96cbe00640 1 --2- 172.21.15.28:0/920560433 >> v2:172.21.15.103:3300/0 conn(0x7f96cc0d2a00 0x7f96cc0b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:33.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.647+0000 7f96caa00640 1 --2- 172.21.15.28:0/920560433 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96cc0d38f0 0x7f96cc06c890 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0d5ed0 tx=0x7f96b402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbab74bdfc4fc69d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:33.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.648+0000 7f96ca000640 1 -- 172.21.15.28:0/920560433 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4037070 con 0x7f96cc0d38f0 2024-04-26T20:58:33.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.648+0000 7f96ca000640 1 -- 172.21.15.28:0/920560433 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b4032440 con 0x7f96cc0d38f0 2024-04-26T20:58:33.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.648+0000 7f96ca000640 1 -- 172.21.15.28:0/920560433 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b4032810 con 0x7f96cc0d38f0 2024-04-26T20:58:33.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.648+0000 7f96d1200640 1 -- 172.21.15.28:0/920560433 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96cc0d38f0 msgr2=0x7f96cc06c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:33.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.648+0000 7f96d1200640 1 --2- 172.21.15.28:0/920560433 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96cc0d38f0 0x7f96cc06c890 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f96cc0d5ed0 tx=0x7f96b402a1e0 comp rx=0 tx=0).stop 2024-04-26T20:58:33.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.648+0000 7f96d1200640 1 -- 172.21.15.28:0/920560433 shutdown_connections 2024-04-26T20:58:33.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.648+0000 7f96d1200640 1 --2- 172.21.15.28:0/920560433 >> v2:172.21.15.103:3300/0 conn(0x7f96cc0d2a00 0x7f96cc0b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:33.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.648+0000 7f96d1200640 1 --2- 172.21.15.28:0/920560433 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96cc0d38f0 0x7f96cc06c890 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:33.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.648+0000 7f96d1200640 1 --2- 172.21.15.28:0/920560433 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f96cc15f2e0 0x7f96cc06c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:33.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.648+0000 7f96d1200640 1 -- 172.21.15.28:0/920560433 >> 172.21.15.28:0/920560433 conn(0x7f96cc0d4e30 msgr2=0x7f96cc0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:33.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.648+0000 7f96d1200640 1 -- 172.21.15.28:0/920560433 shutdown_connections 2024-04-26T20:58:33.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.648+0000 7f96d1200640 1 -- 172.21.15.28:0/920560433 wait complete. 2024-04-26T20:58:33.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.650+0000 7f96d1200640 1 Processor -- start 2024-04-26T20:58:33.651 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.650+0000 7f96d1200640 1 -- start start 2024-04-26T20:58:33.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.651+0000 7f96d1200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f96cc0d2a00 0x7f96cc1f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:33.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.651+0000 7f96d1200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96cc0d38f0 0x7f96cc1e76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:33.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.651+0000 7f96d1200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96cc15f2e0 0x7f96cc1f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:33.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.651+0000 7f96d1200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc06d430 con 0x7f96cc15f2e0 2024-04-26T20:58:33.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.651+0000 7f96d1200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc06d2b0 con 0x7f96cc0d38f0 2024-04-26T20:58:33.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.651+0000 7f96d1200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f96cc0b3de0 con 0x7f96cc0d2a00 2024-04-26T20:58:33.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.652+0000 7f96cbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96cc15f2e0 0x7f96cc1f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:33.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.652+0000 7f96cbe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96cc15f2e0 0x7f96cc1f84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44748/0 (socket says 172.21.15.28:44748) 2024-04-26T20:58:33.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.652+0000 7f96cbe00640 1 -- 172.21.15.28:0/2813522124 learned_addr learned my addr 172.21.15.28:0/2813522124 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:33.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.652+0000 7f96caa00640 1 --2- 172.21.15.28:0/2813522124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96cc0d38f0 0x7f96cc1e76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:33.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.652+0000 7f96cb400640 1 --2- 172.21.15.28:0/2813522124 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f96cc0d2a00 0x7f96cc1f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:33.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.652+0000 7f96cbe00640 1 -- 172.21.15.28:0/2813522124 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f96cc0d2a00 msgr2=0x7f96cc1f8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:33.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.652+0000 7f96cbe00640 1 --2- 172.21.15.28:0/2813522124 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f96cc0d2a00 0x7f96cc1f8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:33.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.652+0000 7f96cbe00640 1 -- 172.21.15.28:0/2813522124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96cc0d38f0 msgr2=0x7f96cc1e76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:33.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.652+0000 7f96cbe00640 1 --2- 172.21.15.28:0/2813522124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96cc0d38f0 0x7f96cc1e76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:33.654 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.652+0000 7f96cbe00640 1 -- 172.21.15.28:0/2813522124 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96cc1e82b0 con 0x7f96cc15f2e0 2024-04-26T20:58:33.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.652+0000 7f96cbe00640 1 --2- 172.21.15.28:0/2813522124 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96cc15f2e0 0x7f96cc1f84c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f96c400cef0 tx=0x7f96c400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:33.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.653+0000 7f96c3400640 1 -- 172.21.15.28:0/2813522124 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c401a070 con 0x7f96cc15f2e0 2024-04-26T20:58:33.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.653+0000 7f96c3400640 1 -- 172.21.15.28:0/2813522124 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c400d840 con 0x7f96cc15f2e0 2024-04-26T20:58:33.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.653+0000 7f96c3400640 1 -- 172.21.15.28:0/2813522124 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c400dc30 con 0x7f96cc15f2e0 2024-04-26T20:58:33.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.653+0000 7f96d1200640 1 -- 172.21.15.28:0/2813522124 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96cc1e8590 con 0x7f96cc15f2e0 2024-04-26T20:58:33.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.653+0000 7f96d1200640 1 -- 172.21.15.28:0/2813522124 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96cc0c9d50 con 0x7f96cc15f2e0 2024-04-26T20:58:33.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.654+0000 7f96d1200640 1 -- 172.21.15.28:0/2813522124 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96cc1e8760 con 0x7f96cc15f2e0 2024-04-26T20:58:33.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.655+0000 7f96c3400640 1 -- 172.21.15.28:0/2813522124 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96c401d070 con 0x7f96cc15f2e0 2024-04-26T20:58:33.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.656+0000 7f96c3400640 1 --2- 172.21.15.28:0/2813522124 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f96a806c8f0 0x7f96a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:33.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.656+0000 7f96c3400640 1 -- 172.21.15.28:0/2813522124 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f96c408c0f0 con 0x7f96cc15f2e0 2024-04-26T20:58:33.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.656+0000 7f96cb400640 1 --2- 172.21.15.28:0/2813522124 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f96a806c8f0 0x7f96a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:33.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.657+0000 7f96cb400640 1 --2- 172.21.15.28:0/2813522124 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f96a806c8f0 0x7f96a806edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00b7c0 tx=0x7f96bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:33.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.661+0000 7f96c3400640 1 -- 172.21.15.28:0/2813522124 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f96c405a780 con 0x7f96cc15f2e0 2024-04-26T20:58:33.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:33.944+0000 7f96d1200640 1 -- 172.21.15.28:0/2813522124 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_dump_tree.TestDumpTree.test_basic"]} v 0) -- 0x7f96cc0c9860 con 0x7f96cc15f2e0 2024-04-26T20:58:34.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:34.950+0000 7f96c3400640 1 -- 172.21.15.28:0/2813522124 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_dump_tree.TestDumpTree.test_basic"]}]=0 v117) ==== 130+0+0 (secure 0 0 0) 0x7f96c405a080 con 0x7f96cc15f2e0 2024-04-26T20:58:34.955 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:34.954+0000 7f96d1200640 1 -- 172.21.15.28:0/2813522124 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f96a806c8f0 msgr2=0x7f96a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:34.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:34.954+0000 7f96d1200640 1 --2- 172.21.15.28:0/2813522124 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f96a806c8f0 0x7f96a806edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f96bc00b7c0 tx=0x7f96bc008040 comp rx=0 tx=0).stop 2024-04-26T20:58:34.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:34.954+0000 7f96d1200640 1 -- 172.21.15.28:0/2813522124 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96cc15f2e0 msgr2=0x7f96cc1f84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:34.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:34.954+0000 7f96d1200640 1 --2- 172.21.15.28:0/2813522124 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96cc15f2e0 0x7f96cc1f84c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f96c400cef0 tx=0x7f96c400a710 comp rx=0 tx=0).stop 2024-04-26T20:58:34.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:34.954+0000 7f96d1200640 1 -- 172.21.15.28:0/2813522124 shutdown_connections 2024-04-26T20:58:34.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:34.954+0000 7f96d1200640 1 --2- 172.21.15.28:0/2813522124 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f96a806c8f0 0x7f96a806edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:34.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:34.954+0000 7f96d1200640 1 --2- 172.21.15.28:0/2813522124 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96cc15f2e0 0x7f96cc1f84c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:34.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:34.954+0000 7f96d1200640 1 --2- 172.21.15.28:0/2813522124 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f96cc0d38f0 0x7f96cc1e76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:34.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:34.954+0000 7f96d1200640 1 --2- 172.21.15.28:0/2813522124 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f96cc0d2a00 0x7f96cc1f8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:34.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:34.954+0000 7f96d1200640 1 -- 172.21.15.28:0/2813522124 >> 172.21.15.28:0/2813522124 conn(0x7f96cc0d4e30 msgr2=0x7f96cc15db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:34.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:34.954+0000 7f96d1200640 1 -- 172.21.15.28:0/2813522124 shutdown_connections 2024-04-26T20:58:34.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:34.955+0000 7f96d1200640 1 -- 172.21.15.28:0/2813522124 wait complete. 2024-04-26T20:58:34.978 INFO:tasks.cephfs_test_runner:test_basic (tasks.cephfs.test_dump_tree.TestDumpTree) ... ok 2024-04-26T20:58:34.978 INFO:tasks.cephfs_test_runner:Starting test: test_random (tasks.cephfs.test_dump_tree.TestDumpTree) 2024-04-26T20:58:34.979 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:58:35.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.195+0000 7f4be0a00640 1 Processor -- start 2024-04-26T20:58:35.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.195+0000 7f4be0a00640 1 -- start start 2024-04-26T20:58:35.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.197+0000 7f4be0a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4bdc146060 0x7f4bdc071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:35.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.197+0000 7f4be0a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bdc1610c0 0x7f4bdc0cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:35.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.197+0000 7f4be0a00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bdc161510 con 0x7f4bdc146670 2024-04-26T20:58:35.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.197+0000 7f4be0a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f4bdc06f510 con 0x7f4bdc146060 2024-04-26T20:58:35.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.197+0000 7f4be0a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f4bdc146440 con 0x7f4bdc1610c0 2024-04-26T20:58:35.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.197+0000 7f4bda800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4bdc146060 0x7f4bdc071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:35.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.197+0000 7f4bda800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4bdc146060 0x7f4bdc071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44752/0 (socket says 172.21.15.28:44752) 2024-04-26T20:58:35.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.197+0000 7f4bda800640 1 -- 172.21.15.28:0/239118750 learned_addr learned my addr 172.21.15.28:0/239118750 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:35.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.198+0000 7f4bda800640 1 -- 172.21.15.28:0/239118750 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bdc1610c0 msgr2=0x7f4bdc0cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:35.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.198+0000 7f4bdb200640 1 --2- 172.21.15.28:0/239118750 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bdc1610c0 0x7f4bdc0cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:35.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.198+0000 7f4bda800640 1 --2- 172.21.15.28:0/239118750 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bdc1610c0 0x7f4bdc0cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:35.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.198+0000 7f4bda800640 1 -- 172.21.15.28:0/239118750 >> v1:172.21.15.103:6789/0 conn(0x7f4bdc146670 legacy=0x7f4bdc1609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:35.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.198+0000 7f4bda800640 1 -- 172.21.15.28:0/239118750 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bdc06f010 con 0x7f4bdc146060 2024-04-26T20:58:35.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.199+0000 7f4bda800640 1 --2- 172.21.15.28:0/239118750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bdc146060 0x7f4bdc071e20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009870 tx=0x7f4bcc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50576b5ef537ff78 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:35.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.199+0000 7f4bd9e00640 1 -- 172.21.15.28:0/239118750 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc037070 con 0x7f4bdc146060 2024-04-26T20:58:35.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.199+0000 7f4bd9e00640 1 -- 172.21.15.28:0/239118750 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bcc032440 con 0x7f4bdc146060 2024-04-26T20:58:35.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.199+0000 7f4bd9e00640 1 -- 172.21.15.28:0/239118750 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc032810 con 0x7f4bdc146060 2024-04-26T20:58:35.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.200+0000 7f4be0a00640 1 -- 172.21.15.28:0/239118750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bdc146060 msgr2=0x7f4bdc071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:35.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.200+0000 7f4be0a00640 1 --2- 172.21.15.28:0/239118750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bdc146060 0x7f4bdc071e20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009870 tx=0x7f4bcc02a180 comp rx=0 tx=0).stop 2024-04-26T20:58:35.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.200+0000 7f4be0a00640 1 -- 172.21.15.28:0/239118750 shutdown_connections 2024-04-26T20:58:35.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.200+0000 7f4be0a00640 1 --2- 172.21.15.28:0/239118750 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bdc1610c0 0x7f4bdc0cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:35.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.200+0000 7f4be0a00640 1 --2- 172.21.15.28:0/239118750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bdc146060 0x7f4bdc071e20 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:35.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.200+0000 7f4be0a00640 1 -- 172.21.15.28:0/239118750 >> 172.21.15.28:0/239118750 conn(0x7f4bdc0d0a50 msgr2=0x7f4bdc0d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:35.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.200+0000 7f4be0a00640 1 -- 172.21.15.28:0/239118750 shutdown_connections 2024-04-26T20:58:35.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.200+0000 7f4be0a00640 1 -- 172.21.15.28:0/239118750 wait complete. 2024-04-26T20:58:35.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.202+0000 7f4be0a00640 1 Processor -- start 2024-04-26T20:58:35.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.202+0000 7f4be0a00640 1 -- start start 2024-04-26T20:58:35.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.203+0000 7f4be0a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bdc146060 0x7f4bdc13e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:35.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.203+0000 7f4be0a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bdc146670 0x7f4bdc1436b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:35.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.203+0000 7f4be0a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bdc1610c0 0x7f4bdc15d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:35.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.203+0000 7f4be0a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4bdc06ff60 con 0x7f4bdc1610c0 2024-04-26T20:58:35.204 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.203+0000 7f4be0a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4bdc06fc80 con 0x7f4bdc146670 2024-04-26T20:58:35.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.203+0000 7f4be0a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f4bdc0cb5e0 con 0x7f4bdc146060 2024-04-26T20:58:35.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.203+0000 7f4bdb200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bdc1610c0 0x7f4bdc15d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:35.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.203+0000 7f4bdb200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bdc1610c0 0x7f4bdc15d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44754/0 (socket says 172.21.15.28:44754) 2024-04-26T20:58:35.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.203+0000 7f4bdb200640 1 -- 172.21.15.28:0/2905527108 learned_addr learned my addr 172.21.15.28:0/2905527108 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:35.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.204+0000 7f4bdb200640 1 -- 172.21.15.28:0/2905527108 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bdc146060 msgr2=0x7f4bdc13e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:35.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.204+0000 7f4bdb200640 1 --2- 172.21.15.28:0/2905527108 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bdc146060 0x7f4bdc13e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:35.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.204+0000 7f4bdb200640 1 -- 172.21.15.28:0/2905527108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bdc146670 msgr2=0x7f4bdc1436b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:35.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.204+0000 7f4bdb200640 1 --2- 172.21.15.28:0/2905527108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bdc146670 0x7f4bdc1436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:35.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.204+0000 7f4bdb200640 1 -- 172.21.15.28:0/2905527108 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bdc1412a0 con 0x7f4bdc1610c0 2024-04-26T20:58:35.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.204+0000 7f4bdb200640 1 --2- 172.21.15.28:0/2905527108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bdc1610c0 0x7f4bdc15d5b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400cc30 tx=0x7f4bd400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:35.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.204+0000 7f4bd2a00640 1 -- 172.21.15.28:0/2905527108 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd400ce40 con 0x7f4bdc1610c0 2024-04-26T20:58:35.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.204+0000 7f4bd2a00640 1 -- 172.21.15.28:0/2905527108 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd4003990 con 0x7f4bdc1610c0 2024-04-26T20:58:35.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.204+0000 7f4bd2a00640 1 -- 172.21.15.28:0/2905527108 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd4003d40 con 0x7f4bdc1610c0 2024-04-26T20:58:35.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.204+0000 7f4be0a00640 1 -- 172.21.15.28:0/2905527108 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bdc0b4900 con 0x7f4bdc1610c0 2024-04-26T20:58:35.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.205+0000 7f4be0a00640 1 -- 172.21.15.28:0/2905527108 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bdc0b4e80 con 0x7f4bdc1610c0 2024-04-26T20:58:35.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.206+0000 7f4be0a00640 1 -- 172.21.15.28:0/2905527108 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bdc06a360 con 0x7f4bdc1610c0 2024-04-26T20:58:35.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.207+0000 7f4bd2a00640 1 -- 172.21.15.28:0/2905527108 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4bd4003ee0 con 0x7f4bdc1610c0 2024-04-26T20:58:35.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.208+0000 7f4bd2a00640 1 --2- 172.21.15.28:0/2905527108 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4bb806c8f0 0x7f4bb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:35.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.208+0000 7f4bd2a00640 1 -- 172.21.15.28:0/2905527108 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f4bd408bcd0 con 0x7f4bdc1610c0 2024-04-26T20:58:35.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.208+0000 7f4bda800640 1 --2- 172.21.15.28:0/2905527108 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4bb806c8f0 0x7f4bb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:35.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.209+0000 7f4bda800640 1 --2- 172.21.15.28:0/2905527108 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4bb806c8f0 0x7f4bb806edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc034000 tx=0x7f4bcc02c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:35.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.213+0000 7f4bd2a00640 1 -- 172.21.15.28:0/2905527108 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4bd4014030 con 0x7f4bdc1610c0 2024-04-26T20:58:35.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.423+0000 7f4be0a00640 1 -- 172.21.15.28:0/2905527108 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4bdc0b0ac0 con 0x7f4bdc1610c0 2024-04-26T20:58:35.426 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.424+0000 7f4bd2a00640 1 -- 172.21.15.28:0/2905527108 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+12998 (secure 0 0 0) 0x7f4bd405a490 con 0x7f4bdc1610c0 2024-04-26T20:58:35.426 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:35.426 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":37,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:58:32.933387+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.103:6835/1781724684":"2024-04-27T20:58:29.508394+0000","172.21.15.103:6833/1781724684":"2024-04-27T20:58:29.508394+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:58:35.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.429+0000 7f4be0a00640 1 -- 172.21.15.28:0/2905527108 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4bb806c8f0 msgr2=0x7f4bb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:35.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.429+0000 7f4be0a00640 1 --2- 172.21.15.28:0/2905527108 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4bb806c8f0 0x7f4bb806edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc034000 tx=0x7f4bcc02c000 comp rx=0 tx=0).stop 2024-04-26T20:58:35.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.429+0000 7f4be0a00640 1 -- 172.21.15.28:0/2905527108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bdc1610c0 msgr2=0x7f4bdc15d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:35.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.429+0000 7f4be0a00640 1 --2- 172.21.15.28:0/2905527108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bdc1610c0 0x7f4bdc15d5b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4bd400cc30 tx=0x7f4bd400a5c0 comp rx=0 tx=0).stop 2024-04-26T20:58:35.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.429+0000 7f4be0a00640 1 -- 172.21.15.28:0/2905527108 shutdown_connections 2024-04-26T20:58:35.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.429+0000 7f4be0a00640 1 --2- 172.21.15.28:0/2905527108 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4bb806c8f0 0x7f4bb806edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:35.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.429+0000 7f4be0a00640 1 --2- 172.21.15.28:0/2905527108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bdc1610c0 0x7f4bdc15d5b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:35.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.429+0000 7f4be0a00640 1 --2- 172.21.15.28:0/2905527108 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bdc146670 0x7f4bdc1436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:35.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.429+0000 7f4be0a00640 1 --2- 172.21.15.28:0/2905527108 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bdc146060 0x7f4bdc13e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:35.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.429+0000 7f4be0a00640 1 -- 172.21.15.28:0/2905527108 >> 172.21.15.28:0/2905527108 conn(0x7f4bdc0d0a50 msgr2=0x7f4bdc0d0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:35.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.429+0000 7f4be0a00640 1 -- 172.21.15.28:0/2905527108 shutdown_connections 2024-04-26T20:58:35.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.429+0000 7f4be0a00640 1 -- 172.21.15.28:0/2905527108 wait complete. 2024-04-26T20:58:35.450 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:58:35.450 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:58:35.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.666+0000 7fedbc800640 1 Processor -- start 2024-04-26T20:58:35.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.666+0000 7fedbc800640 1 -- start start 2024-04-26T20:58:35.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.668+0000 7fedbc800640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fedb40b4f90 0x7fedb40735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:35.669 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.668+0000 7fedbc800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedb406f2d0 con 0x7fedb406ed20 2024-04-26T20:58:35.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.668+0000 7fedbc800640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedb406f5b0 con 0x7fedb40731f0 2024-04-26T20:58:35.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.668+0000 7fedbc800640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fedb406f890 con 0x7fedb40b4f90 2024-04-26T20:58:35.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.668+0000 7fedba800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fedb406ed20 0x7fedb4168ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:44602/0 (socket says 172.21.15.28:44602) 2024-04-26T20:58:35.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.668+0000 7fedba800640 1 -- 172.21.15.28:0/4026967567 learned_addr learned my addr 172.21.15.28:0/4026967567 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:35.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.669+0000 7fedb9400640 1 --2- 172.21.15.28:0/4026967567 >> v2:172.21.15.103:3300/0 conn(0x7fedb40b4f90 0x7fedb40735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:35.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.669+0000 7fedb8a00640 1 -- 172.21.15.28:0/4026967567 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2404173306 0 0) 0x7fedb406f2d0 con 0x7fedb406ed20 2024-04-26T20:58:35.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.669+0000 7fedb8a00640 1 -- 172.21.15.28:0/4026967567 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fed9c003520 con 0x7fedb406ed20 2024-04-26T20:58:35.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.669+0000 7fedb8a00640 1 -- 172.21.15.28:0/4026967567 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3740193740 0 0) 0x7fed9c003520 con 0x7fedb406ed20 2024-04-26T20:58:35.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.669+0000 7fedb8a00640 1 -- 172.21.15.28:0/4026967567 >> v2:172.21.15.103:3300/0 conn(0x7fedb40b4f90 msgr2=0x7fedb40735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:35.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.669+0000 7fedb8a00640 1 --2- 172.21.15.28:0/4026967567 >> v2:172.21.15.103:3300/0 conn(0x7fedb40b4f90 0x7fedb40735d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:35.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.669+0000 7fedb8a00640 1 -- 172.21.15.28:0/4026967567 >> v1:172.21.15.103:6789/0 conn(0x7fedb40731f0 legacy=0x7fedb406e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:35.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.669+0000 7fedb8a00640 1 -- 172.21.15.28:0/4026967567 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb406fb70 con 0x7fedb406ed20 2024-04-26T20:58:35.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.669+0000 7fedb8a00640 1 -- 172.21.15.28:0/4026967567 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fedb00019f0 con 0x7fedb406ed20 2024-04-26T20:58:35.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.669+0000 7fedb8a00640 1 -- 172.21.15.28:0/4026967567 >> v1:172.21.15.28:6789/0 conn(0x7fedb406ed20 legacy=0x7fedb4168ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:35.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.669+0000 7fedb8a00640 1 --2- 172.21.15.28:0/4026967567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed9c0039b0 0x7fed9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:35.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.669+0000 7fedb8a00640 1 --2- 172.21.15.28:0/4026967567 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fed9c004800 0x7fed9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:35.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.670+0000 7fedb8a00640 1 --2- 172.21.15.28:0/4026967567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c0051c0 0x7fed9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:35.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.670+0000 7fedb8a00640 1 -- 172.21.15.28:0/4026967567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fedb406f890 con 0x7fed9c0039b0 2024-04-26T20:58:35.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.670+0000 7fedb8a00640 1 -- 172.21.15.28:0/4026967567 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c00c060 con 0x7fed9c0051c0 2024-04-26T20:58:35.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.670+0000 7fedb8a00640 1 -- 172.21.15.28:0/4026967567 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fed9c00c260 con 0x7fed9c004800 2024-04-26T20:58:35.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.670+0000 7fedb9e00640 1 --2- 172.21.15.28:0/4026967567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed9c0039b0 0x7fed9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:35.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.670+0000 7fedb9e00640 1 -- 172.21.15.28:0/4026967567 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fed9c004800 msgr2=0x7fed9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:35.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.670+0000 7fedb9e00640 1 --2- 172.21.15.28:0/4026967567 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fed9c004800 0x7fed9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:35.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.670+0000 7fedb9e00640 1 -- 172.21.15.28:0/4026967567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c0051c0 msgr2=0x7fed9c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:35.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.670+0000 7fedb9e00640 1 --2- 172.21.15.28:0/4026967567 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c0051c0 0x7fed9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:35.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.670+0000 7fedb9e00640 1 -- 172.21.15.28:0/4026967567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9c003520 con 0x7fed9c0039b0 2024-04-26T20:58:35.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.670+0000 7fedba800640 1 -- 172.21.15.28:0/4026967567 reap_dead start 2024-04-26T20:58:35.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.671+0000 7fedb9e00640 1 --2- 172.21.15.28:0/4026967567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed9c0039b0 0x7fed9c003dd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7feda8001750 tx=0x7fed9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71f078c2fd891ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:35.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.672+0000 7fedb8a00640 1 -- 172.21.15.28:0/4026967567 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8001370 con 0x7fed9c0039b0 2024-04-26T20:58:35.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.672+0000 7fedb8a00640 1 -- 172.21.15.28:0/4026967567 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda80052d0 con 0x7fed9c0039b0 2024-04-26T20:58:35.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.672+0000 7fedb8a00640 1 -- 172.21.15.28:0/4026967567 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda802abc0 con 0x7fed9c0039b0 2024-04-26T20:58:35.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.672+0000 7fedbc800640 1 -- 172.21.15.28:0/4026967567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed9c0039b0 msgr2=0x7fed9c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:35.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.672+0000 7fedbc800640 1 --2- 172.21.15.28:0/4026967567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed9c0039b0 0x7fed9c003dd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7feda8001750 tx=0x7fed9c005830 comp rx=0 tx=0).stop 2024-04-26T20:58:35.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.672+0000 7fedbc800640 1 -- 172.21.15.28:0/4026967567 shutdown_connections 2024-04-26T20:58:35.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.672+0000 7fedbc800640 1 --2- 172.21.15.28:0/4026967567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed9c0039b0 0x7fed9c003dd0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:35.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.672+0000 7fedbc800640 1 -- 172.21.15.28:0/4026967567 >> 172.21.15.28:0/4026967567 conn(0x7fedb406c770 msgr2=0x7fedb406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:35.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.672+0000 7fedbc800640 1 -- 172.21.15.28:0/4026967567 shutdown_connections 2024-04-26T20:58:35.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.672+0000 7fedbc800640 1 -- 172.21.15.28:0/4026967567 wait complete. 2024-04-26T20:58:35.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.674+0000 7fedbc800640 1 Processor -- start 2024-04-26T20:58:35.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.674+0000 7fedbc800640 1 -- start start 2024-04-26T20:58:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.675+0000 7fedbc800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c0039b0 0x7fedb413f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.675+0000 7fedbc800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedb413da80 0x7fedb4153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.675+0000 7fedbc800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fedb4142ea0 0x7fedb4143350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.676+0000 7fedbc800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fedb406e790 con 0x7fedb413da80 2024-04-26T20:58:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.676+0000 7fedbc800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fedb406e610 con 0x7fed9c0039b0 2024-04-26T20:58:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.676+0000 7fedbc800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fedb406e910 con 0x7fedb4142ea0 2024-04-26T20:58:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.676+0000 7fedb9400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedb413da80 0x7fedb4153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.676+0000 7fedb9400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedb413da80 0x7fedb4153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44772/0 (socket says 172.21.15.28:44772) 2024-04-26T20:58:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.676+0000 7fedb9400640 1 -- 172.21.15.28:0/1380288250 learned_addr learned my addr 172.21.15.28:0/1380288250 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.676+0000 7fedb9e00640 1 --2- 172.21.15.28:0/1380288250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c0039b0 0x7fedb413f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:35.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.676+0000 7fedba800640 1 --2- 172.21.15.28:0/1380288250 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fedb4142ea0 0x7fedb4143350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:35.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.676+0000 7fedb9400640 1 -- 172.21.15.28:0/1380288250 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fedb4142ea0 msgr2=0x7fedb4143350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:35.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.676+0000 7fedb9400640 1 --2- 172.21.15.28:0/1380288250 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fedb4142ea0 0x7fedb4143350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:35.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.676+0000 7fedb9400640 1 -- 172.21.15.28:0/1380288250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c0039b0 msgr2=0x7fedb413f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:35.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.676+0000 7fedb9400640 1 --2- 172.21.15.28:0/1380288250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c0039b0 0x7fedb413f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:35.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.676+0000 7fedb9400640 1 -- 172.21.15.28:0/1380288250 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb413e220 con 0x7fedb413da80 2024-04-26T20:58:35.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.677+0000 7fedb9400640 1 --2- 172.21.15.28:0/1380288250 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedb413da80 0x7fedb4153f90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7feda400e7c0 tx=0x7feda400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:35.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.677+0000 7fedae000640 1 -- 172.21.15.28:0/1380288250 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda4017070 con 0x7fedb413da80 2024-04-26T20:58:35.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.677+0000 7fedae000640 1 -- 172.21.15.28:0/1380288250 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda400c940 con 0x7fedb413da80 2024-04-26T20:58:35.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.677+0000 7fedae000640 1 -- 172.21.15.28:0/1380288250 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda400cd30 con 0x7fedb413da80 2024-04-26T20:58:35.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.677+0000 7fedbc800640 1 -- 172.21.15.28:0/1380288250 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb413e500 con 0x7fedb413da80 2024-04-26T20:58:35.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.677+0000 7fedbc800640 1 -- 172.21.15.28:0/1380288250 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb4140840 con 0x7fedb413da80 2024-04-26T20:58:35.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.678+0000 7fedbc800640 1 -- 172.21.15.28:0/1380288250 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedb406a360 con 0x7fedb413da80 2024-04-26T20:58:35.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.680+0000 7fedae000640 1 -- 172.21.15.28:0/1380288250 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feda401a070 con 0x7fedb413da80 2024-04-26T20:58:35.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.681+0000 7fedae000640 1 --2- 172.21.15.28:0/1380288250 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fed9806c940 0x7fed9806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:35.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.681+0000 7fedae000640 1 -- 172.21.15.28:0/1380288250 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7feda408b910 con 0x7fedb413da80 2024-04-26T20:58:35.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.681+0000 7fedb9e00640 1 --2- 172.21.15.28:0/1380288250 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fed9806c940 0x7fed9806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:35.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.682+0000 7fedb9e00640 1 --2- 172.21.15.28:0/1380288250 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fed9806c940 0x7fed9806ee00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feda8032000 tx=0x7feda802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:35.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.686+0000 7fedae000640 1 -- 172.21.15.28:0/1380288250 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feda405a1b0 con 0x7fedb413da80 2024-04-26T20:58:35.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.888+0000 7fedbc800640 1 -- 172.21.15.28:0/1380288250 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fedb4162570 con 0x7fedb413da80 2024-04-26T20:58:35.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.889+0000 7fedae000640 1 -- 172.21.15.28:0/1380288250 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) ==== 93+0+10 (secure 0 0 0) 0x7feda4059ab0 con 0x7fedb413da80 2024-04-26T20:58:35.890 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:58:35.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.892+0000 7fedbc800640 1 -- 172.21.15.28:0/1380288250 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fed9806c940 msgr2=0x7fed9806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:35.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.892+0000 7fedbc800640 1 --2- 172.21.15.28:0/1380288250 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fed9806c940 0x7fed9806ee00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7feda8032000 tx=0x7feda802e000 comp rx=0 tx=0).stop 2024-04-26T20:58:35.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.892+0000 7fedbc800640 1 -- 172.21.15.28:0/1380288250 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedb413da80 msgr2=0x7fedb4153f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:35.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.892+0000 7fedbc800640 1 --2- 172.21.15.28:0/1380288250 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedb413da80 0x7fedb4153f90 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7feda400e7c0 tx=0x7feda400ec80 comp rx=0 tx=0).stop 2024-04-26T20:58:35.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.892+0000 7fedbc800640 1 -- 172.21.15.28:0/1380288250 shutdown_connections 2024-04-26T20:58:35.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.892+0000 7fedbc800640 1 --2- 172.21.15.28:0/1380288250 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fed9806c940 0x7fed9806ee00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:35.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.892+0000 7fedbc800640 1 --2- 172.21.15.28:0/1380288250 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fedb4142ea0 0x7fedb4143350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:35.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.892+0000 7fedbc800640 1 --2- 172.21.15.28:0/1380288250 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fedb413da80 0x7fedb4153f90 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:35.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.892+0000 7fedbc800640 1 --2- 172.21.15.28:0/1380288250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c0039b0 0x7fedb413f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:35.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.892+0000 7fedbc800640 1 -- 172.21.15.28:0/1380288250 >> 172.21.15.28:0/1380288250 conn(0x7fedb406c770 msgr2=0x7fedb415f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:35.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.892+0000 7fedbc800640 1 -- 172.21.15.28:0/1380288250 shutdown_connections 2024-04-26T20:58:35.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:35.892+0000 7fedbc800640 1 -- 172.21.15.28:0/1380288250 wait complete. 2024-04-26T20:58:35.914 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_dump_tree.TestDumpTree.test_random' 2024-04-26T20:58:36.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.113+0000 7f4f6be00640 1 Processor -- start 2024-04-26T20:58:36.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.113+0000 7f4f6be00640 1 -- start start 2024-04-26T20:58:36.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.115+0000 7f4f6be00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4f6c15aab0 0x7f4f6c0cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:36.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.115+0000 7f4f6be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4f6c1611f0 0x7f4f6c0724d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:36.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.115+0000 7f4f6be00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f6c15ae90 con 0x7f4f6c073d10 2024-04-26T20:58:36.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.115+0000 7f4f6be00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f4f6c0cfa40 con 0x7f4f6c1611f0 2024-04-26T20:58:36.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.115+0000 7f4f6be00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f4f6c072040 con 0x7f4f6c15aab0 2024-04-26T20:58:36.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.115+0000 7f4f6b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4f6c1611f0 0x7f4f6c0724d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:36.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.115+0000 7f4f6a000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f4f6c073d10 0x7f4f6c160ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:44608/0 (socket says 172.21.15.28:44608) 2024-04-26T20:58:36.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.115+0000 7f4f6a000640 1 -- 172.21.15.28:0/3063923349 learned_addr learned my addr 172.21.15.28:0/3063923349 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:36.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.116+0000 7f4f6aa00640 1 --2- 172.21.15.28:0/3063923349 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4f6c15aab0 0x7f4f6c0cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:36.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.116+0000 7f4f6b400640 1 -- 172.21.15.28:0/3063923349 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4f6c15aab0 msgr2=0x7f4f6c0cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:36.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.116+0000 7f4f6b400640 1 --2- 172.21.15.28:0/3063923349 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4f6c15aab0 0x7f4f6c0cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:36.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.116+0000 7f4f6b400640 1 -- 172.21.15.28:0/3063923349 >> v1:172.21.15.28:6789/0 conn(0x7f4f6c073d10 legacy=0x7f4f6c160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:36.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.116+0000 7f4f6b400640 1 -- 172.21.15.28:0/3063923349 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f6c0cfc90 con 0x7f4f6c1611f0 2024-04-26T20:58:36.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.116+0000 7f4f69600640 1 -- 172.21.15.28:0/3063923349 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1249093626 0 0) 0x7f4f6c15ae90 con 0x7f4f6c073d10 2024-04-26T20:58:36.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.116+0000 7f4f6aa00640 1 --2- 172.21.15.28:0/3063923349 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4f6c15aab0 0x7f4f6c0cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:36.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.117+0000 7f4f6b400640 1 --2- 172.21.15.28:0/3063923349 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f6c1611f0 0x7f4f6c0724d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4f64009d90 tx=0x7f4f6402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828f8950b0c4343c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:36.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.117+0000 7f4f69600640 1 -- 172.21.15.28:0/3063923349 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64031600 con 0x7f4f6c1611f0 2024-04-26T20:58:36.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.117+0000 7f4f69600640 1 -- 172.21.15.28:0/3063923349 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f640317a0 con 0x7f4f6c1611f0 2024-04-26T20:58:36.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.117+0000 7f4f69600640 1 -- 172.21.15.28:0/3063923349 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64031b70 con 0x7f4f6c1611f0 2024-04-26T20:58:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.117+0000 7f4f6be00640 1 -- 172.21.15.28:0/3063923349 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f6c1611f0 msgr2=0x7f4f6c0724d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.117+0000 7f4f6be00640 1 --2- 172.21.15.28:0/3063923349 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f6c1611f0 0x7f4f6c0724d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4f64009d90 tx=0x7f4f6402a5d0 comp rx=0 tx=0).stop 2024-04-26T20:58:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.118+0000 7f4f6be00640 1 -- 172.21.15.28:0/3063923349 shutdown_connections 2024-04-26T20:58:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.118+0000 7f4f6be00640 1 --2- 172.21.15.28:0/3063923349 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f6c1611f0 0x7f4f6c0724d0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.118+0000 7f4f6be00640 1 --2- 172.21.15.28:0/3063923349 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4f6c15aab0 0x7f4f6c0cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.118+0000 7f4f6be00640 1 -- 172.21.15.28:0/3063923349 >> 172.21.15.28:0/3063923349 conn(0x7f4f6c0b2720 msgr2=0x7f4f6c0b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.118+0000 7f4f6be00640 1 -- 172.21.15.28:0/3063923349 shutdown_connections 2024-04-26T20:58:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.118+0000 7f4f6be00640 1 -- 172.21.15.28:0/3063923349 wait complete. 2024-04-26T20:58:36.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.119+0000 7f4f6be00640 1 Processor -- start 2024-04-26T20:58:36.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.119+0000 7f4f6be00640 1 -- start start 2024-04-26T20:58:36.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.120+0000 7f4f6be00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4f6c073d10 0x7f4f6c0b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:36.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.120+0000 7f4f6be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f6c15aab0 0x7f4f6c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:36.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.120+0000 7f4f6be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f6c1611f0 0x7f4f6c0aef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:36.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.120+0000 7f4f6be00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6c074150 con 0x7f4f6c1611f0 2024-04-26T20:58:36.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.120+0000 7f4f6be00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4f6c06df00 con 0x7f4f6c15aab0 2024-04-26T20:58:36.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.120+0000 7f4f6be00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f4f6c0d0250 con 0x7f4f6c073d10 2024-04-26T20:58:36.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.121+0000 7f4f6b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f6c1611f0 0x7f4f6c0aef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:36.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.121+0000 7f4f6b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f6c1611f0 0x7f4f6c0aef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44792/0 (socket says 172.21.15.28:44792) 2024-04-26T20:58:36.122 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.121+0000 7f4f6b400640 1 -- 172.21.15.28:0/1553824099 learned_addr learned my addr 172.21.15.28:0/1553824099 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:36.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.121+0000 7f4f6aa00640 1 --2- 172.21.15.28:0/1553824099 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4f6c073d10 0x7f4f6c0b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:36.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.121+0000 7f4f6b400640 1 -- 172.21.15.28:0/1553824099 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4f6c073d10 msgr2=0x7f4f6c0b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:36.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.121+0000 7f4f6b400640 1 --2- 172.21.15.28:0/1553824099 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4f6c073d10 0x7f4f6c0b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:36.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.121+0000 7f4f6b400640 1 -- 172.21.15.28:0/1553824099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f6c15aab0 msgr2=0x7f4f6c06e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:36.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.121+0000 7f4f6a000640 1 --2- 172.21.15.28:0/1553824099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f6c15aab0 0x7f4f6c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:36.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.121+0000 7f4f6b400640 1 --2- 172.21.15.28:0/1553824099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f6c15aab0 0x7f4f6c06e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:36.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.121+0000 7f4f6b400640 1 -- 172.21.15.28:0/1553824099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f6c0afb50 con 0x7f4f6c1611f0 2024-04-26T20:58:36.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.121+0000 7f4f6aa00640 1 --2- 172.21.15.28:0/1553824099 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4f6c073d10 0x7f4f6c0b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:36.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.121+0000 7f4f6b400640 1 --2- 172.21.15.28:0/1553824099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f6c1611f0 0x7f4f6c0aef00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4f64009eb0 tx=0x7f4f64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:36.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.122+0000 7f4f62a00640 1 -- 172.21.15.28:0/1553824099 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6403f070 con 0x7f4f6c1611f0 2024-04-26T20:58:36.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.122+0000 7f4f62a00640 1 -- 172.21.15.28:0/1553824099 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f64030b60 con 0x7f4f6c1611f0 2024-04-26T20:58:36.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.122+0000 7f4f62a00640 1 -- 172.21.15.28:0/1553824099 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f64044460 con 0x7f4f6c1611f0 2024-04-26T20:58:36.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.122+0000 7f4f6be00640 1 -- 172.21.15.28:0/1553824099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f6c0af870 con 0x7f4f6c1611f0 2024-04-26T20:58:36.124 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.122+0000 7f4f6be00640 1 -- 172.21.15.28:0/1553824099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f6c0b3b90 con 0x7f4f6c1611f0 2024-04-26T20:58:36.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.123+0000 7f4f6be00640 1 -- 172.21.15.28:0/1553824099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f6c0b4080 con 0x7f4f6c1611f0 2024-04-26T20:58:36.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.124+0000 7f4f62a00640 1 -- 172.21.15.28:0/1553824099 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f6403b410 con 0x7f4f6c1611f0 2024-04-26T20:58:36.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.125+0000 7f4f62a00640 1 --2- 172.21.15.28:0/1553824099 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4f4806c8f0 0x7f4f4806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:36.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.125+0000 7f4f62a00640 1 -- 172.21.15.28:0/1553824099 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f4f640acfc0 con 0x7f4f6c1611f0 2024-04-26T20:58:36.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.125+0000 7f4f6aa00640 1 --2- 172.21.15.28:0/1553824099 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4f4806c8f0 0x7f4f4806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.126+0000 7f4f6aa00640 1 --2- 172.21.15.28:0/1553824099 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4f4806c8f0 0x7f4f4806edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c006fd0 tx=0x7f4f5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:36.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.129+0000 7f4f62a00640 1 -- 172.21.15.28:0/1553824099 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4f6407b710 con 0x7f4f6c1611f0 2024-04-26T20:58:36.408 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.407+0000 7f4f6be00640 1 -- 172.21.15.28:0/1553824099 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_dump_tree.TestDumpTree.test_random"]} v 0) -- 0x7f4f6c06a360 con 0x7f4f6c1611f0 2024-04-26T20:58:36.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.962+0000 7f4f62a00640 1 -- 172.21.15.28:0/1553824099 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_dump_tree.TestDumpTree.test_random"]}]=0 v119) ==== 134+0+0 (secure 0 0 0) 0x7f4f6407b010 con 0x7f4f6c1611f0 2024-04-26T20:58:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.966+0000 7f4f6be00640 1 -- 172.21.15.28:0/1553824099 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4f4806c8f0 msgr2=0x7f4f4806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.966+0000 7f4f6be00640 1 --2- 172.21.15.28:0/1553824099 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4f4806c8f0 0x7f4f4806edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c006fd0 tx=0x7f4f5c008040 comp rx=0 tx=0).stop 2024-04-26T20:58:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.966+0000 7f4f6be00640 1 -- 172.21.15.28:0/1553824099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f6c1611f0 msgr2=0x7f4f6c0aef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.966+0000 7f4f6be00640 1 --2- 172.21.15.28:0/1553824099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f6c1611f0 0x7f4f6c0aef00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f4f64009eb0 tx=0x7f4f64004a40 comp rx=0 tx=0).stop 2024-04-26T20:58:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.966+0000 7f4f6be00640 1 -- 172.21.15.28:0/1553824099 shutdown_connections 2024-04-26T20:58:36.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.966+0000 7f4f6be00640 1 --2- 172.21.15.28:0/1553824099 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4f4806c8f0 0x7f4f4806edb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.966+0000 7f4f6be00640 1 --2- 172.21.15.28:0/1553824099 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4f6c1611f0 0x7f4f6c0aef00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.966+0000 7f4f6be00640 1 --2- 172.21.15.28:0/1553824099 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f6c15aab0 0x7f4f6c06e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.966+0000 7f4f6be00640 1 --2- 172.21.15.28:0/1553824099 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4f6c073d10 0x7f4f6c0b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.966+0000 7f4f6be00640 1 -- 172.21.15.28:0/1553824099 >> 172.21.15.28:0/1553824099 conn(0x7f4f6c0b2720 msgr2=0x7f4f6c06ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.966+0000 7f4f6be00640 1 -- 172.21.15.28:0/1553824099 shutdown_connections 2024-04-26T20:58:36.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:36.966+0000 7f4f6be00640 1 -- 172.21.15.28:0/1553824099 wait complete. 2024-04-26T20:58:36.991 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T20:58:37.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.205+0000 7faa45e00640 1 Processor -- start 2024-04-26T20:58:37.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.205+0000 7faa45e00640 1 -- start start 2024-04-26T20:58:37.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.206+0000 7faa45e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7faa4015efd0 0x7faa400d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:37.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.206+0000 7faa45e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa400af6e0 con 0x7faa400b5c50 2024-04-26T20:58:37.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.207+0000 7faa45e00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa400b3ea0 con 0x7faa400b4f40 2024-04-26T20:58:37.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.207+0000 7faa45e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7faa400b4180 con 0x7faa4015efd0 2024-04-26T20:58:37.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.207+0000 7faa45400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7faa400b5c50 0x7faa400aefd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:44616/0 (socket says 172.21.15.28:44616) 2024-04-26T20:58:37.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.207+0000 7faa45400640 1 -- 172.21.15.28:0/2808004733 learned_addr learned my addr 172.21.15.28:0/2808004733 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:37.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.207+0000 7faa3f400640 1 -- 172.21.15.28:0/2808004733 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3394314185 0 0) 0x7faa400af6e0 con 0x7faa400b5c50 2024-04-26T20:58:37.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.207+0000 7faa3f400640 1 -- 172.21.15.28:0/2808004733 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa28003520 con 0x7faa400b5c50 2024-04-26T20:58:37.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.207+0000 7faa44a00640 1 --2- 172.21.15.28:0/2808004733 >> v2:172.21.15.103:3300/0 conn(0x7faa4015efd0 0x7faa400d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:37.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.208+0000 7faa3f400640 1 -- 172.21.15.28:0/2808004733 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2696656192 0 0) 0x7faa28003520 con 0x7faa400b5c50 2024-04-26T20:58:37.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.208+0000 7faa3f400640 1 -- 172.21.15.28:0/2808004733 >> v2:172.21.15.103:3300/0 conn(0x7faa4015efd0 msgr2=0x7faa400d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:37.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.208+0000 7faa3f400640 1 --2- 172.21.15.28:0/2808004733 >> v2:172.21.15.103:3300/0 conn(0x7faa4015efd0 0x7faa400d0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:37.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.208+0000 7faa3f400640 1 -- 172.21.15.28:0/2808004733 >> v1:172.21.15.103:6789/0 conn(0x7faa400b4f40 legacy=0x7faa400ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:37.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.208+0000 7faa3f400640 1 -- 172.21.15.28:0/2808004733 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa400b4460 con 0x7faa400b5c50 2024-04-26T20:58:37.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.208+0000 7faa3f400640 1 -- 172.21.15.28:0/2808004733 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7faa380019f0 con 0x7faa400b5c50 2024-04-26T20:58:37.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.208+0000 7faa3f400640 1 -- 172.21.15.28:0/2808004733 >> v1:172.21.15.28:6789/0 conn(0x7faa400b5c50 legacy=0x7faa400aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:37.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.208+0000 7faa3f400640 1 --2- 172.21.15.28:0/2808004733 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faa280039b0 0x7faa28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:37.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.208+0000 7faa3f400640 1 --2- 172.21.15.28:0/2808004733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa28004800 0x7faa28004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:37.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.208+0000 7faa3f400640 1 --2- 172.21.15.28:0/2808004733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa280051c0 0x7faa2800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:37.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.208+0000 7faa3f400640 1 -- 172.21.15.28:0/2808004733 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7faa400b4180 con 0x7faa28004800 2024-04-26T20:58:37.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.208+0000 7faa3f400640 1 -- 172.21.15.28:0/2808004733 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faa2800c110 con 0x7faa280051c0 2024-04-26T20:58:37.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.208+0000 7faa3f400640 1 -- 172.21.15.28:0/2808004733 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7faa2800c310 con 0x7faa280039b0 2024-04-26T20:58:37.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.208+0000 7faa44a00640 1 --2- 172.21.15.28:0/2808004733 >> v2:172.21.15.103:3300/0 conn(0x7faa4015efd0 0x7faa400d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:58:37.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.208+0000 7faa3fe00640 1 --2- 172.21.15.28:0/2808004733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa28004800 0x7faa28004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:37.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.209+0000 7faa3fe00640 1 -- 172.21.15.28:0/2808004733 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faa280039b0 msgr2=0x7faa28003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:37.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.209+0000 7faa3fe00640 1 --2- 172.21.15.28:0/2808004733 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faa280039b0 0x7faa28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:37.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.209+0000 7faa3fe00640 1 -- 172.21.15.28:0/2808004733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa280051c0 msgr2=0x7faa2800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:37.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.209+0000 7faa3fe00640 1 --2- 172.21.15.28:0/2808004733 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa280051c0 0x7faa2800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:37.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.209+0000 7faa3fe00640 1 -- 172.21.15.28:0/2808004733 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa28003520 con 0x7faa28004800 2024-04-26T20:58:37.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.209+0000 7faa45400640 1 -- 172.21.15.28:0/2808004733 reap_dead start 2024-04-26T20:58:37.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.210+0000 7faa3fe00640 1 --2- 172.21.15.28:0/2808004733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa28004800 0x7faa28004c80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7faa2c001750 tx=0x7faa28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d3b811fe65c5bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:37.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.210+0000 7faa3f400640 1 -- 172.21.15.28:0/2808004733 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c001370 con 0x7faa28004800 2024-04-26T20:58:37.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.210+0000 7faa3f400640 1 -- 172.21.15.28:0/2808004733 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa2c0052d0 con 0x7faa28004800 2024-04-26T20:58:37.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.210+0000 7faa3f400640 1 -- 172.21.15.28:0/2808004733 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c02abc0 con 0x7faa28004800 2024-04-26T20:58:37.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.211+0000 7faa45e00640 1 -- 172.21.15.28:0/2808004733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa28004800 msgr2=0x7faa28004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:37.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.211+0000 7faa45e00640 1 --2- 172.21.15.28:0/2808004733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa28004800 0x7faa28004c80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7faa2c001750 tx=0x7faa28005830 comp rx=0 tx=0).stop 2024-04-26T20:58:37.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.211+0000 7faa45e00640 1 -- 172.21.15.28:0/2808004733 shutdown_connections 2024-04-26T20:58:37.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.211+0000 7faa45e00640 1 --2- 172.21.15.28:0/2808004733 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa28004800 0x7faa28004c80 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:37.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.211+0000 7faa45e00640 1 -- 172.21.15.28:0/2808004733 >> 172.21.15.28:0/2808004733 conn(0x7faa4006c530 msgr2=0x7faa4006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:37.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.211+0000 7faa45e00640 1 -- 172.21.15.28:0/2808004733 shutdown_connections 2024-04-26T20:58:37.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.211+0000 7faa45e00640 1 -- 172.21.15.28:0/2808004733 wait complete. 2024-04-26T20:58:37.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.213+0000 7faa45e00640 1 Processor -- start 2024-04-26T20:58:37.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.213+0000 7faa45e00640 1 -- start start 2024-04-26T20:58:37.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.214+0000 7faa45e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa28004800 0x7faa40149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:37.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.214+0000 7faa45e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faa40161230 0x7faa401616b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:37.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.214+0000 7faa45e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa40156580 0x7faa40156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:37.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.214+0000 7faa45e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7faa4015f3b0 con 0x7faa28004800 2024-04-26T20:58:37.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.214+0000 7faa45e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faa400b4a20 con 0x7faa40156580 2024-04-26T20:58:37.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.214+0000 7faa45e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7faa400b56c0 con 0x7faa40161230 2024-04-26T20:58:37.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.214+0000 7faa44a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa28004800 0x7faa40149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:37.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.214+0000 7faa44a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa28004800 0x7faa40149780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44818/0 (socket says 172.21.15.28:44818) 2024-04-26T20:58:37.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.214+0000 7faa44a00640 1 -- 172.21.15.28:0/1534175194 learned_addr learned my addr 172.21.15.28:0/1534175194 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:37.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.214+0000 7faa3fe00640 1 --2- 172.21.15.28:0/1534175194 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faa40161230 0x7faa401616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:37.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.214+0000 7faa44a00640 1 -- 172.21.15.28:0/1534175194 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faa40161230 msgr2=0x7faa401616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:37.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.214+0000 7faa45400640 1 --2- 172.21.15.28:0/1534175194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa40156580 0x7faa40156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:37.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.214+0000 7faa44a00640 1 --2- 172.21.15.28:0/1534175194 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faa40161230 0x7faa401616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:37.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.214+0000 7faa44a00640 1 -- 172.21.15.28:0/1534175194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa40156580 msgr2=0x7faa40156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:37.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.214+0000 7faa44a00640 1 --2- 172.21.15.28:0/1534175194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa40156580 0x7faa40156a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:37.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.214+0000 7faa44a00640 1 -- 172.21.15.28:0/1534175194 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa400ce100 con 0x7faa28004800 2024-04-26T20:58:37.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.215+0000 7faa44a00640 1 --2- 172.21.15.28:0/1534175194 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa28004800 0x7faa40149780 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7faa3400e7c0 tx=0x7faa3400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:37.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.215+0000 7faa3cc00640 1 -- 172.21.15.28:0/1534175194 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3400f040 con 0x7faa28004800 2024-04-26T20:58:37.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.215+0000 7faa3cc00640 1 -- 172.21.15.28:0/1534175194 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa34004030 con 0x7faa28004800 2024-04-26T20:58:37.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.215+0000 7faa3cc00640 1 -- 172.21.15.28:0/1534175194 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa34004400 con 0x7faa28004800 2024-04-26T20:58:37.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.215+0000 7faa45e00640 1 -- 172.21.15.28:0/1534175194 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa40149cc0 con 0x7faa28004800 2024-04-26T20:58:37.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.215+0000 7faa45e00640 1 -- 172.21.15.28:0/1534175194 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa401620f0 con 0x7faa28004800 2024-04-26T20:58:37.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.216+0000 7faa45e00640 1 -- 172.21.15.28:0/1534175194 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa4006a250 con 0x7faa28004800 2024-04-26T20:58:37.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.217+0000 7faa3cc00640 1 -- 172.21.15.28:0/1534175194 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa3400c6d0 con 0x7faa28004800 2024-04-26T20:58:37.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.218+0000 7faa3cc00640 1 --2- 172.21.15.28:0/1534175194 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7faa2006c920 0x7faa2006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:37.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.218+0000 7faa3cc00640 1 -- 172.21.15.28:0/1534175194 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7faa3408b570 con 0x7faa28004800 2024-04-26T20:58:37.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.219+0000 7faa3fe00640 1 --2- 172.21.15.28:0/1534175194 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7faa2006c920 0x7faa2006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:37.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.219+0000 7faa3fe00640 1 --2- 172.21.15.28:0/1534175194 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7faa2006c920 0x7faa2006ede0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faa2c032000 tx=0x7faa2c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:37.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.223+0000 7faa3cc00640 1 -- 172.21.15.28:0/1534175194 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faa34017020 con 0x7faa28004800 2024-04-26T20:58:37.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.426+0000 7faa45e00640 1 -- 172.21.15.28:0/1534175194 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7faa400aff10 con 0x7faa28004800 2024-04-26T20:58:37.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.433+0000 7faa3cc00640 1 -- 172.21.15.28:0/1534175194 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7faa34059c00 con 0x7faa28004800 2024-04-26T20:58:37.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.436+0000 7faa45e00640 1 -- 172.21.15.28:0/1534175194 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7faa2006c920 msgr2=0x7faa2006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:37.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.436+0000 7faa45e00640 1 --2- 172.21.15.28:0/1534175194 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7faa2006c920 0x7faa2006ede0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faa2c032000 tx=0x7faa2c02e000 comp rx=0 tx=0).stop 2024-04-26T20:58:37.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.436+0000 7faa45e00640 1 -- 172.21.15.28:0/1534175194 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa28004800 msgr2=0x7faa40149780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:37.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.436+0000 7faa45e00640 1 --2- 172.21.15.28:0/1534175194 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa28004800 0x7faa40149780 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7faa3400e7c0 tx=0x7faa3400ec80 comp rx=0 tx=0).stop 2024-04-26T20:58:37.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.437+0000 7faa45e00640 1 -- 172.21.15.28:0/1534175194 shutdown_connections 2024-04-26T20:58:37.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.437+0000 7faa45e00640 1 --2- 172.21.15.28:0/1534175194 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7faa2006c920 0x7faa2006ede0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:37.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.437+0000 7faa45e00640 1 --2- 172.21.15.28:0/1534175194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa40156580 0x7faa40156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:37.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.437+0000 7faa45e00640 1 --2- 172.21.15.28:0/1534175194 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faa40161230 0x7faa401616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:37.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.437+0000 7faa45e00640 1 --2- 172.21.15.28:0/1534175194 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa28004800 0x7faa40149780 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:37.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.437+0000 7faa45e00640 1 -- 172.21.15.28:0/1534175194 >> 172.21.15.28:0/1534175194 conn(0x7faa4006c530 msgr2=0x7faa4006e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:37.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.437+0000 7faa45e00640 1 -- 172.21.15.28:0/1534175194 shutdown_connections 2024-04-26T20:58:37.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:37.437+0000 7faa45e00640 1 -- 172.21.15.28:0/1534175194 wait complete. 2024-04-26T20:58:37.457 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T20:58:37.457 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:58:37.564 DEBUG:teuthology.orchestra.run.smithi103:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T20:58:37.681 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T20:58:37.681 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:58:37.681 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:58:37.741 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:58:37.741 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:58:37.741 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:58:37.797 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052734,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:58:37.798 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T20:58:37.798 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:58:37.858 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T20:58:37.858 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T20:58:37.858 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T20:58:37.915 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052734,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T20:58:37.915 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T20:58:37.915 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:58:38.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.140+0000 7f4104c00640 1 Processor -- start 2024-04-26T20:58:38.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.141+0000 7f4104c00640 1 -- start start 2024-04-26T20:58:38.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.142+0000 7f4104c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f410014c300 0x7f41000ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:38.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.142+0000 7f4104c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41000cb480 0x7f41000cb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:38.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.142+0000 7f4104c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4100070190 con 0x7f410014ce60 2024-04-26T20:58:38.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.142+0000 7f4104c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f410014c6e0 con 0x7f410014c300 2024-04-26T20:58:38.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.142+0000 7f4104c00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f410006f9e0 con 0x7f41000cb480 2024-04-26T20:58:38.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.143+0000 7f40ff400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f410014c300 0x7f41000ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:38.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.143+0000 7f40ff400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f410014c300 0x7f41000ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44820/0 (socket says 172.21.15.28:44820) 2024-04-26T20:58:38.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.143+0000 7f40ff400640 1 -- 172.21.15.28:0/4040491290 learned_addr learned my addr 172.21.15.28:0/4040491290 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:38.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.143+0000 7f40f7e00640 1 --1- 172.21.15.28:0/4040491290 >> v1:172.21.15.28:6789/0 conn(0x7f410014ce60 0x7f41000ceb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:44624/0 (socket says 172.21.15.28:44624) 2024-04-26T20:58:38.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.143+0000 7f40ffe00640 1 --2- 172.21.15.28:0/4040491290 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41000cb480 0x7f41000cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:38.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.143+0000 7f40ff400640 1 -- 172.21.15.28:0/4040491290 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41000cb480 msgr2=0x7f41000cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:38.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.143+0000 7f40ff400640 1 --2- 172.21.15.28:0/4040491290 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41000cb480 0x7f41000cb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:38.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.143+0000 7f40ff400640 1 -- 172.21.15.28:0/4040491290 >> v1:172.21.15.28:6789/0 conn(0x7f410014ce60 legacy=0x7f41000ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:38.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.143+0000 7f40ff400640 1 -- 172.21.15.28:0/4040491290 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f410006fc10 con 0x7f410014c300 2024-04-26T20:58:38.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.144+0000 7f40ff400640 1 --2- 172.21.15.28:0/4040491290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f410014c300 0x7f41000ce620 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009870 tx=0x7f40f002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ea67eab4f0a79b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:38.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.145+0000 7f40fea00640 1 -- 172.21.15.28:0/4040491290 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0037070 con 0x7f410014c300 2024-04-26T20:58:38.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.145+0000 7f40fea00640 1 -- 172.21.15.28:0/4040491290 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f0032440 con 0x7f410014c300 2024-04-26T20:58:38.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.145+0000 7f40fea00640 1 -- 172.21.15.28:0/4040491290 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0032810 con 0x7f410014c300 2024-04-26T20:58:38.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.145+0000 7f4104c00640 1 -- 172.21.15.28:0/4040491290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f410014c300 msgr2=0x7f41000ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:38.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.145+0000 7f4104c00640 1 --2- 172.21.15.28:0/4040491290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f410014c300 0x7f41000ce620 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f40f0009870 tx=0x7f40f002a250 comp rx=0 tx=0).stop 2024-04-26T20:58:38.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.145+0000 7f4104c00640 1 -- 172.21.15.28:0/4040491290 shutdown_connections 2024-04-26T20:58:38.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.145+0000 7f4104c00640 1 --2- 172.21.15.28:0/4040491290 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41000cb480 0x7f41000cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:38.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.145+0000 7f4104c00640 1 --2- 172.21.15.28:0/4040491290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f410014c300 0x7f41000ce620 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:38.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.145+0000 7f4104c00640 1 -- 172.21.15.28:0/4040491290 >> 172.21.15.28:0/4040491290 conn(0x7f41000d0a50 msgr2=0x7f41000d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:38.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.145+0000 7f4104c00640 1 -- 172.21.15.28:0/4040491290 shutdown_connections 2024-04-26T20:58:38.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.146+0000 7f4104c00640 1 -- 172.21.15.28:0/4040491290 wait complete. 2024-04-26T20:58:38.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.147+0000 7f4104c00640 1 Processor -- start 2024-04-26T20:58:38.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.147+0000 7f4104c00640 1 -- start start 2024-04-26T20:58:38.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.148+0000 7f4104c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41000cb480 0x7f41001613b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:38.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.148+0000 7f4104c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f410014c300 0x7f41000b5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:38.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.148+0000 7f4104c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f410014ce60 0x7f41000b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:38.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.148+0000 7f4104c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f410006e4d0 con 0x7f41000cb480 2024-04-26T20:58:38.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.148+0000 7f4104c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f410006feb0 con 0x7f410014ce60 2024-04-26T20:58:38.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.148+0000 7f4104c00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f410006de80 con 0x7f410014c300 2024-04-26T20:58:38.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.149+0000 7f40ff400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41000cb480 0x7f41001613b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:38.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.149+0000 7f40ff400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41000cb480 0x7f41001613b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44832/0 (socket says 172.21.15.28:44832) 2024-04-26T20:58:38.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.149+0000 7f40ff400640 1 -- 172.21.15.28:0/2360286347 learned_addr learned my addr 172.21.15.28:0/2360286347 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:38.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.149+0000 7f40f7e00640 1 --2- 172.21.15.28:0/2360286347 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f410014c300 0x7f41000b5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:38.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.149+0000 7f40ff400640 1 -- 172.21.15.28:0/2360286347 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f410014c300 msgr2=0x7f41000b5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:38.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.149+0000 7f40ff400640 1 --2- 172.21.15.28:0/2360286347 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f410014c300 0x7f41000b5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:38.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.149+0000 7f40ff400640 1 -- 172.21.15.28:0/2360286347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f410014ce60 msgr2=0x7f41000b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:38.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.149+0000 7f40ff400640 1 --2- 172.21.15.28:0/2360286347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f410014ce60 0x7f41000b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:38.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.149+0000 7f40ff400640 1 -- 172.21.15.28:0/2360286347 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4100144dc0 con 0x7f41000cb480 2024-04-26T20:58:38.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.149+0000 7f40f7e00640 1 --2- 172.21.15.28:0/2360286347 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f410014c300 0x7f41000b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:38.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.150+0000 7f40ff400640 1 --2- 172.21.15.28:0/2360286347 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41000cb480 0x7f41001613b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f40f0000c00 tx=0x7f40f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:38.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.150+0000 7f40f7400640 1 -- 172.21.15.28:0/2360286347 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0037040 con 0x7f41000cb480 2024-04-26T20:58:38.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.150+0000 7f40f7400640 1 -- 172.21.15.28:0/2360286347 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40f0042070 con 0x7f41000cb480 2024-04-26T20:58:38.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.150+0000 7f40f7400640 1 -- 172.21.15.28:0/2360286347 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0030bd0 con 0x7f41000cb480 2024-04-26T20:58:38.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.150+0000 7f4104c00640 1 -- 172.21.15.28:0/2360286347 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41000b4790 con 0x7f41000cb480 2024-04-26T20:58:38.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.150+0000 7f4104c00640 1 -- 172.21.15.28:0/2360286347 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4100144ae0 con 0x7f41000cb480 2024-04-26T20:58:38.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.151+0000 7f4104c00640 1 -- 172.21.15.28:0/2360286347 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f410006a360 con 0x7f41000cb480 2024-04-26T20:58:38.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.152+0000 7f40f7400640 1 -- 172.21.15.28:0/2360286347 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f40f0047460 con 0x7f41000cb480 2024-04-26T20:58:38.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.154+0000 7f40f7400640 1 --2- 172.21.15.28:0/2360286347 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f40dc06c8f0 0x7f40dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:38.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.154+0000 7f40f7400640 1 -- 172.21.15.28:0/2360286347 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f40f00ac4a0 con 0x7f41000cb480 2024-04-26T20:58:38.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.154+0000 7f40f7e00640 1 --2- 172.21.15.28:0/2360286347 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f40dc06c8f0 0x7f40dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:38.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.155+0000 7f40f7e00640 1 --2- 172.21.15.28:0/2360286347 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f40dc06c8f0 0x7f40dc06edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f41000b49f0 tx=0x7f40e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:38.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.159+0000 7f40f7400640 1 -- 172.21.15.28:0/2360286347 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f40f007abc0 con 0x7f41000cb480 2024-04-26T20:58:38.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.408+0000 7f4104c00640 1 -- 172.21.15.28:0/2360286347 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41000b24b0 con 0x7f41000cb480 2024-04-26T20:58:38.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.408+0000 7f40f7400640 1 -- 172.21.15.28:0/2360286347 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3875 (secure 0 0 0) 0x7f40f007a4c0 con 0x7f41000cb480 2024-04-26T20:58:38.410 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:38.410 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4664,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/1364678224","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":1364678224},{"type":"v1","addr":"172.21.15.103:6834","nonce":1364678224}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4790,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2225652646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":2225652646},{"type":"v1","addr":"172.21.15.103:6835","nonce":2225652646}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4849,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4081926466","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":4081926466},{"type":"v1","addr":"172.21.15.28:6837","nonce":4081926466}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5175,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/1370252546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":1370252546},{"type":"v1","addr":"172.21.15.28:6835","nonce":1370252546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-04-26T20:58:38.410 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 19 2024-04-26T20:58:38.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.412+0000 7f4104c00640 1 -- 172.21.15.28:0/2360286347 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f40dc06c8f0 msgr2=0x7f40dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:38.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.412+0000 7f4104c00640 1 --2- 172.21.15.28:0/2360286347 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f40dc06c8f0 0x7f40dc06edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f41000b49f0 tx=0x7f40e8009040 comp rx=0 tx=0).stop 2024-04-26T20:58:38.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.412+0000 7f4104c00640 1 -- 172.21.15.28:0/2360286347 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41000cb480 msgr2=0x7f41001613b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:38.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.412+0000 7f4104c00640 1 --2- 172.21.15.28:0/2360286347 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41000cb480 0x7f41001613b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f40f0000c00 tx=0x7f40f00046c0 comp rx=0 tx=0).stop 2024-04-26T20:58:38.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.412+0000 7f4104c00640 1 -- 172.21.15.28:0/2360286347 shutdown_connections 2024-04-26T20:58:38.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.412+0000 7f4104c00640 1 --2- 172.21.15.28:0/2360286347 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f40dc06c8f0 0x7f40dc06edb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:38.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.412+0000 7f4104c00640 1 --2- 172.21.15.28:0/2360286347 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f410014ce60 0x7f41000b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:38.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.412+0000 7f4104c00640 1 --2- 172.21.15.28:0/2360286347 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f410014c300 0x7f41000b5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:38.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.412+0000 7f4104c00640 1 --2- 172.21.15.28:0/2360286347 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41000cb480 0x7f41001613b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:38.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.412+0000 7f4104c00640 1 -- 172.21.15.28:0/2360286347 >> 172.21.15.28:0/2360286347 conn(0x7f41000d0a50 msgr2=0x7f41000d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:38.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.412+0000 7f4104c00640 1 -- 172.21.15.28:0/2360286347 shutdown_connections 2024-04-26T20:58:38.414 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.412+0000 7f4104c00640 1 -- 172.21.15.28:0/2360286347 wait complete. 2024-04-26T20:58:38.433 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T20:58:38.433 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T20:58:38.433 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T20:58:38.433 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:58:38.434 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T20:58:38.434 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T20:58:38.434 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T20:58:38.434 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:58:38.434 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T20:58:38.434 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T20:58:38.434 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T20:58:38.434 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:58:38.435 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T20:58:38.435 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T20:58:38.435 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T20:58:38.435 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T20:58:38.451 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T20:58:38.451 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T20:58:38.452 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T20:58:38.452 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T20:58:38.459 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T20:58:38.459 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T20:58:38.460 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T20:58:38.460 DEBUG:teuthology.orchestra.run.smithi103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T20:58:38.499 INFO:tasks.ceph.mds.a:Started 2024-04-26T20:58:38.500 DEBUG:teuthology.parallel:result is None 2024-04-26T20:58:38.506 INFO:tasks.ceph.mds.c:Started 2024-04-26T20:58:38.506 DEBUG:teuthology.parallel:result is None 2024-04-26T20:58:38.507 INFO:tasks.ceph.mds.b:Started 2024-04-26T20:58:38.507 DEBUG:teuthology.parallel:result is None 2024-04-26T20:58:38.513 INFO:tasks.ceph.mds.d:Started 2024-04-26T20:58:38.513 DEBUG:teuthology.parallel:result is None 2024-04-26T20:58:38.513 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T20:58:38.654 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2024-04-26T20:58:38.656 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-04-26T20:58:38.661 INFO:tasks.ceph.mds.b.smithi103.stdout:starting mds.b at 2024-04-26T20:58:38.661 INFO:tasks.ceph.mds.d.smithi103.stdout:starting mds.d at 2024-04-26T20:58:38.662 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-26T20:58:38.662 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-26T20:58:38.662 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-26T20:58:38.662 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-26T20:58:38.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.697+0000 7fdd42c00640 1 Processor -- start 2024-04-26T20:58:38.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.697+0000 7fdd42c00640 1 -- start start 2024-04-26T20:58:38.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.699+0000 7fdd42c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdd3c162e70 0x7fdd3c1632b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:38.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.699+0000 7fdd42c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd3c06d220 con 0x7fdd3c0b2430 2024-04-26T20:58:38.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.699+0000 7fdd42c00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd3c06d500 con 0x7fdd3c15ea20 2024-04-26T20:58:38.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.699+0000 7fdd42c00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fdd3c1637f0 con 0x7fdd3c162e70 2024-04-26T20:58:38.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.699+0000 7fdd3be00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fdd3c0b2430 0x7fdd3c06c6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:44652/0 (socket says 172.21.15.28:44652) 2024-04-26T20:58:38.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.699+0000 7fdd3be00640 1 -- 172.21.15.28:0/886605061 learned_addr learned my addr 172.21.15.28:0/886605061 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:38.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.699+0000 7fdd3b400640 1 --2- 172.21.15.28:0/886605061 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdd3c162e70 0x7fdd3c1632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:38.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.700+0000 7fdd3aa00640 1 -- 172.21.15.28:0/886605061 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1517051930 0 0) 0x7fdd3c06d220 con 0x7fdd3c0b2430 2024-04-26T20:58:38.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.700+0000 7fdd3aa00640 1 -- 172.21.15.28:0/886605061 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd20003520 con 0x7fdd3c0b2430 2024-04-26T20:58:38.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.700+0000 7fdd3b400640 1 -- 172.21.15.28:0/886605061 >> v1:172.21.15.103:6789/0 conn(0x7fdd3c15ea20 legacy=0x7fdd3c15eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:38.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.700+0000 7fdd3b400640 1 -- 172.21.15.28:0/886605061 >> v1:172.21.15.28:6789/0 conn(0x7fdd3c0b2430 legacy=0x7fdd3c06c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:38.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.700+0000 7fdd3b400640 1 -- 172.21.15.28:0/886605061 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3c06d7e0 con 0x7fdd3c162e70 2024-04-26T20:58:38.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.701+0000 7fdd3b400640 1 --2- 172.21.15.28:0/886605061 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdd3c162e70 0x7fdd3c1632b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdd240097d0 tx=0x7fdd2402a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a05643637d6dd545 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:38.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.701+0000 7fdd3aa00640 1 -- 172.21.15.28:0/886605061 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24037070 con 0x7fdd3c162e70 2024-04-26T20:58:38.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.701+0000 7fdd3aa00640 1 -- 172.21.15.28:0/886605061 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd24032440 con 0x7fdd3c162e70 2024-04-26T20:58:38.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.701+0000 7fdd3aa00640 1 -- 172.21.15.28:0/886605061 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd24032810 con 0x7fdd3c162e70 2024-04-26T20:58:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.702+0000 7fdd42c00640 1 -- 172.21.15.28:0/886605061 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdd3c162e70 msgr2=0x7fdd3c1632b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.702+0000 7fdd42c00640 1 --2- 172.21.15.28:0/886605061 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdd3c162e70 0x7fdd3c1632b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdd240097d0 tx=0x7fdd2402a200 comp rx=0 tx=0).stop 2024-04-26T20:58:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.702+0000 7fdd42c00640 1 -- 172.21.15.28:0/886605061 shutdown_connections 2024-04-26T20:58:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.702+0000 7fdd42c00640 1 --2- 172.21.15.28:0/886605061 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdd3c162e70 0x7fdd3c1632b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.702+0000 7fdd42c00640 1 -- 172.21.15.28:0/886605061 >> 172.21.15.28:0/886605061 conn(0x7fdd3c09ba50 msgr2=0x7fdd3c09be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.702+0000 7fdd42c00640 1 -- 172.21.15.28:0/886605061 shutdown_connections 2024-04-26T20:58:38.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.702+0000 7fdd42c00640 1 -- 172.21.15.28:0/886605061 wait complete. 2024-04-26T20:58:38.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.704+0000 7fdd42c00640 1 Processor -- start 2024-04-26T20:58:38.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.704+0000 7fdd42c00640 1 -- start start 2024-04-26T20:58:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.705+0000 7fdd42c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdd3c0b2430 0x7fdd3c0b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.705+0000 7fdd42c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd3c15ea20 0x7fdd3c0b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.705+0000 7fdd42c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd3c162e70 0x7fdd3c1f8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.705+0000 7fdd42c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c15f5c0 con 0x7fdd3c162e70 2024-04-26T20:58:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.705+0000 7fdd42c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3c163c30 con 0x7fdd3c15ea20 2024-04-26T20:58:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.705+0000 7fdd42c00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fdd3c15f740 con 0x7fdd3c0b2430 2024-04-26T20:58:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.705+0000 7fdd40c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd3c162e70 0x7fdd3c1f8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.705+0000 7fdd40c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd3c162e70 0x7fdd3c1f8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44882/0 (socket says 172.21.15.28:44882) 2024-04-26T20:58:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.705+0000 7fdd40c00640 1 -- 172.21.15.28:0/4195716830 learned_addr learned my addr 172.21.15.28:0/4195716830 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.705+0000 7fdd3b400640 1 --2- 172.21.15.28:0/4195716830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd3c15ea20 0x7fdd3c0b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:38.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.705+0000 7fdd3be00640 1 --2- 172.21.15.28:0/4195716830 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdd3c0b2430 0x7fdd3c0b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:38.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.706+0000 7fdd40c00640 1 -- 172.21.15.28:0/4195716830 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdd3c0b2430 msgr2=0x7fdd3c0b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:38.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.706+0000 7fdd40c00640 1 --2- 172.21.15.28:0/4195716830 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdd3c0b2430 0x7fdd3c0b4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:38.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.706+0000 7fdd40c00640 1 -- 172.21.15.28:0/4195716830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd3c15ea20 msgr2=0x7fdd3c0b6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:38.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.706+0000 7fdd40c00640 1 --2- 172.21.15.28:0/4195716830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd3c15ea20 0x7fdd3c0b6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:38.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.706+0000 7fdd40c00640 1 -- 172.21.15.28:0/4195716830 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd3c1f8a40 con 0x7fdd3c162e70 2024-04-26T20:58:38.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.706+0000 7fdd40c00640 1 --2- 172.21.15.28:0/4195716830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd3c162e70 0x7fdd3c1f8500 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdd34001eb0 tx=0x7fdd3400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:38.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.706+0000 7fdd33e00640 1 -- 172.21.15.28:0/4195716830 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd34017070 con 0x7fdd3c162e70 2024-04-26T20:58:38.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.706+0000 7fdd33e00640 1 -- 172.21.15.28:0/4195716830 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd340052d0 con 0x7fdd3c162e70 2024-04-26T20:58:38.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.706+0000 7fdd42c00640 1 -- 172.21.15.28:0/4195716830 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd3c1f8ce0 con 0x7fdd3c162e70 2024-04-26T20:58:38.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.706+0000 7fdd33e00640 1 -- 172.21.15.28:0/4195716830 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd34003810 con 0x7fdd3c162e70 2024-04-26T20:58:38.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.707+0000 7fdd42c00640 1 -- 172.21.15.28:0/4195716830 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd3c1f92a0 con 0x7fdd3c162e70 2024-04-26T20:58:38.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.708+0000 7fdd42c00640 1 -- 172.21.15.28:0/4195716830 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd3c06a400 con 0x7fdd3c162e70 2024-04-26T20:58:38.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.709+0000 7fdd33e00640 1 -- 172.21.15.28:0/4195716830 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd3401a070 con 0x7fdd3c162e70 2024-04-26T20:58:38.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.710+0000 7fdd33e00640 1 --2- 172.21.15.28:0/4195716830 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fdd1806c8f0 0x7fdd1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:38.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.710+0000 7fdd33e00640 1 -- 172.21.15.28:0/4195716830 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fdd3408afe0 con 0x7fdd3c162e70 2024-04-26T20:58:38.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.710+0000 7fdd3be00640 1 --2- 172.21.15.28:0/4195716830 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fdd1806c8f0 0x7fdd1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:38.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.711+0000 7fdd3be00640 1 --2- 172.21.15.28:0/4195716830 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fdd1806c8f0 0x7fdd1806edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c001560 tx=0x7fdd2c001a90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:38.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.713+0000 7fdd33e00640 1 -- 172.21.15.28:0/4195716830 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd34021020 con 0x7fdd3c162e70 2024-04-26T20:58:38.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.930+0000 7fdd42c00640 1 -- 172.21.15.28:0/4195716830 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fdd3c0b5690 con 0x7fdd3c162e70 2024-04-26T20:58:38.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.985+0000 7fdd33e00640 1 -- 172.21.15.28:0/4195716830 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v39) ==== 97+0+0 (secure 0 0 0) 0x7fdd340590a0 con 0x7fdd3c162e70 2024-04-26T20:58:38.986 INFO:teuthology.orchestra.run.smithi028.stderr: removed all blocklist entries 2024-04-26T20:58:38.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.988+0000 7fdd42c00640 1 -- 172.21.15.28:0/4195716830 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fdd1806c8f0 msgr2=0x7fdd1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:38.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.988+0000 7fdd42c00640 1 --2- 172.21.15.28:0/4195716830 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fdd1806c8f0 0x7fdd1806edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c001560 tx=0x7fdd2c001a90 comp rx=0 tx=0).stop 2024-04-26T20:58:38.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.988+0000 7fdd42c00640 1 -- 172.21.15.28:0/4195716830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd3c162e70 msgr2=0x7fdd3c1f8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:38.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.988+0000 7fdd42c00640 1 --2- 172.21.15.28:0/4195716830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd3c162e70 0x7fdd3c1f8500 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdd34001eb0 tx=0x7fdd3400c890 comp rx=0 tx=0).stop 2024-04-26T20:58:38.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.988+0000 7fdd42c00640 1 -- 172.21.15.28:0/4195716830 shutdown_connections 2024-04-26T20:58:38.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.988+0000 7fdd42c00640 1 --2- 172.21.15.28:0/4195716830 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fdd1806c8f0 0x7fdd1806edb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:38.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.988+0000 7fdd42c00640 1 --2- 172.21.15.28:0/4195716830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd3c162e70 0x7fdd3c1f8500 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:38.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.988+0000 7fdd42c00640 1 --2- 172.21.15.28:0/4195716830 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd3c15ea20 0x7fdd3c0b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:38.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.988+0000 7fdd42c00640 1 --2- 172.21.15.28:0/4195716830 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdd3c0b2430 0x7fdd3c0b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:38.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.988+0000 7fdd42c00640 1 -- 172.21.15.28:0/4195716830 >> 172.21.15.28:0/4195716830 conn(0x7fdd3c09ba50 msgr2=0x7fdd3c0b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:38.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.988+0000 7fdd42c00640 1 -- 172.21.15.28:0/4195716830 shutdown_connections 2024-04-26T20:58:38.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:38.988+0000 7fdd42c00640 1 -- 172.21.15.28:0/4195716830 wait complete. 2024-04-26T20:58:39.011 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T20:58:39.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.205+0000 7f961de00640 1 Processor -- start 2024-04-26T20:58:39.206 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.205+0000 7f961de00640 1 -- start start 2024-04-26T20:58:39.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.206+0000 7f961de00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9618156180 0x7f96180b4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:39.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.207+0000 7f961de00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9618073eb0 0x7f961806e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:39.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.207+0000 7f961de00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96180721d0 con 0x7f96180b48a0 2024-04-26T20:58:39.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.207+0000 7f961de00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f9618072de0 con 0x7f9618156180 2024-04-26T20:58:39.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.207+0000 7f961de00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f96180724b0 con 0x7f9618073eb0 2024-04-26T20:58:39.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.207+0000 7f961ca00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9618156180 0x7f96180b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:39.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.207+0000 7f961ca00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9618156180 0x7f96180b4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44890/0 (socket says 172.21.15.28:44890) 2024-04-26T20:58:39.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.207+0000 7f961ca00640 1 -- 172.21.15.28:0/971739715 learned_addr learned my addr 172.21.15.28:0/971739715 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:39.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.207+0000 7f961ca00640 1 -- 172.21.15.28:0/971739715 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9618073eb0 msgr2=0x7f961806e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:39.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.207+0000 7f961ca00640 1 --2- 172.21.15.28:0/971739715 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9618073eb0 0x7f961806e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:39.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.207+0000 7f961ca00640 1 -- 172.21.15.28:0/971739715 >> v1:172.21.15.103:6789/0 conn(0x7f96180b48a0 legacy=0x7f96180737a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:39.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.207+0000 7f961ca00640 1 -- 172.21.15.28:0/971739715 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9618072790 con 0x7f9618156180 2024-04-26T20:58:39.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.209+0000 7f961ca00640 1 --2- 172.21.15.28:0/971739715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9618156180 0x7f96180b4360 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9608009e50 tx=0x7f960802a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4854863b29204ed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:39.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.209+0000 7f9617400640 1 -- 172.21.15.28:0/971739715 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608036070 con 0x7f9618156180 2024-04-26T20:58:39.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.209+0000 7f9617400640 1 -- 172.21.15.28:0/971739715 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9608031600 con 0x7f9618156180 2024-04-26T20:58:39.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.209+0000 7f9617400640 1 -- 172.21.15.28:0/971739715 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96080319d0 con 0x7f9618156180 2024-04-26T20:58:39.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.210+0000 7f961de00640 1 -- 172.21.15.28:0/971739715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9618156180 msgr2=0x7f96180b4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:39.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.210+0000 7f961de00640 1 --2- 172.21.15.28:0/971739715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9618156180 0x7f96180b4360 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9608009e50 tx=0x7f960802a750 comp rx=0 tx=0).stop 2024-04-26T20:58:39.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.210+0000 7f961de00640 1 -- 172.21.15.28:0/971739715 shutdown_connections 2024-04-26T20:58:39.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.210+0000 7f961de00640 1 --2- 172.21.15.28:0/971739715 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9618073eb0 0x7f961806e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:39.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.210+0000 7f961de00640 1 --2- 172.21.15.28:0/971739715 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9618156180 0x7f96180b4360 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:39.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.210+0000 7f961de00640 1 -- 172.21.15.28:0/971739715 >> 172.21.15.28:0/971739715 conn(0x7f96180b2720 msgr2=0x7f96180b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:39.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.210+0000 7f961de00640 1 -- 172.21.15.28:0/971739715 shutdown_connections 2024-04-26T20:58:39.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.210+0000 7f961de00640 1 -- 172.21.15.28:0/971739715 wait complete. 2024-04-26T20:58:39.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.211+0000 7f961de00640 1 Processor -- start 2024-04-26T20:58:39.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.212+0000 7f961de00640 1 -- start start 2024-04-26T20:58:39.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.212+0000 7f961de00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9618073eb0 0x7f9618142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:39.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.213+0000 7f961de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96180b48a0 0x7f96181f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:39.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.213+0000 7f961de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9618156180 0x7f96181585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:39.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.213+0000 7f961de00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9618073270 con 0x7f96180b48a0 2024-04-26T20:58:39.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.213+0000 7f961de00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f96180b3f50 con 0x7f9618156180 2024-04-26T20:58:39.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.213+0000 7f961de00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f96180733f0 con 0x7f9618073eb0 2024-04-26T20:58:39.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.213+0000 7f9617e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96180b48a0 0x7f96181f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:39.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.213+0000 7f9617e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96180b48a0 0x7f96181f4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44898/0 (socket says 172.21.15.28:44898) 2024-04-26T20:58:39.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.213+0000 7f9617e00640 1 -- 172.21.15.28:0/4263823067 learned_addr learned my addr 172.21.15.28:0/4263823067 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:39.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.213+0000 7f961d400640 1 --2- 172.21.15.28:0/4263823067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9618156180 0x7f96181585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:39.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.213+0000 7f9617e00640 1 -- 172.21.15.28:0/4263823067 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9618073eb0 msgr2=0x7f9618142cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:39.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.213+0000 7f9617e00640 1 --2- 172.21.15.28:0/4263823067 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9618073eb0 0x7f9618142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:39.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.213+0000 7f9617e00640 1 -- 172.21.15.28:0/4263823067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9618156180 msgr2=0x7f96181585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:39.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.213+0000 7f9617e00640 1 --2- 172.21.15.28:0/4263823067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9618156180 0x7f96181585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:39.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.213+0000 7f9617e00640 1 -- 172.21.15.28:0/4263823067 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96180b1a20 con 0x7f96180b48a0 2024-04-26T20:58:39.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.214+0000 7f9617e00640 1 --2- 172.21.15.28:0/4263823067 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96180b48a0 0x7f96181f4d10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f960000c430 tx=0x7f960000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:39.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.214+0000 7f9614c00640 1 -- 172.21.15.28:0/4263823067 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9600011790 con 0x7f96180b48a0 2024-04-26T20:58:39.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.214+0000 7f9614c00640 1 -- 172.21.15.28:0/4263823067 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9600017070 con 0x7f96180b48a0 2024-04-26T20:58:39.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.214+0000 7f9614c00640 1 -- 172.21.15.28:0/4263823067 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9600011bb0 con 0x7f96180b48a0 2024-04-26T20:58:39.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.214+0000 7f961de00640 1 -- 172.21.15.28:0/4263823067 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96180aeb10 con 0x7f96180b48a0 2024-04-26T20:58:39.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.214+0000 7f961de00640 1 -- 172.21.15.28:0/4263823067 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9618143230 con 0x7f96180b48a0 2024-04-26T20:58:39.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.215+0000 7f961de00640 1 -- 172.21.15.28:0/4263823067 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f961806a360 con 0x7f96180b48a0 2024-04-26T20:58:39.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.216+0000 7f9614c00640 1 -- 172.21.15.28:0/4263823067 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f960001c460 con 0x7f96180b48a0 2024-04-26T20:58:39.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.217+0000 7f9614c00640 1 --2- 172.21.15.28:0/4263823067 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f95f406c8f0 0x7f95f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:39.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.217+0000 7f9614c00640 1 -- 172.21.15.28:0/4263823067 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f960008b6e0 con 0x7f96180b48a0 2024-04-26T20:58:39.218 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.217+0000 7f961ca00640 1 --2- 172.21.15.28:0/4263823067 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f95f406c8f0 0x7f95f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:39.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.218+0000 7f961ca00640 1 --2- 172.21.15.28:0/4263823067 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f95f406c8f0 0x7f95f406edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9608033000 tx=0x7f960802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:39.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.222+0000 7f9614c00640 1 -- 172.21.15.28:0/4263823067 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9600059e50 con 0x7f96180b48a0 2024-04-26T20:58:39.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.490+0000 7f961de00640 1 -- 172.21.15.28:0/4263823067 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f961815dda0 con 0x7f96180b48a0 2024-04-26T20:58:39.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.491+0000 7f9614c00640 1 -- 172.21.15.28:0/4263823067 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) ==== 80+0+5304 (secure 0 0 0) 0x7f9600059750 con 0x7f96180b48a0 2024-04-26T20:58:39.492 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:39.493 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T20:58:39.493 INFO:teuthology.orchestra.run.smithi028.stdout: "auth_dump": [ 2024-04-26T20:58:39.493 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:39.493 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.a", 2024-04-26T20:58:39.493 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDUFCxm570OOxAAwJmAfkKZtNjHl5XzgYUjSQ==", 2024-04-26T20:58:39.493 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:39.493 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-04-26T20:58:39.493 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-04-26T20:58:39.493 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-04-26T20:58:39.493 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:39.493 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:39.493 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:39.493 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:39.494 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.b", 2024-04-26T20:58:39.494 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDVFCxm5oQyFhAAhrxIFQkOub2iuwfXwrSi4g==", 2024-04-26T20:58:39.494 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:39.494 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-04-26T20:58:39.494 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-04-26T20:58:39.494 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-04-26T20:58:39.494 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:39.494 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:39.494 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:39.494 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:39.494 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.c", 2024-04-26T20:58:39.494 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDVFCxmpDSsChAAYAKo1Jj6ExMaDgsWl0NT3g==", 2024-04-26T20:58:39.494 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:39.495 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-04-26T20:58:39.495 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-04-26T20:58:39.495 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-04-26T20:58:39.495 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:39.495 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:39.495 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:39.495 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:39.495 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.d", 2024-04-26T20:58:39.495 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDVFCxm9YPLIhAAbgtxdShF5uQEQMCabsUY0A==", 2024-04-26T20:58:39.495 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:39.495 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-04-26T20:58:39.495 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-04-26T20:58:39.495 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-04-26T20:58:39.495 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:39.496 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:39.496 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:39.496 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:39.496 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.0", 2024-04-26T20:58:39.496 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDXFCxmfzl7NRAAGayH8lsv4xFqpTC/xS/l3A==", 2024-04-26T20:58:39.496 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:39.496 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-04-26T20:58:39.496 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-04-26T20:58:39.496 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:39.496 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:39.496 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:39.496 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:39.496 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.1", 2024-04-26T20:58:39.496 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDbFCxmkZiIFRAAMBKG5zWUKbjum+GOHLr4Cg==", 2024-04-26T20:58:39.497 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:39.497 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-04-26T20:58:39.497 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-04-26T20:58:39.497 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:39.497 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:39.497 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:39.497 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:39.497 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.2", 2024-04-26T20:58:39.497 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDeFCxmKVSqIRAAawTDWXhcZjzrq2dswt+H2g==", 2024-04-26T20:58:39.497 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:39.497 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-04-26T20:58:39.497 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-04-26T20:58:39.497 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:39.497 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:39.497 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:39.498 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:39.498 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.3", 2024-04-26T20:58:39.498 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDhFCxmtKLGOhAAqvk9Jg+vsKnIgems1e8Ytg==", 2024-04-26T20:58:39.498 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:39.498 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-04-26T20:58:39.498 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-04-26T20:58:39.498 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:39.498 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:39.498 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:39.498 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:39.498 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.4", 2024-04-26T20:58:39.498 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDoFCxmwx9WChAA1zzH0x5gZXgcrVUQey4dwQ==", 2024-04-26T20:58:39.498 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:39.499 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-04-26T20:58:39.499 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-04-26T20:58:39.499 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:39.499 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:39.499 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:39.499 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:39.499 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.5", 2024-04-26T20:58:39.499 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDrFCxmsGDzJBAAnhFkDmONidO/0RgaVFbDGg==", 2024-04-26T20:58:39.499 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:39.499 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-04-26T20:58:39.499 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-04-26T20:58:39.499 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:39.499 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:39.499 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:39.500 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:39.500 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.6", 2024-04-26T20:58:39.500 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDuFCxmkj9/LxAA+8IvdhuCnsvEM56+y9stnQ==", 2024-04-26T20:58:39.500 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:39.500 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-04-26T20:58:39.500 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-04-26T20:58:39.500 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:39.500 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:39.500 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:39.500 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:39.500 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.7", 2024-04-26T20:58:39.500 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDxFCxmvsb6OhAA50znn/4LbyZqxI1xgMJIEw==", 2024-04-26T20:58:39.500 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:39.500 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2024-04-26T20:58:39.501 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2024-04-26T20:58:39.501 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:39.501 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:39.501 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:39.501 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:39.501 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.0", 2024-04-26T20:58:39.501 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDVFCxm/rQRLhAAGsnaSU0YseRGCeDvxvvp/A==", 2024-04-26T20:58:39.501 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:39.501 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-04-26T20:58:39.501 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow r", 2024-04-26T20:58:39.501 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-26T20:58:39.501 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:39.501 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:39.501 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:39.501 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.1", 2024-04-26T20:58:39.502 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDWFCxmFhfoAhAA3JGoo80M5CvuNjJyFg3akw==", 2024-04-26T20:58:39.502 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:39.502 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-04-26T20:58:39.502 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-04-26T20:58:39.502 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-04-26T20:58:39.502 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-04-26T20:58:39.502 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:39.502 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:39.502 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:39.502 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.2", 2024-04-26T20:58:39.502 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDVFCxm+ntKNRAABSG+8Hggkp8p4fQhi9UudA==", 2024-04-26T20:58:39.502 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:39.502 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-04-26T20:58:39.502 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-04-26T20:58:39.503 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-04-26T20:58:39.503 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-04-26T20:58:39.503 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:39.503 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:39.503 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:39.503 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.3", 2024-04-26T20:58:39.503 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDWFCxmur+LCxAAqUus6BuHeNXG2R5zM2j00A==", 2024-04-26T20:58:39.503 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:39.503 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2024-04-26T20:58:39.503 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2024-04-26T20:58:39.503 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2024-04-26T20:58:39.503 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2024-04-26T20:58:39.503 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:39.503 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:39.503 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:39.504 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.admin", 2024-04-26T20:58:39.504 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDUFCxmHtVlAxAAd3pFr6eAayMMfaQSnXoTwg==", 2024-04-26T20:58:39.504 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:39.504 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-04-26T20:58:39.504 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2024-04-26T20:58:39.504 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2024-04-26T20:58:39.504 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:39.504 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:39.504 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:39.504 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:39.504 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.x", 2024-04-26T20:58:39.504 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDUFCxmh49VMxAAYQbMibMZBy6hsiO53AZYXw==", 2024-04-26T20:58:39.504 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:39.504 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-04-26T20:58:39.504 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2024-04-26T20:58:39.504 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:39.505 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:39.505 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:39.505 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:39.505 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.y", 2024-04-26T20:58:39.505 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQDUFCxmod8TKxAA5myHdFdSXxwOb3F4fMcyiA==", 2024-04-26T20:58:39.505 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2024-04-26T20:58:39.505 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2024-04-26T20:58:39.505 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2024-04-26T20:58:39.505 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2024-04-26T20:58:39.505 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:39.505 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:39.505 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2024-04-26T20:58:39.505 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T20:58:39.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.494+0000 7f961de00640 1 -- 172.21.15.28:0/4263823067 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f95f406c8f0 msgr2=0x7f95f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:39.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.494+0000 7f961de00640 1 --2- 172.21.15.28:0/4263823067 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f95f406c8f0 0x7f95f406edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9608033000 tx=0x7f960802e000 comp rx=0 tx=0).stop 2024-04-26T20:58:39.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.494+0000 7f961de00640 1 -- 172.21.15.28:0/4263823067 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96180b48a0 msgr2=0x7f96181f4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:39.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.494+0000 7f961de00640 1 --2- 172.21.15.28:0/4263823067 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96180b48a0 0x7f96181f4d10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f960000c430 tx=0x7f960000c8f0 comp rx=0 tx=0).stop 2024-04-26T20:58:39.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.494+0000 7f961de00640 1 -- 172.21.15.28:0/4263823067 shutdown_connections 2024-04-26T20:58:39.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.494+0000 7f961de00640 1 --2- 172.21.15.28:0/4263823067 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f95f406c8f0 0x7f95f406edb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:39.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.494+0000 7f961de00640 1 --2- 172.21.15.28:0/4263823067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9618156180 0x7f96181585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:39.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.494+0000 7f961de00640 1 --2- 172.21.15.28:0/4263823067 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96180b48a0 0x7f96181f4d10 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:39.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.494+0000 7f961de00640 1 --2- 172.21.15.28:0/4263823067 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9618073eb0 0x7f9618142cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:39.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.494+0000 7f961de00640 1 -- 172.21.15.28:0/4263823067 >> 172.21.15.28:0/4263823067 conn(0x7f96180b2720 msgr2=0x7f9618160750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:39.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.495+0000 7f961de00640 1 -- 172.21.15.28:0/4263823067 shutdown_connections 2024-04-26T20:58:39.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.495+0000 7f961de00640 1 -- 172.21.15.28:0/4263823067 wait complete. 2024-04-26T20:58:39.512 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:58:39.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.710+0000 7ff41ae00640 1 Processor -- start 2024-04-26T20:58:39.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.710+0000 7ff41ae00640 1 -- start start 2024-04-26T20:58:39.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.711+0000 7ff41ae00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7ff41415f120 0x7ff41406f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:39.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.711+0000 7ff41ae00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff41406f900 0x7ff4140b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:39.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.711+0000 7ff41ae00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff41415f600 con 0x7ff41406d360 2024-04-26T20:58:39.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.711+0000 7ff41ae00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7ff41415f8e0 con 0x7ff41415f120 2024-04-26T20:58:39.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.711+0000 7ff41ae00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff41415fbc0 con 0x7ff41406f900 2024-04-26T20:58:39.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.712+0000 7ff419a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff41406d360 0x7ff41406d740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:44660/0 (socket says 172.21.15.28:44660) 2024-04-26T20:58:39.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.712+0000 7ff419a00640 1 -- 172.21.15.28:0/371347903 learned_addr learned my addr 172.21.15.28:0/371347903 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:39.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.712+0000 7ff419000640 1 --2- 172.21.15.28:0/371347903 >> v2:172.21.15.103:3300/0 conn(0x7ff41415f120 0x7ff41406f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:39.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.712+0000 7ff41a400640 1 --2- 172.21.15.28:0/371347903 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff41406f900 0x7ff4140b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:39.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.712+0000 7ff413e00640 1 -- 172.21.15.28:0/371347903 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3766390309 0 0) 0x7ff41415f600 con 0x7ff41406d360 2024-04-26T20:58:39.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.712+0000 7ff413e00640 1 -- 172.21.15.28:0/371347903 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3fc003520 con 0x7ff41406d360 2024-04-26T20:58:39.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.713+0000 7ff413e00640 1 -- 172.21.15.28:0/371347903 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4071968805 0 0) 0x7ff3fc003520 con 0x7ff41406d360 2024-04-26T20:58:39.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.713+0000 7ff413e00640 1 -- 172.21.15.28:0/371347903 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff41406f900 msgr2=0x7ff4140b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:39.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.713+0000 7ff413e00640 1 --2- 172.21.15.28:0/371347903 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff41406f900 0x7ff4140b4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:39.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.713+0000 7ff413e00640 1 -- 172.21.15.28:0/371347903 >> v2:172.21.15.103:3300/0 conn(0x7ff41415f120 msgr2=0x7ff41406f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:39.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.713+0000 7ff413e00640 1 --2- 172.21.15.28:0/371347903 >> v2:172.21.15.103:3300/0 conn(0x7ff41415f120 0x7ff41406f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:39.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.713+0000 7ff413e00640 1 -- 172.21.15.28:0/371347903 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41415fea0 con 0x7ff41406d360 2024-04-26T20:58:39.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.713+0000 7ff413e00640 1 -- 172.21.15.28:0/371347903 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7ff4080019f0 con 0x7ff41406d360 2024-04-26T20:58:39.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.713+0000 7ff41a400640 1 --2- 172.21.15.28:0/371347903 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff41406f900 0x7ff4140b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:58:39.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.713+0000 7ff413e00640 1 -- 172.21.15.28:0/371347903 >> v1:172.21.15.28:6789/0 conn(0x7ff41406d360 legacy=0x7ff41406d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:39.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.713+0000 7ff413e00640 1 --2- 172.21.15.28:0/371347903 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff41406f900 0x7ff3fc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:39.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.713+0000 7ff413e00640 1 --2- 172.21.15.28:0/371347903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3fc003f40 0x7ff3fc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:39.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.713+0000 7ff413e00640 1 --2- 172.21.15.28:0/371347903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3fc004d40 0x7ff3fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:39.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.713+0000 7ff413e00640 1 -- 172.21.15.28:0/371347903 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff41415f8e0 con 0x7ff3fc004d40 2024-04-26T20:58:39.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.713+0000 7ff413e00640 1 -- 172.21.15.28:0/371347903 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff41415fbc0 con 0x7ff3fc003f40 2024-04-26T20:58:39.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.713+0000 7ff413e00640 1 -- 172.21.15.28:0/371347903 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff3fc00ba40 con 0x7ff41406f900 2024-04-26T20:58:39.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.713+0000 7ff41a400640 1 --2- 172.21.15.28:0/371347903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3fc004d40 0x7ff3fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:39.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.714+0000 7ff419a00640 1 --2- 172.21.15.28:0/371347903 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff41406f900 0x7ff3fc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:39.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.714+0000 7ff419000640 1 --2- 172.21.15.28:0/371347903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3fc003f40 0x7ff3fc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:39.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.714+0000 7ff41a400640 1 -- 172.21.15.28:0/371347903 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff41406f900 msgr2=0x7ff3fc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:39.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.714+0000 7ff41a400640 1 --2- 172.21.15.28:0/371347903 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff41406f900 0x7ff3fc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:39.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.714+0000 7ff41a400640 1 -- 172.21.15.28:0/371347903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3fc003f40 msgr2=0x7ff3fc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:39.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.714+0000 7ff41a400640 1 --2- 172.21.15.28:0/371347903 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3fc003f40 0x7ff3fc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:39.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.714+0000 7ff41a400640 1 -- 172.21.15.28:0/371347903 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3fc003520 con 0x7ff3fc004d40 2024-04-26T20:58:39.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.715+0000 7ff41a400640 1 --2- 172.21.15.28:0/371347903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3fc004d40 0x7ff3fc0051c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0098a0 tx=0x7ff3fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=714cbeca05d240da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:39.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.715+0000 7ff413e00640 1 -- 172.21.15.28:0/371347903 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c00f040 con 0x7ff3fc004d40 2024-04-26T20:58:39.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.715+0000 7ff413e00640 1 -- 172.21.15.28:0/371347903 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff40c02ace0 con 0x7ff3fc004d40 2024-04-26T20:58:39.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.715+0000 7ff413e00640 1 -- 172.21.15.28:0/371347903 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c004250 con 0x7ff3fc004d40 2024-04-26T20:58:39.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.716+0000 7ff41ae00640 1 -- 172.21.15.28:0/371347903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3fc004d40 msgr2=0x7ff3fc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:39.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.716+0000 7ff41ae00640 1 --2- 172.21.15.28:0/371347903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff3fc004d40 0x7ff3fc0051c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0098a0 tx=0x7ff3fc005830 comp rx=0 tx=0).stop 2024-04-26T20:58:39.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.716+0000 7ff41a400640 1 -- 172.21.15.28:0/371347903 reap_dead start 2024-04-26T20:58:39.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.716+0000 7ff41ae00640 1 -- 172.21.15.28:0/371347903 shutdown_connections 2024-04-26T20:58:39.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.716+0000 7ff41ae00640 1 -- 172.21.15.28:0/371347903 >> 172.21.15.28:0/371347903 conn(0x7ff414072e90 msgr2=0x7ff4140732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:39.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.716+0000 7ff41ae00640 1 -- 172.21.15.28:0/371347903 shutdown_connections 2024-04-26T20:58:39.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.716+0000 7ff41ae00640 1 -- 172.21.15.28:0/371347903 wait complete. 2024-04-26T20:58:39.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.718+0000 7ff41ae00640 1 Processor -- start 2024-04-26T20:58:39.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.718+0000 7ff41ae00640 1 -- start start 2024-04-26T20:58:39.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.719+0000 7ff41ae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff414161990 0x7ff41415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:39.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.719+0000 7ff41ae00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff414154a70 0x7ff414157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:39.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.719+0000 7ff41ae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff41415b700 0x7ff414157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:39.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.719+0000 7ff41ae00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ff41406ff30 con 0x7ff41415b700 2024-04-26T20:58:39.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.719+0000 7ff41ae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff41406fdb0 con 0x7ff414161990 2024-04-26T20:58:39.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.719+0000 7ff41ae00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ff4140b4ca0 con 0x7ff414154a70 2024-04-26T20:58:39.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.719+0000 7ff41a400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff41415b700 0x7ff414157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:39.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.719+0000 7ff41a400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff41415b700 0x7ff414157c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:44924/0 (socket says 172.21.15.28:44924) 2024-04-26T20:58:39.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.719+0000 7ff41a400640 1 -- 172.21.15.28:0/674933584 learned_addr learned my addr 172.21.15.28:0/674933584 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:39.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.720+0000 7ff419a00640 1 --2- 172.21.15.28:0/674933584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff414161990 0x7ff41415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:39.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.720+0000 7ff419000640 1 --2- 172.21.15.28:0/674933584 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff414154a70 0x7ff414157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:39.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.720+0000 7ff41a400640 1 -- 172.21.15.28:0/674933584 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff414154a70 msgr2=0x7ff414157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:39.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.720+0000 7ff41a400640 1 --2- 172.21.15.28:0/674933584 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff414154a70 0x7ff414157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:39.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.720+0000 7ff41a400640 1 -- 172.21.15.28:0/674933584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff414161990 msgr2=0x7ff41415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:39.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.720+0000 7ff41a400640 1 --2- 172.21.15.28:0/674933584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff414161990 0x7ff41415b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:39.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.720+0000 7ff41a400640 1 -- 172.21.15.28:0/674933584 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff414156990 con 0x7ff41415b700 2024-04-26T20:58:39.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.720+0000 7ff41a400640 1 --2- 172.21.15.28:0/674933584 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff41415b700 0x7ff414157c90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0099c0 tx=0x7ff40c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:39.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.720+0000 7ff411600640 1 -- 172.21.15.28:0/674933584 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c006e10 con 0x7ff41415b700 2024-04-26T20:58:39.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.720+0000 7ff411600640 1 -- 172.21.15.28:0/674933584 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff40c044070 con 0x7ff41415b700 2024-04-26T20:58:39.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.720+0000 7ff411600640 1 -- 172.21.15.28:0/674933584 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c00f000 con 0x7ff41415b700 2024-04-26T20:58:39.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.720+0000 7ff41ae00640 1 -- 172.21.15.28:0/674933584 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff414156c70 con 0x7ff41415b700 2024-04-26T20:58:39.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.721+0000 7ff41ae00640 1 -- 172.21.15.28:0/674933584 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff414157230 con 0x7ff41415b700 2024-04-26T20:58:39.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.722+0000 7ff41ae00640 1 -- 172.21.15.28:0/674933584 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff414070fe0 con 0x7ff41415b700 2024-04-26T20:58:39.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.723+0000 7ff411600640 1 -- 172.21.15.28:0/674933584 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff40c03b040 con 0x7ff41415b700 2024-04-26T20:58:39.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.724+0000 7ff411600640 1 --2- 172.21.15.28:0/674933584 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff3f406c8f0 0x7ff3f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:39.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.724+0000 7ff411600640 1 -- 172.21.15.28:0/674933584 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7ff40c0ab580 con 0x7ff41415b700 2024-04-26T20:58:39.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.724+0000 7ff419a00640 1 --2- 172.21.15.28:0/674933584 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff3f406c8f0 0x7ff3f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:39.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.725+0000 7ff419a00640 1 --2- 172.21.15.28:0/674933584 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff3f406c8f0 0x7ff3f406edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff408004bc0 tx=0x7ff408009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:39.730 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.729+0000 7ff411600640 1 -- 172.21.15.28:0/674933584 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff414070fe0 con 0x7ff41415b700 2024-04-26T20:58:39.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.933+0000 7ff41ae00640 1 -- 172.21.15.28:0/674933584 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff414157430 con 0x7ff41415b700 2024-04-26T20:58:39.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.935+0000 7ff411600640 1 -- 172.21.15.28:0/674933584 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+12867 (secure 0 0 0) 0x7ff414157430 con 0x7ff41415b700 2024-04-26T20:58:39.936 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:39.937 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":41,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:58:38.997340+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:58:39.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.938+0000 7ff41ae00640 1 -- 172.21.15.28:0/674933584 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff3f406c8f0 msgr2=0x7ff3f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:39.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.938+0000 7ff41ae00640 1 --2- 172.21.15.28:0/674933584 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff3f406c8f0 0x7ff3f406edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff408004bc0 tx=0x7ff408009040 comp rx=0 tx=0).stop 2024-04-26T20:58:39.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.938+0000 7ff41ae00640 1 -- 172.21.15.28:0/674933584 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff41415b700 msgr2=0x7ff414157c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:39.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.938+0000 7ff41ae00640 1 --2- 172.21.15.28:0/674933584 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff41415b700 0x7ff414157c90 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0099c0 tx=0x7ff40c004730 comp rx=0 tx=0).stop 2024-04-26T20:58:39.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.938+0000 7ff41ae00640 1 -- 172.21.15.28:0/674933584 shutdown_connections 2024-04-26T20:58:39.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.938+0000 7ff41ae00640 1 --2- 172.21.15.28:0/674933584 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff3f406c8f0 0x7ff3f406edb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:39.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.938+0000 7ff41ae00640 1 --2- 172.21.15.28:0/674933584 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff41415b700 0x7ff414157c90 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:39.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.938+0000 7ff41ae00640 1 --2- 172.21.15.28:0/674933584 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ff414154a70 0x7ff414157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:39.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.938+0000 7ff41ae00640 1 --2- 172.21.15.28:0/674933584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff414161990 0x7ff41415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:39.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.938+0000 7ff41ae00640 1 -- 172.21.15.28:0/674933584 >> 172.21.15.28:0/674933584 conn(0x7ff414072e90 msgr2=0x7ff4140757b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:39.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.938+0000 7ff41ae00640 1 -- 172.21.15.28:0/674933584 shutdown_connections 2024-04-26T20:58:39.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:39.938+0000 7ff41ae00640 1 -- 172.21.15.28:0/674933584 wait complete. 2024-04-26T20:58:39.958 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T20:58:39.959 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T20:58:40.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.153+0000 7f3c27200640 1 Processor -- start 2024-04-26T20:58:40.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.153+0000 7f3c27200640 1 -- start start 2024-04-26T20:58:40.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.154+0000 7f3c27200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3c201465c0 0x7f3c2006e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:40.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.155+0000 7f3c27200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c200b5ba0 con 0x7f3c200d1420 2024-04-26T20:58:40.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.155+0000 7f3c27200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c201469a0 con 0x7f3c200cf040 2024-04-26T20:58:40.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.155+0000 7f3c27200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f3c200cf520 con 0x7f3c201465c0 2024-04-26T20:58:40.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.155+0000 7f3c26800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3c200d1420 0x7f3c200b3a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52900/0 (socket says 172.21.15.28:52900) 2024-04-26T20:58:40.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.155+0000 7f3c26800640 1 -- 172.21.15.28:0/1513386993 learned_addr learned my addr 172.21.15.28:0/1513386993 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:40.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.155+0000 7f3c24a00640 1 -- 172.21.15.28:0/1513386993 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 499617376 0 0) 0x7f3c1c001110 con 0x7f3c200d1420 2024-04-26T20:58:40.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.155+0000 7f3c24a00640 1 -- 172.21.15.28:0/1513386993 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c0c003520 con 0x7f3c200d1420 2024-04-26T20:58:40.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.156+0000 7f3c25e00640 1 --2- 172.21.15.28:0/1513386993 >> v2:172.21.15.103:3300/0 conn(0x7f3c201465c0 0x7f3c2006e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:40.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.156+0000 7f3c24a00640 1 -- 172.21.15.28:0/1513386993 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2025852548 0 0) 0x7f3c0c003520 con 0x7f3c200d1420 2024-04-26T20:58:40.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.156+0000 7f3c24a00640 1 -- 172.21.15.28:0/1513386993 >> v2:172.21.15.103:3300/0 conn(0x7f3c201465c0 msgr2=0x7f3c2006e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:40.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.156+0000 7f3c24a00640 1 --2- 172.21.15.28:0/1513386993 >> v2:172.21.15.103:3300/0 conn(0x7f3c201465c0 0x7f3c2006e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.156+0000 7f3c24a00640 1 -- 172.21.15.28:0/1513386993 >> v1:172.21.15.103:6789/0 conn(0x7f3c200cf040 legacy=0x7f3c200d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:40.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.156+0000 7f3c24a00640 1 -- 172.21.15.28:0/1513386993 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c200cf800 con 0x7f3c200d1420 2024-04-26T20:58:40.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.156+0000 7f3c24a00640 1 -- 172.21.15.28:0/1513386993 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f3c1c001840 con 0x7f3c200d1420 2024-04-26T20:58:40.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.156+0000 7f3c24a00640 1 -- 172.21.15.28:0/1513386993 >> v1:172.21.15.28:6789/0 conn(0x7f3c200d1420 legacy=0x7f3c200b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:40.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.156+0000 7f3c24a00640 1 --2- 172.21.15.28:0/1513386993 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3c0c003a00 0x7f3c0c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:40.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.156+0000 7f3c24a00640 1 --2- 172.21.15.28:0/1513386993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c0c004800 0x7f3c0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:40.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.156+0000 7f3c24a00640 1 --2- 172.21.15.28:0/1513386993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c0c0052e0 0x7f3c0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:40.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.156+0000 7f3c24a00640 1 -- 172.21.15.28:0/1513386993 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3c200cf520 con 0x7f3c0c0052e0 2024-04-26T20:58:40.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.156+0000 7f3c24a00640 1 -- 172.21.15.28:0/1513386993 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c00c200 con 0x7f3c0c004800 2024-04-26T20:58:40.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.156+0000 7f3c24a00640 1 -- 172.21.15.28:0/1513386993 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f3c0c00c400 con 0x7f3c0c003a00 2024-04-26T20:58:40.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.156+0000 7f3c26800640 1 --2- 172.21.15.28:0/1513386993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c0c0052e0 0x7f3c0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:40.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.157+0000 7f3c26800640 1 -- 172.21.15.28:0/1513386993 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3c0c003a00 msgr2=0x7f3c0c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:40.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.157+0000 7f3c26800640 1 --2- 172.21.15.28:0/1513386993 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3c0c003a00 0x7f3c0c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.157+0000 7f3c26800640 1 -- 172.21.15.28:0/1513386993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c0c004800 msgr2=0x7f3c0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:40.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.157+0000 7f3c26800640 1 --2- 172.21.15.28:0/1513386993 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c0c004800 0x7f3c0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.157+0000 7f3c26800640 1 -- 172.21.15.28:0/1513386993 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0c003520 con 0x7f3c0c0052e0 2024-04-26T20:58:40.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.157+0000 7f3c26800640 1 -- 172.21.15.28:0/1513386993 reap_dead start 2024-04-26T20:58:40.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.158+0000 7f3c26800640 1 --2- 172.21.15.28:0/1513386993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c0c0052e0 0x7f3c0c00b9c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00c320 tx=0x7f3c0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadcf50755451ba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:40.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.158+0000 7f3c24a00640 1 -- 172.21.15.28:0/1513386993 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c035070 con 0x7f3c0c0052e0 2024-04-26T20:58:40.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.158+0000 7f3c24a00640 1 -- 172.21.15.28:0/1513386993 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c1c005ad0 con 0x7f3c0c0052e0 2024-04-26T20:58:40.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.158+0000 7f3c24a00640 1 -- 172.21.15.28:0/1513386993 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c007e10 con 0x7f3c0c0052e0 2024-04-26T20:58:40.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.158+0000 7f3c27200640 1 -- 172.21.15.28:0/1513386993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c0c0052e0 msgr2=0x7f3c0c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:40.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.158+0000 7f3c27200640 1 --2- 172.21.15.28:0/1513386993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c0c0052e0 0x7f3c0c00b9c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00c320 tx=0x7f3c0c005830 comp rx=0 tx=0).stop 2024-04-26T20:58:40.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.158+0000 7f3c27200640 1 -- 172.21.15.28:0/1513386993 shutdown_connections 2024-04-26T20:58:40.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.158+0000 7f3c27200640 1 --2- 172.21.15.28:0/1513386993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c0c0052e0 0x7f3c0c00b9c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.158+0000 7f3c27200640 1 -- 172.21.15.28:0/1513386993 >> 172.21.15.28:0/1513386993 conn(0x7f3c2006c530 msgr2=0x7f3c2006c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:40.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.159+0000 7f3c27200640 1 -- 172.21.15.28:0/1513386993 shutdown_connections 2024-04-26T20:58:40.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.159+0000 7f3c27200640 1 -- 172.21.15.28:0/1513386993 wait complete. 2024-04-26T20:58:40.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.160+0000 7f3c27200640 1 Processor -- start 2024-04-26T20:58:40.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.160+0000 7f3c27200640 1 -- start start 2024-04-26T20:58:40.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.162+0000 7f3c27200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c0c0052e0 0x7f3c201f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:40.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.162+0000 7f3c27200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c2014cec0 0x7f3c2013ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:40.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.162+0000 7f3c27200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3c2013d3a0 0x7f3c201e7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:40.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.162+0000 7f3c27200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3c200d00a0 con 0x7f3c0c0052e0 2024-04-26T20:58:40.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.162+0000 7f3c27200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c200cfdc0 con 0x7f3c2014cec0 2024-04-26T20:58:40.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.162+0000 7f3c27200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f3c200d0d10 con 0x7f3c2013d3a0 2024-04-26T20:58:40.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.162+0000 7f3c25e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c0c0052e0 0x7f3c201f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:40.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.162+0000 7f3c25e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c0c0052e0 0x7f3c201f28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53772/0 (socket says 172.21.15.28:53772) 2024-04-26T20:58:40.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.162+0000 7f3c25e00640 1 -- 172.21.15.28:0/396686469 learned_addr learned my addr 172.21.15.28:0/396686469 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:40.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.162+0000 7f3c25e00640 1 -- 172.21.15.28:0/396686469 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3c2013d3a0 msgr2=0x7f3c201e7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:40.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.162+0000 7f3c25e00640 1 --2- 172.21.15.28:0/396686469 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3c2013d3a0 0x7f3c201e7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.162+0000 7f3c25e00640 1 -- 172.21.15.28:0/396686469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c2014cec0 msgr2=0x7f3c2013ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:40.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.162+0000 7f3c25e00640 1 --2- 172.21.15.28:0/396686469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c2014cec0 0x7f3c2013ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.162+0000 7f3c25e00640 1 -- 172.21.15.28:0/396686469 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c201f2e20 con 0x7f3c0c0052e0 2024-04-26T20:58:40.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.163+0000 7f3c25e00640 1 --2- 172.21.15.28:0/396686469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c0c0052e0 0x7f3c201f28e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3c18007e60 tx=0x7f3c1800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:40.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.163+0000 7f3c16000640 1 -- 172.21.15.28:0/396686469 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18014420 con 0x7f3c0c0052e0 2024-04-26T20:58:40.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.163+0000 7f3c16000640 1 -- 172.21.15.28:0/396686469 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c18019070 con 0x7f3c0c0052e0 2024-04-26T20:58:40.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.163+0000 7f3c16000640 1 -- 172.21.15.28:0/396686469 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18014800 con 0x7f3c0c0052e0 2024-04-26T20:58:40.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.163+0000 7f3c27200640 1 -- 172.21.15.28:0/396686469 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2013afe0 con 0x7f3c0c0052e0 2024-04-26T20:58:40.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.163+0000 7f3c27200640 1 -- 172.21.15.28:0/396686469 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c2013b2a0 con 0x7f3c0c0052e0 2024-04-26T20:58:40.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.164+0000 7f3c27200640 1 -- 172.21.15.28:0/396686469 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c2013d7c0 con 0x7f3c0c0052e0 2024-04-26T20:58:40.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.165+0000 7f3c16000640 1 -- 172.21.15.28:0/396686469 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c180149a0 con 0x7f3c0c0052e0 2024-04-26T20:58:40.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.166+0000 7f3c16000640 1 --2- 172.21.15.28:0/396686469 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3c0406c920 0x7f3c0406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:40.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.167+0000 7f3c16000640 1 -- 172.21.15.28:0/396686469 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f3c1808c090 con 0x7f3c0c0052e0 2024-04-26T20:58:40.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.167+0000 7f3c25400640 1 --2- 172.21.15.28:0/396686469 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3c0406c920 0x7f3c0406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:40.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.167+0000 7f3c25400640 1 --2- 172.21.15.28:0/396686469 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3c0406c920 0x7f3c0406ede0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3c201e8910 tx=0x7f3c100079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:40.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.171+0000 7f3c16000640 1 -- 172.21.15.28:0/396686469 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3c1805a770 con 0x7f3c0c0052e0 2024-04-26T20:58:40.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.371+0000 7f3c27200640 1 -- 172.21.15.28:0/396686469 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3c2006a250 con 0x7f3c0c0052e0 2024-04-26T20:58:40.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.372+0000 7f3c16000640 1 -- 172.21.15.28:0/396686469 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7f3c1805a070 con 0x7f3c0c0052e0 2024-04-26T20:58:40.373 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T20:58:40.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.374+0000 7f3c27200640 1 -- 172.21.15.28:0/396686469 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3c0406c920 msgr2=0x7f3c0406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:40.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.374+0000 7f3c27200640 1 --2- 172.21.15.28:0/396686469 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3c0406c920 0x7f3c0406ede0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3c201e8910 tx=0x7f3c100079d0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.374+0000 7f3c27200640 1 -- 172.21.15.28:0/396686469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c0c0052e0 msgr2=0x7f3c201f28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:40.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.374+0000 7f3c27200640 1 --2- 172.21.15.28:0/396686469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c0c0052e0 0x7f3c201f28e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3c18007e60 tx=0x7f3c1800e4b0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.374+0000 7f3c27200640 1 -- 172.21.15.28:0/396686469 shutdown_connections 2024-04-26T20:58:40.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.374+0000 7f3c27200640 1 --2- 172.21.15.28:0/396686469 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3c0406c920 0x7f3c0406ede0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.374+0000 7f3c27200640 1 --2- 172.21.15.28:0/396686469 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3c2013d3a0 0x7f3c201e7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.374+0000 7f3c27200640 1 --2- 172.21.15.28:0/396686469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c2014cec0 0x7f3c2013ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.374+0000 7f3c27200640 1 --2- 172.21.15.28:0/396686469 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3c0c0052e0 0x7f3c201f28e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.375+0000 7f3c27200640 1 -- 172.21.15.28:0/396686469 >> 172.21.15.28:0/396686469 conn(0x7f3c2006c530 msgr2=0x7f3c2006c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:40.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.375+0000 7f3c27200640 1 -- 172.21.15.28:0/396686469 shutdown_connections 2024-04-26T20:58:40.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.375+0000 7f3c27200640 1 -- 172.21.15.28:0/396686469 wait complete. 2024-04-26T20:58:40.396 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T20:58:40.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.597+0000 7feb94a00640 1 Processor -- start 2024-04-26T20:58:40.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.597+0000 7feb94a00640 1 -- start start 2024-04-26T20:58:40.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.599+0000 7feb94a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb90163ae0 0x7feb900b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:40.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.599+0000 7feb94a00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7feb9006f020 0x7feb90163ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:40.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.599+0000 7feb94a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7feb90164400 0x7feb90072790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:40.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.599+0000 7feb94a00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7feb90072020 con 0x7feb90164400 2024-04-26T20:58:40.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.599+0000 7feb94a00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7feb9006f830 con 0x7feb9006f020 2024-04-26T20:58:40.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.599+0000 7feb94a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7feb9006fb10 con 0x7feb90163ae0 2024-04-26T20:58:40.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.599+0000 7feb8e800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7feb90164400 0x7feb90072790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:40.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.599+0000 7feb8e800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7feb90164400 0x7feb90072790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53774/0 (socket says 172.21.15.28:53774) 2024-04-26T20:58:40.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.599+0000 7feb8e800640 1 -- 172.21.15.28:0/305617082 learned_addr learned my addr 172.21.15.28:0/305617082 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:40.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.600+0000 7feb8e800640 1 -- 172.21.15.28:0/305617082 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb90163ae0 msgr2=0x7feb900b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:40.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.600+0000 7feb8e800640 1 --2- 172.21.15.28:0/305617082 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb90163ae0 0x7feb900b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.600+0000 7feb8e800640 1 -- 172.21.15.28:0/305617082 >> v2:172.21.15.103:3300/0 conn(0x7feb9006f020 msgr2=0x7feb90163ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:40.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.600+0000 7feb8e800640 1 --2- 172.21.15.28:0/305617082 >> v2:172.21.15.103:3300/0 conn(0x7feb9006f020 0x7feb90163ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.600+0000 7feb8e800640 1 -- 172.21.15.28:0/305617082 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9006fdf0 con 0x7feb90164400 2024-04-26T20:58:40.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.601+0000 7feb8e800640 1 --2- 172.21.15.28:0/305617082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb90164400 0x7feb90072790 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7feb90070150 tx=0x7feb8802a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57fb53878a65dc83 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:40.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.601+0000 7feb8d400640 1 -- 172.21.15.28:0/305617082 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88037070 con 0x7feb90164400 2024-04-26T20:58:40.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.601+0000 7feb8d400640 1 -- 172.21.15.28:0/305617082 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb88032440 con 0x7feb90164400 2024-04-26T20:58:40.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.601+0000 7feb8d400640 1 -- 172.21.15.28:0/305617082 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88032810 con 0x7feb90164400 2024-04-26T20:58:40.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.602+0000 7feb94a00640 1 -- 172.21.15.28:0/305617082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb90164400 msgr2=0x7feb90072790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:40.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.602+0000 7feb94a00640 1 --2- 172.21.15.28:0/305617082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb90164400 0x7feb90072790 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7feb90070150 tx=0x7feb8802a1d0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.602+0000 7feb94a00640 1 -- 172.21.15.28:0/305617082 shutdown_connections 2024-04-26T20:58:40.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.602+0000 7feb94a00640 1 --2- 172.21.15.28:0/305617082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb90164400 0x7feb90072790 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.602+0000 7feb94a00640 1 --2- 172.21.15.28:0/305617082 >> v2:172.21.15.103:3300/0 conn(0x7feb9006f020 0x7feb90163ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.602+0000 7feb94a00640 1 --2- 172.21.15.28:0/305617082 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb90163ae0 0x7feb900b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.602+0000 7feb94a00640 1 -- 172.21.15.28:0/305617082 >> 172.21.15.28:0/305617082 conn(0x7feb9006c770 msgr2=0x7feb9006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:40.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.602+0000 7feb94a00640 1 -- 172.21.15.28:0/305617082 shutdown_connections 2024-04-26T20:58:40.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.602+0000 7feb94a00640 1 -- 172.21.15.28:0/305617082 wait complete. 2024-04-26T20:58:40.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.603+0000 7feb94a00640 1 Processor -- start 2024-04-26T20:58:40.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.604+0000 7feb94a00640 1 -- start start 2024-04-26T20:58:40.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.605+0000 7feb94a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb9006f020 0x7feb90144da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:40.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.605+0000 7feb94a00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb90163ae0 0x7feb901fbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:40.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.605+0000 7feb94a00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb90164400 0x7feb90072f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:40.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.605+0000 7feb94a00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7feb9006f480 con 0x7feb9006f020 2024-04-26T20:58:40.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.605+0000 7feb94a00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb90164890 con 0x7feb90163ae0 2024-04-26T20:58:40.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.605+0000 7feb94a00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7feb90147c40 con 0x7feb90164400 2024-04-26T20:58:40.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.605+0000 7feb8de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb9006f020 0x7feb90144da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:40.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.605+0000 7feb8de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb9006f020 0x7feb90144da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53782/0 (socket says 172.21.15.28:53782) 2024-04-26T20:58:40.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.605+0000 7feb8de00640 1 -- 172.21.15.28:0/2436194320 learned_addr learned my addr 172.21.15.28:0/2436194320 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:40.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.605+0000 7feb8de00640 1 -- 172.21.15.28:0/2436194320 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb90164400 msgr2=0x7feb90072f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:40.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.605+0000 7feb8de00640 1 --2- 172.21.15.28:0/2436194320 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb90164400 0x7feb90072f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.605+0000 7feb8de00640 1 -- 172.21.15.28:0/2436194320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb90163ae0 msgr2=0x7feb901fbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:40.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.605+0000 7feb8de00640 1 --2- 172.21.15.28:0/2436194320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb90163ae0 0x7feb901fbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.605+0000 7feb8de00640 1 -- 172.21.15.28:0/2436194320 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb900cf210 con 0x7feb9006f020 2024-04-26T20:58:40.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.606+0000 7feb8de00640 1 --2- 172.21.15.28:0/2436194320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb9006f020 0x7feb90144da0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7feb8000ba50 tx=0x7feb8000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:40.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.606+0000 7feb86a00640 1 -- 172.21.15.28:0/2436194320 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80019070 con 0x7feb9006f020 2024-04-26T20:58:40.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.606+0000 7feb86a00640 1 -- 172.21.15.28:0/2436194320 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb8000cc00 con 0x7feb9006f020 2024-04-26T20:58:40.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.606+0000 7feb86a00640 1 -- 172.21.15.28:0/2436194320 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80014440 con 0x7feb9006f020 2024-04-26T20:58:40.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.606+0000 7feb94a00640 1 -- 172.21.15.28:0/2436194320 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb900cf4f0 con 0x7feb9006f020 2024-04-26T20:58:40.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.606+0000 7feb94a00640 1 -- 172.21.15.28:0/2436194320 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb901fc440 con 0x7feb9006f020 2024-04-26T20:58:40.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.607+0000 7feb94a00640 1 -- 172.21.15.28:0/2436194320 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb9006a400 con 0x7feb9006f020 2024-04-26T20:58:40.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.608+0000 7feb86a00640 1 -- 172.21.15.28:0/2436194320 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb8001c070 con 0x7feb9006f020 2024-04-26T20:58:40.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.609+0000 7feb86a00640 1 --2- 172.21.15.28:0/2436194320 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7feb6c06c8f0 0x7feb6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:40.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.609+0000 7feb86a00640 1 -- 172.21.15.28:0/2436194320 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7feb8008b630 con 0x7feb9006f020 2024-04-26T20:58:40.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.609+0000 7feb85400640 1 --2- 172.21.15.28:0/2436194320 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7feb6c06c8f0 0x7feb6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:40.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.610+0000 7feb85400640 1 --2- 172.21.15.28:0/2436194320 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7feb6c06c8f0 0x7feb6c06edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7feb78009670 tx=0x7feb78006bf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:40.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.614+0000 7feb86a00640 1 -- 172.21.15.28:0/2436194320 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feb80059d10 con 0x7feb9006f020 2024-04-26T20:58:40.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.819+0000 7feb94a00640 1 -- 172.21.15.28:0/2436194320 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7feb90161160 con 0x7feb9006f020 2024-04-26T20:58:40.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.820+0000 7feb86a00640 1 -- 172.21.15.28:0/2436194320 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) ==== 84+0+65 (secure 0 0 0) 0x7feb80059610 con 0x7feb9006f020 2024-04-26T20:58:40.821 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:40.822 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2024-04-26T20:58:40.822 INFO:teuthology.orchestra.run.smithi028.stdout: { 2024-04-26T20:58:40.822 INFO:teuthology.orchestra.run.smithi028.stdout: "poolnum": 1, 2024-04-26T20:58:40.822 INFO:teuthology.orchestra.run.smithi028.stdout: "poolname": ".mgr" 2024-04-26T20:58:40.822 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:40.822 INFO:teuthology.orchestra.run.smithi028.stdout:] 2024-04-26T20:58:40.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.823+0000 7feb94a00640 1 -- 172.21.15.28:0/2436194320 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7feb6c06c8f0 msgr2=0x7feb6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:40.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.823+0000 7feb94a00640 1 --2- 172.21.15.28:0/2436194320 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7feb6c06c8f0 0x7feb6c06edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7feb78009670 tx=0x7feb78006bf0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.823+0000 7feb94a00640 1 -- 172.21.15.28:0/2436194320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb9006f020 msgr2=0x7feb90144da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:40.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.823+0000 7feb94a00640 1 --2- 172.21.15.28:0/2436194320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb9006f020 0x7feb90144da0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7feb8000ba50 tx=0x7feb8000bf10 comp rx=0 tx=0).stop 2024-04-26T20:58:40.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.823+0000 7feb94a00640 1 -- 172.21.15.28:0/2436194320 shutdown_connections 2024-04-26T20:58:40.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.823+0000 7feb94a00640 1 --2- 172.21.15.28:0/2436194320 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7feb6c06c8f0 0x7feb6c06edb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.823+0000 7feb94a00640 1 --2- 172.21.15.28:0/2436194320 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7feb90164400 0x7feb90072f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.823+0000 7feb94a00640 1 --2- 172.21.15.28:0/2436194320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb90163ae0 0x7feb901fbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.823+0000 7feb94a00640 1 --2- 172.21.15.28:0/2436194320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feb9006f020 0x7feb90144da0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:40.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.823+0000 7feb94a00640 1 -- 172.21.15.28:0/2436194320 >> 172.21.15.28:0/2436194320 conn(0x7feb9006c770 msgr2=0x7feb9015a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:40.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.823+0000 7feb94a00640 1 -- 172.21.15.28:0/2436194320 shutdown_connections 2024-04-26T20:58:40.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:40.823+0000 7feb94a00640 1 -- 172.21.15.28:0/2436194320 wait complete. 2024-04-26T20:58:40.844 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T20:58:40.844 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T20:58:41.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.065+0000 7fd5a1400640 1 Processor -- start 2024-04-26T20:58:41.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.065+0000 7fd5a1400640 1 -- start start 2024-04-26T20:58:41.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.067+0000 7fd5a1400640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fd59c145790 0x7fd59c072790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:41.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.067+0000 7fd5a1400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd59c15ce20 0x7fd59c15d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:41.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.067+0000 7fd5a1400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd59c06f0a0 0x7fd59c06f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:41.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.067+0000 7fd5a1400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd59c06fa50 con 0x7fd59c15ce20 2024-04-26T20:58:41.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.067+0000 7fd5a1400640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fd59c0cefe0 con 0x7fd59c145790 2024-04-26T20:58:41.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.067+0000 7fd5a1400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fd59c0cf2c0 con 0x7fd59c06f0a0 2024-04-26T20:58:41.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.067+0000 7fd59b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd59c15ce20 0x7fd59c15d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:41.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.067+0000 7fd59b400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd59c15ce20 0x7fd59c15d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53784/0 (socket says 172.21.15.28:53784) 2024-04-26T20:58:41.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.067+0000 7fd59b400640 1 -- 172.21.15.28:0/2811021041 learned_addr learned my addr 172.21.15.28:0/2811021041 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:41.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.068+0000 7fd59b400640 1 -- 172.21.15.28:0/2811021041 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd59c06f0a0 msgr2=0x7fd59c06f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:41.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.068+0000 7fd59b400640 1 --2- 172.21.15.28:0/2811021041 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd59c06f0a0 0x7fd59c06f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:41.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.068+0000 7fd59b400640 1 -- 172.21.15.28:0/2811021041 >> v2:172.21.15.103:3300/0 conn(0x7fd59c145790 msgr2=0x7fd59c072790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:41.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.068+0000 7fd59b400640 1 --2- 172.21.15.28:0/2811021041 >> v2:172.21.15.103:3300/0 conn(0x7fd59c145790 0x7fd59c072790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:41.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.068+0000 7fd59b400640 1 -- 172.21.15.28:0/2811021041 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59c0cf5a0 con 0x7fd59c15ce20 2024-04-26T20:58:41.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.069+0000 7fd59b400640 1 --2- 172.21.15.28:0/2811021041 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd59c15ce20 0x7fd59c15d540 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd59c06d800 tx=0x7fd58802c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6837ba42a46d91de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:41.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.069+0000 7fd59aa00640 1 -- 172.21.15.28:0/2811021041 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd588033400 con 0x7fd59c15ce20 2024-04-26T20:58:41.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.069+0000 7fd59aa00640 1 -- 172.21.15.28:0/2811021041 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5880335a0 con 0x7fd59c15ce20 2024-04-26T20:58:41.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.069+0000 7fd59aa00640 1 -- 172.21.15.28:0/2811021041 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd588033970 con 0x7fd59c15ce20 2024-04-26T20:58:41.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.070+0000 7fd5a1400640 1 -- 172.21.15.28:0/2811021041 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd59c15ce20 msgr2=0x7fd59c15d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:41.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.070+0000 7fd5a1400640 1 --2- 172.21.15.28:0/2811021041 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd59c15ce20 0x7fd59c15d540 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd59c06d800 tx=0x7fd58802c8b0 comp rx=0 tx=0).stop 2024-04-26T20:58:41.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.070+0000 7fd5a1400640 1 -- 172.21.15.28:0/2811021041 shutdown_connections 2024-04-26T20:58:41.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.070+0000 7fd5a1400640 1 --2- 172.21.15.28:0/2811021041 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd59c06f0a0 0x7fd59c06f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:41.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.070+0000 7fd5a1400640 1 --2- 172.21.15.28:0/2811021041 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd59c15ce20 0x7fd59c15d540 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:41.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.070+0000 7fd5a1400640 1 --2- 172.21.15.28:0/2811021041 >> v2:172.21.15.103:3300/0 conn(0x7fd59c145790 0x7fd59c072790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:41.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.070+0000 7fd5a1400640 1 -- 172.21.15.28:0/2811021041 >> 172.21.15.28:0/2811021041 conn(0x7fd59c0b2820 msgr2=0x7fd59c0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:41.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.070+0000 7fd5a1400640 1 -- 172.21.15.28:0/2811021041 shutdown_connections 2024-04-26T20:58:41.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.070+0000 7fd5a1400640 1 -- 172.21.15.28:0/2811021041 wait complete. 2024-04-26T20:58:41.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.072+0000 7fd5a1400640 1 Processor -- start 2024-04-26T20:58:41.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.072+0000 7fd5a1400640 1 -- start start 2024-04-26T20:58:41.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.073+0000 7fd5a1400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd59c06f0a0 0x7fd59c0b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:41.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.073+0000 7fd5a1400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd59c145790 0x7fd59c0cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:41.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.073+0000 7fd5a1400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd59c15ce20 0x7fd59c0c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:41.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.073+0000 7fd5a1400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c15da80 con 0x7fd59c145790 2024-04-26T20:58:41.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.073+0000 7fd5a1400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c15d2d0 con 0x7fd59c06f0a0 2024-04-26T20:58:41.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.073+0000 7fd5a1400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fd59c15dc00 con 0x7fd59c15ce20 2024-04-26T20:58:41.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.073+0000 7fd59b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd59c145790 0x7fd59c0cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:41.075 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.073+0000 7fd59b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd59c145790 0x7fd59c0cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53800/0 (socket says 172.21.15.28:53800) 2024-04-26T20:58:41.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.073+0000 7fd59b400640 1 -- 172.21.15.28:0/3380663482 learned_addr learned my addr 172.21.15.28:0/3380663482 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:41.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.074+0000 7fd5a0a00640 1 --2- 172.21.15.28:0/3380663482 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd59c15ce20 0x7fd59c0c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:41.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.074+0000 7fd59be00640 1 --2- 172.21.15.28:0/3380663482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd59c06f0a0 0x7fd59c0b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:41.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.074+0000 7fd59b400640 1 -- 172.21.15.28:0/3380663482 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd59c15ce20 msgr2=0x7fd59c0c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:41.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.074+0000 7fd59b400640 1 --2- 172.21.15.28:0/3380663482 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd59c15ce20 0x7fd59c0c6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:41.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.074+0000 7fd59b400640 1 -- 172.21.15.28:0/3380663482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd59c06f0a0 msgr2=0x7fd59c0b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:41.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.074+0000 7fd59b400640 1 --2- 172.21.15.28:0/3380663482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd59c06f0a0 0x7fd59c0b5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:41.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.074+0000 7fd59b400640 1 -- 172.21.15.28:0/3380663482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59c0c6cf0 con 0x7fd59c145790 2024-04-26T20:58:41.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.074+0000 7fd59b400640 1 --2- 172.21.15.28:0/3380663482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd59c145790 0x7fd59c0cc590 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd58802cea0 tx=0x7fd588004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:41.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.075+0000 7fd593e00640 1 -- 172.21.15.28:0/3380663482 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd588009210 con 0x7fd59c145790 2024-04-26T20:58:41.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.075+0000 7fd593e00640 1 -- 172.21.15.28:0/3380663482 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd58803d650 con 0x7fd59c145790 2024-04-26T20:58:41.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.075+0000 7fd593e00640 1 -- 172.21.15.28:0/3380663482 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58803da20 con 0x7fd59c145790 2024-04-26T20:58:41.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.075+0000 7fd5a1400640 1 -- 172.21.15.28:0/3380663482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd59c0c6fd0 con 0x7fd59c145790 2024-04-26T20:58:41.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.075+0000 7fd5a1400640 1 -- 172.21.15.28:0/3380663482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd59c0c7590 con 0x7fd59c145790 2024-04-26T20:58:41.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.076+0000 7fd5a1400640 1 -- 172.21.15.28:0/3380663482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd59c06a3d0 con 0x7fd59c145790 2024-04-26T20:58:41.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.077+0000 7fd593e00640 1 -- 172.21.15.28:0/3380663482 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd588046020 con 0x7fd59c145790 2024-04-26T20:58:41.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.078+0000 7fd593e00640 1 --2- 172.21.15.28:0/3380663482 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd57c06c8f0 0x7fd57c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:41.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.079+0000 7fd593e00640 1 -- 172.21.15.28:0/3380663482 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7fd5880abd60 con 0x7fd59c145790 2024-04-26T20:58:41.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.079+0000 7fd59be00640 1 --2- 172.21.15.28:0/3380663482 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd57c06c8f0 0x7fd57c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:41.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.079+0000 7fd59be00640 1 --2- 172.21.15.28:0/3380663482 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd57c06c8f0 0x7fd57c06edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00b810 tx=0x7fd58c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:41.084 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.083+0000 7fd593e00640 1 -- 172.21.15.28:0/3380663482 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd58804d020 con 0x7fd59c145790 2024-04-26T20:58:41.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:41.290+0000 7fd5a1400640 1 -- 172.21.15.28:0/3380663482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd59c15d730 con 0x7fd59c145790 2024-04-26T20:58:42.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.014+0000 7fd593e00640 1 -- 172.21.15.28:0/3380663482 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) ==== 138+0+0 (secure 0 0 0) 0x7fd58807a3b0 con 0x7fd59c145790 2024-04-26T20:58:42.016 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' created 2024-04-26T20:58:42.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.019+0000 7fd5a1400640 1 -- 172.21.15.28:0/3380663482 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd57c06c8f0 msgr2=0x7fd57c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:42.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.019+0000 7fd5a1400640 1 --2- 172.21.15.28:0/3380663482 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd57c06c8f0 0x7fd57c06edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00b810 tx=0x7fd58c008040 comp rx=0 tx=0).stop 2024-04-26T20:58:42.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.019+0000 7fd5a1400640 1 -- 172.21.15.28:0/3380663482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd59c145790 msgr2=0x7fd59c0cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:42.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.019+0000 7fd5a1400640 1 --2- 172.21.15.28:0/3380663482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd59c145790 0x7fd59c0cc590 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd58802cea0 tx=0x7fd588004a40 comp rx=0 tx=0).stop 2024-04-26T20:58:42.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.019+0000 7fd5a1400640 1 -- 172.21.15.28:0/3380663482 shutdown_connections 2024-04-26T20:58:42.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.019+0000 7fd5a1400640 1 --2- 172.21.15.28:0/3380663482 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd57c06c8f0 0x7fd57c06edb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:42.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.019+0000 7fd5a1400640 1 --2- 172.21.15.28:0/3380663482 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd59c15ce20 0x7fd59c0c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:42.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.019+0000 7fd5a1400640 1 --2- 172.21.15.28:0/3380663482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd59c145790 0x7fd59c0cc590 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:42.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.019+0000 7fd5a1400640 1 --2- 172.21.15.28:0/3380663482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd59c06f0a0 0x7fd59c0b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:42.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.019+0000 7fd5a1400640 1 -- 172.21.15.28:0/3380663482 >> 172.21.15.28:0/3380663482 conn(0x7fd59c0b2820 msgr2=0x7fd59c0cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:42.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.019+0000 7fd5a1400640 1 -- 172.21.15.28:0/3380663482 shutdown_connections 2024-04-26T20:58:42.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.019+0000 7fd5a1400640 1 -- 172.21.15.28:0/3380663482 wait complete. 2024-04-26T20:58:42.050 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T20:58:42.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.257+0000 7fb27e200640 1 Processor -- start 2024-04-26T20:58:42.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.257+0000 7fb27e200640 1 -- start start 2024-04-26T20:58:42.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.259+0000 7fb27e200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb278073f50 0x7fb2780ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:42.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.259+0000 7fb27e200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2780ce8d0 0x7fb27806e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:42.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.259+0000 7fb27e200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb278074400 con 0x7fb27806fc10 2024-04-26T20:58:42.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.259+0000 7fb27e200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb2780ced00 con 0x7fb278073f50 2024-04-26T20:58:42.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.259+0000 7fb27e200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb27815e310 con 0x7fb2780ce8d0 2024-04-26T20:58:42.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.259+0000 7fb276a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb278073f50 0x7fb2780ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:42.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.259+0000 7fb276a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb278073f50 0x7fb2780ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53804/0 (socket says 172.21.15.28:53804) 2024-04-26T20:58:42.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.259+0000 7fb276a00640 1 -- 172.21.15.28:0/933345838 learned_addr learned my addr 172.21.15.28:0/933345838 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:42.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.259+0000 7fb277400640 1 --1- 172.21.15.28:0/933345838 >> v1:172.21.15.28:6789/0 conn(0x7fb27806fc10 0x7fb278073580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52908/0 (socket says 172.21.15.28:52908) 2024-04-26T20:58:42.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.260+0000 7fb277e00640 1 --2- 172.21.15.28:0/933345838 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2780ce8d0 0x7fb27806e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:42.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.260+0000 7fb276a00640 1 -- 172.21.15.28:0/933345838 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2780ce8d0 msgr2=0x7fb27806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:42.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.260+0000 7fb276a00640 1 --2- 172.21.15.28:0/933345838 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2780ce8d0 0x7fb27806e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:42.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.260+0000 7fb276a00640 1 -- 172.21.15.28:0/933345838 >> v1:172.21.15.28:6789/0 conn(0x7fb27806fc10 legacy=0x7fb278073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:42.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.260+0000 7fb276a00640 1 -- 172.21.15.28:0/933345838 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb278072aa0 con 0x7fb278073f50 2024-04-26T20:58:42.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.260+0000 7fb276000640 1 -- 172.21.15.28:0/933345838 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1359029194 0 0) 0x7fb278074400 con 0x7fb27806fc10 2024-04-26T20:58:42.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.261+0000 7fb276a00640 1 --2- 172.21.15.28:0/933345838 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb278073f50 0x7fb2780ce390 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb264009870 tx=0x7fb26402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef1ca576c7214d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:42.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.261+0000 7fb276000640 1 -- 172.21.15.28:0/933345838 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb264037070 con 0x7fb278073f50 2024-04-26T20:58:42.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.261+0000 7fb276000640 1 -- 172.21.15.28:0/933345838 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb264032440 con 0x7fb278073f50 2024-04-26T20:58:42.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.261+0000 7fb276000640 1 -- 172.21.15.28:0/933345838 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb264032810 con 0x7fb278073f50 2024-04-26T20:58:42.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.262+0000 7fb27e200640 1 -- 172.21.15.28:0/933345838 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb278073f50 msgr2=0x7fb2780ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:42.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.262+0000 7fb27e200640 1 --2- 172.21.15.28:0/933345838 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb278073f50 0x7fb2780ce390 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb264009870 tx=0x7fb26402a1c0 comp rx=0 tx=0).stop 2024-04-26T20:58:42.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.262+0000 7fb27e200640 1 -- 172.21.15.28:0/933345838 shutdown_connections 2024-04-26T20:58:42.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.262+0000 7fb27e200640 1 --2- 172.21.15.28:0/933345838 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb2780ce8d0 0x7fb27806e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:42.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.262+0000 7fb27e200640 1 --2- 172.21.15.28:0/933345838 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb278073f50 0x7fb2780ce390 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:42.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.262+0000 7fb27e200640 1 -- 172.21.15.28:0/933345838 >> 172.21.15.28:0/933345838 conn(0x7fb27806c770 msgr2=0x7fb27806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:42.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.262+0000 7fb27e200640 1 -- 172.21.15.28:0/933345838 shutdown_connections 2024-04-26T20:58:42.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.262+0000 7fb27e200640 1 -- 172.21.15.28:0/933345838 wait complete. 2024-04-26T20:58:42.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.264+0000 7fb27e200640 1 Processor -- start 2024-04-26T20:58:42.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.264+0000 7fb27e200640 1 -- start start 2024-04-26T20:58:42.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.265+0000 7fb27e200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb27806fc10 0x7fb27814aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:42.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.265+0000 7fb27e200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb278073f50 0x7fb27813b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:42.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.265+0000 7fb27e200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2780ce8d0 0x7fb27814da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:42.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.265+0000 7fb27e200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb27815dfa0 con 0x7fb2780ce8d0 2024-04-26T20:58:42.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.265+0000 7fb27e200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb278073060 con 0x7fb27806fc10 2024-04-26T20:58:42.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.265+0000 7fb27e200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb2780d07d0 con 0x7fb278073f50 2024-04-26T20:58:42.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.265+0000 7fb277e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2780ce8d0 0x7fb27814da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:42.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.265+0000 7fb277e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2780ce8d0 0x7fb27814da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53816/0 (socket says 172.21.15.28:53816) 2024-04-26T20:58:42.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.265+0000 7fb277e00640 1 -- 172.21.15.28:0/2427027275 learned_addr learned my addr 172.21.15.28:0/2427027275 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:42.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.266+0000 7fb276a00640 1 --2- 172.21.15.28:0/2427027275 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb278073f50 0x7fb27813b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:42.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.266+0000 7fb277400640 1 --2- 172.21.15.28:0/2427027275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb27806fc10 0x7fb27814aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:42.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.266+0000 7fb277e00640 1 -- 172.21.15.28:0/2427027275 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb278073f50 msgr2=0x7fb27813b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:42.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.266+0000 7fb277e00640 1 --2- 172.21.15.28:0/2427027275 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb278073f50 0x7fb27813b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:42.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.266+0000 7fb277e00640 1 -- 172.21.15.28:0/2427027275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb27806fc10 msgr2=0x7fb27814aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:42.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.266+0000 7fb277e00640 1 --2- 172.21.15.28:0/2427027275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb27806fc10 0x7fb27814aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:42.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.266+0000 7fb277e00640 1 -- 172.21.15.28:0/2427027275 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27814afd0 con 0x7fb2780ce8d0 2024-04-26T20:58:42.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.266+0000 7fb277e00640 1 --2- 172.21.15.28:0/2427027275 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2780ce8d0 0x7fb27814da80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb27000d5d0 tx=0x7fb27000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:42.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.267+0000 7fb26f400640 1 -- 172.21.15.28:0/2427027275 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2700125f0 con 0x7fb2780ce8d0 2024-04-26T20:58:42.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.267+0000 7fb26f400640 1 -- 172.21.15.28:0/2427027275 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb270019070 con 0x7fb2780ce8d0 2024-04-26T20:58:42.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.267+0000 7fb26f400640 1 -- 172.21.15.28:0/2427027275 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2700129f0 con 0x7fb2780ce8d0 2024-04-26T20:58:42.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.267+0000 7fb27e200640 1 -- 172.21.15.28:0/2427027275 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb278146040 con 0x7fb2780ce8d0 2024-04-26T20:58:42.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.267+0000 7fb27e200640 1 -- 172.21.15.28:0/2427027275 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2781457a0 con 0x7fb2780ce8d0 2024-04-26T20:58:42.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.268+0000 7fb27e200640 1 -- 172.21.15.28:0/2427027275 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb27806a360 con 0x7fb2780ce8d0 2024-04-26T20:58:42.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.269+0000 7fb26f400640 1 -- 172.21.15.28:0/2427027275 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb270012b90 con 0x7fb2780ce8d0 2024-04-26T20:58:42.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.270+0000 7fb26f400640 1 --2- 172.21.15.28:0/2427027275 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb25806c920 0x7fb25806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:42.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.270+0000 7fb26f400640 1 -- 172.21.15.28:0/2427027275 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5227+0+0 (secure 0 0 0) 0x7fb27008bdc0 con 0x7fb2780ce8d0 2024-04-26T20:58:42.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.271+0000 7fb277400640 1 --2- 172.21.15.28:0/2427027275 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb25806c920 0x7fb25806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:42.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.271+0000 7fb277400640 1 --2- 172.21.15.28:0/2427027275 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb25806c920 0x7fb25806ede0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb2680012e0 tx=0x7fb268005eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:42.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.278+0000 7fb26f400640 1 -- 172.21.15.28:0/2427027275 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb27005a340 con 0x7fb2780ce8d0 2024-04-26T20:58:42.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:42.482+0000 7fb27e200640 1 -- 172.21.15.28:0/2427027275 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb2780ce6b0 con 0x7fb2780ce8d0 2024-04-26T20:58:43.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.030+0000 7fb26f400640 1 -- 172.21.15.28:0/2427027275 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) ==== 170+0+0 (secure 0 0 0) 0x7fb270059c40 con 0x7fb2780ce8d0 2024-04-26T20:58:43.031 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' created 2024-04-26T20:58:43.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.033+0000 7fb27e200640 1 -- 172.21.15.28:0/2427027275 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb25806c920 msgr2=0x7fb25806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:43.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.033+0000 7fb27e200640 1 --2- 172.21.15.28:0/2427027275 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb25806c920 0x7fb25806ede0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb2680012e0 tx=0x7fb268005eb0 comp rx=0 tx=0).stop 2024-04-26T20:58:43.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.033+0000 7fb27e200640 1 -- 172.21.15.28:0/2427027275 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2780ce8d0 msgr2=0x7fb27814da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:43.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.033+0000 7fb27e200640 1 --2- 172.21.15.28:0/2427027275 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2780ce8d0 0x7fb27814da80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb27000d5d0 tx=0x7fb27000da90 comp rx=0 tx=0).stop 2024-04-26T20:58:43.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.034+0000 7fb27e200640 1 -- 172.21.15.28:0/2427027275 shutdown_connections 2024-04-26T20:58:43.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.034+0000 7fb27e200640 1 --2- 172.21.15.28:0/2427027275 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb25806c920 0x7fb25806ede0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:43.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.034+0000 7fb27e200640 1 --2- 172.21.15.28:0/2427027275 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2780ce8d0 0x7fb27814da80 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:43.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.034+0000 7fb27e200640 1 --2- 172.21.15.28:0/2427027275 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb278073f50 0x7fb27813b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:43.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.034+0000 7fb27e200640 1 --2- 172.21.15.28:0/2427027275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb27806fc10 0x7fb27814aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:43.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.034+0000 7fb27e200640 1 -- 172.21.15.28:0/2427027275 >> 172.21.15.28:0/2427027275 conn(0x7fb27806c770 msgr2=0x7fb2780b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:43.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.035+0000 7fb27e200640 1 -- 172.21.15.28:0/2427027275 shutdown_connections 2024-04-26T20:58:43.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.035+0000 7fb27e200640 1 -- 172.21.15.28:0/2427027275 wait complete. 2024-04-26T20:58:43.079 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T20:58:43.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.260+0000 7f4e63e00640 1 Processor -- start 2024-04-26T20:58:43.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.261+0000 7f4e63e00640 1 -- start start 2024-04-26T20:58:43.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.262+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4e640d1de0 0x7f4e640d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:43.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.262+0000 7f4e63e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f4e6415f360 0x7f4e640d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:43.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.262+0000 7f4e63e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4e640bac00 0x7f4e640d39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:43.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.262+0000 7f4e63e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f4e640d2700 con 0x7f4e640bac00 2024-04-26T20:58:43.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.262+0000 7f4e63e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f4e640ba350 con 0x7f4e6415f360 2024-04-26T20:58:43.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.262+0000 7f4e63e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f4e640b9ab0 con 0x7f4e640d1de0 2024-04-26T20:58:43.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.263+0000 7f4e63400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4e640bac00 0x7f4e640d39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:43.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.263+0000 7f4e63400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4e640bac00 0x7f4e640d39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53822/0 (socket says 172.21.15.28:53822) 2024-04-26T20:58:43.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.263+0000 7f4e63400640 1 -- 172.21.15.28:0/601088325 learned_addr learned my addr 172.21.15.28:0/601088325 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:43.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.263+0000 7f4e62a00640 1 --2- 172.21.15.28:0/601088325 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4e640d1de0 0x7f4e640d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:43.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.263+0000 7f4e62000640 1 --2- 172.21.15.28:0/601088325 >> v2:172.21.15.103:3300/0 conn(0x7f4e6415f360 0x7f4e640d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:43.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.263+0000 7f4e63400640 1 -- 172.21.15.28:0/601088325 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4e640d1de0 msgr2=0x7f4e640d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:43.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.263+0000 7f4e63400640 1 --2- 172.21.15.28:0/601088325 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4e640d1de0 0x7f4e640d21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:43.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.263+0000 7f4e63400640 1 -- 172.21.15.28:0/601088325 >> v2:172.21.15.103:3300/0 conn(0x7f4e6415f360 msgr2=0x7f4e640d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:43.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.263+0000 7f4e63400640 1 --2- 172.21.15.28:0/601088325 >> v2:172.21.15.103:3300/0 conn(0x7f4e6415f360 0x7f4e640d3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:43.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.263+0000 7f4e63400640 1 -- 172.21.15.28:0/601088325 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e640b9d90 con 0x7f4e640bac00 2024-04-26T20:58:43.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.264+0000 7f4e63400640 1 --2- 172.21.15.28:0/601088325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e640bac00 0x7f4e640d39d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4e6408d210 tx=0x7f4e5c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d14a9a11aca729 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:43.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.265+0000 7f4e61600640 1 -- 172.21.15.28:0/601088325 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c037070 con 0x7f4e640bac00 2024-04-26T20:58:43.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.265+0000 7f4e61600640 1 -- 172.21.15.28:0/601088325 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e5c032440 con 0x7f4e640bac00 2024-04-26T20:58:43.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.265+0000 7f4e61600640 1 -- 172.21.15.28:0/601088325 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c032810 con 0x7f4e640bac00 2024-04-26T20:58:43.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.265+0000 7f4e63e00640 1 -- 172.21.15.28:0/601088325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e640bac00 msgr2=0x7f4e640d39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:43.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.265+0000 7f4e63e00640 1 --2- 172.21.15.28:0/601088325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e640bac00 0x7f4e640d39d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4e6408d210 tx=0x7f4e5c02a1e0 comp rx=0 tx=0).stop 2024-04-26T20:58:43.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.265+0000 7f4e63e00640 1 -- 172.21.15.28:0/601088325 shutdown_connections 2024-04-26T20:58:43.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.265+0000 7f4e63e00640 1 --2- 172.21.15.28:0/601088325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e640bac00 0x7f4e640d39d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:43.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.265+0000 7f4e63e00640 1 --2- 172.21.15.28:0/601088325 >> v2:172.21.15.103:3300/0 conn(0x7f4e6415f360 0x7f4e640d3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:43.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.265+0000 7f4e63e00640 1 --2- 172.21.15.28:0/601088325 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4e640d1de0 0x7f4e640d21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:43.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.265+0000 7f4e63e00640 1 -- 172.21.15.28:0/601088325 >> 172.21.15.28:0/601088325 conn(0x7f4e6406c770 msgr2=0x7f4e6406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:43.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.266+0000 7f4e63e00640 1 -- 172.21.15.28:0/601088325 shutdown_connections 2024-04-26T20:58:43.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.266+0000 7f4e63e00640 1 -- 172.21.15.28:0/601088325 wait complete. 2024-04-26T20:58:43.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.267+0000 7f4e63e00640 1 Processor -- start 2024-04-26T20:58:43.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.267+0000 7f4e63e00640 1 -- start start 2024-04-26T20:58:43.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.268+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e640bac00 0x7f4e640abb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:43.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.268+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e640d1de0 0x7f4e641f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:43.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.268+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4e6415f360 0x7f4e641f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:43.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.268+0000 7f4e63e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4e640d3f10 con 0x7f4e640bac00 2024-04-26T20:58:43.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.268+0000 7f4e63e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e640d2f70 con 0x7f4e640d1de0 2024-04-26T20:58:43.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.268+0000 7f4e63e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f4e640d4090 con 0x7f4e6415f360 2024-04-26T20:58:43.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.269+0000 7f4e62a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e640bac00 0x7f4e640abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:43.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.269+0000 7f4e62a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e640bac00 0x7f4e640abb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53828/0 (socket says 172.21.15.28:53828) 2024-04-26T20:58:43.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.269+0000 7f4e62a00640 1 -- 172.21.15.28:0/1934503387 learned_addr learned my addr 172.21.15.28:0/1934503387 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:43.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.269+0000 7f4e62000640 1 --2- 172.21.15.28:0/1934503387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e640d1de0 0x7f4e641f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:43.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.269+0000 7f4e62a00640 1 -- 172.21.15.28:0/1934503387 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4e6415f360 msgr2=0x7f4e641f5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:43.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.269+0000 7f4e62a00640 1 --2- 172.21.15.28:0/1934503387 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4e6415f360 0x7f4e641f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:43.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.269+0000 7f4e62a00640 1 -- 172.21.15.28:0/1934503387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e640d1de0 msgr2=0x7f4e641f4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:43.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.269+0000 7f4e62a00640 1 --2- 172.21.15.28:0/1934503387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e640d1de0 0x7f4e641f4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:43.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.269+0000 7f4e62a00640 1 -- 172.21.15.28:0/1934503387 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e641f58c0 con 0x7f4e640bac00 2024-04-26T20:58:43.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.270+0000 7f4e62a00640 1 --2- 172.21.15.28:0/1934503387 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e640bac00 0x7f4e640abb10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400d800 tx=0x7f4e5400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:43.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.270+0000 7f4e5aa00640 1 -- 172.21.15.28:0/1934503387 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5400b6c0 con 0x7f4e640bac00 2024-04-26T20:58:43.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.270+0000 7f4e5aa00640 1 -- 172.21.15.28:0/1934503387 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e54019070 con 0x7f4e640bac00 2024-04-26T20:58:43.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.270+0000 7f4e5aa00640 1 -- 172.21.15.28:0/1934503387 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5400ba80 con 0x7f4e640bac00 2024-04-26T20:58:43.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.270+0000 7f4e63e00640 1 -- 172.21.15.28:0/1934503387 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e641f5b60 con 0x7f4e640bac00 2024-04-26T20:58:43.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.270+0000 7f4e63e00640 1 -- 172.21.15.28:0/1934503387 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e640ce2d0 con 0x7f4e640bac00 2024-04-26T20:58:43.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.271+0000 7f4e63e00640 1 -- 172.21.15.28:0/1934503387 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e6406a250 con 0x7f4e640bac00 2024-04-26T20:58:43.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.273+0000 7f4e5aa00640 1 -- 172.21.15.28:0/1934503387 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e5400bc20 con 0x7f4e640bac00 2024-04-26T20:58:43.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.273+0000 7f4e5aa00640 1 --2- 172.21.15.28:0/1934503387 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4e4006c8f0 0x7f4e4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:43.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.274+0000 7f4e5aa00640 1 -- 172.21.15.28:0/1934503387 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5609+0+0 (secure 0 0 0) 0x7f4e5408bfe0 con 0x7f4e640bac00 2024-04-26T20:58:43.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.274+0000 7f4e62000640 1 --2- 172.21.15.28:0/1934503387 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4e4006c8f0 0x7f4e4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:43.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.275+0000 7f4e62000640 1 --2- 172.21.15.28:0/1934503387 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4e4006c8f0 0x7f4e4006edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4e640b7c10 tx=0x7f4e4c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:43.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.286+0000 7f4e5aa00640 1 -- 172.21.15.28:0/1934503387 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4e641f58c0 con 0x7f4e640bac00 2024-04-26T20:58:43.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:43.532+0000 7f4e63e00640 1 -- 172.21.15.28:0/1934503387 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4e640d3630 con 0x7f4e640bac00 2024-04-26T20:58:44.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.034+0000 7f4e5aa00640 1 -- 172.21.15.28:0/1934503387 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v22) ==== 172+0+0 (secure 0 0 0) 0x7f4e54011340 con 0x7f4e640bac00 2024-04-26T20:58:44.036 INFO:teuthology.orchestra.run.smithi028.stderr:new fs with metadata pool 6 and data pool 7 2024-04-26T20:58:44.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.039+0000 7f4e63e00640 1 -- 172.21.15.28:0/1934503387 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4e4006c8f0 msgr2=0x7f4e4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:44.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.039+0000 7f4e63e00640 1 --2- 172.21.15.28:0/1934503387 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4e4006c8f0 0x7f4e4006edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4e640b7c10 tx=0x7f4e4c00b040 comp rx=0 tx=0).stop 2024-04-26T20:58:44.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.039+0000 7f4e63e00640 1 -- 172.21.15.28:0/1934503387 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e640bac00 msgr2=0x7f4e640abb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:44.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.039+0000 7f4e63e00640 1 --2- 172.21.15.28:0/1934503387 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e640bac00 0x7f4e640abb10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400d800 tx=0x7f4e5400dcc0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.040+0000 7f4e63e00640 1 -- 172.21.15.28:0/1934503387 shutdown_connections 2024-04-26T20:58:44.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.040+0000 7f4e63e00640 1 --2- 172.21.15.28:0/1934503387 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4e4006c8f0 0x7f4e4006edb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.040+0000 7f4e63e00640 1 --2- 172.21.15.28:0/1934503387 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4e6415f360 0x7f4e641f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.040+0000 7f4e63e00640 1 --2- 172.21.15.28:0/1934503387 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e640d1de0 0x7f4e641f4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.040+0000 7f4e63e00640 1 --2- 172.21.15.28:0/1934503387 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4e640bac00 0x7f4e640abb10 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.040+0000 7f4e63e00640 1 -- 172.21.15.28:0/1934503387 >> 172.21.15.28:0/1934503387 conn(0x7f4e6406c770 msgr2=0x7f4e640b7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:44.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.040+0000 7f4e63e00640 1 -- 172.21.15.28:0/1934503387 shutdown_connections 2024-04-26T20:58:44.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.040+0000 7f4e63e00640 1 -- 172.21.15.28:0/1934503387 wait complete. 2024-04-26T20:58:44.083 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:58:44.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.259+0000 7fae3ae00640 1 Processor -- start 2024-04-26T20:58:44.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.260+0000 7fae3ae00640 1 -- start start 2024-04-26T20:58:44.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.261+0000 7fae3ae00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fae3415f350 0x7fae340d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:44.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.261+0000 7fae3ae00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fae340cf8a0 0x7fae340d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:44.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.261+0000 7fae3ae00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fae341638c0 0x7fae3406c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:44.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.261+0000 7fae3ae00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fae340cffe0 con 0x7fae340cf8a0 2024-04-26T20:58:44.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.261+0000 7fae3ae00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fae340d0240 con 0x7fae3415f350 2024-04-26T20:58:44.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.261+0000 7fae3ae00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fae340d0520 con 0x7fae341638c0 2024-04-26T20:58:44.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.261+0000 7fae33400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fae340cf8a0 0x7fae340d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:44.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.261+0000 7fae33400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fae340cf8a0 0x7fae340d1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53842/0 (socket says 172.21.15.28:53842) 2024-04-26T20:58:44.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.261+0000 7fae33400640 1 -- 172.21.15.28:0/1198830077 learned_addr learned my addr 172.21.15.28:0/1198830077 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:44.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.262+0000 7fae33e00640 1 --2- 172.21.15.28:0/1198830077 >> v2:172.21.15.103:3300/0 conn(0x7fae3415f350 0x7fae340d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:44.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.262+0000 7fae33400640 1 -- 172.21.15.28:0/1198830077 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fae341638c0 msgr2=0x7fae3406c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:44.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.262+0000 7fae38e00640 1 --2- 172.21.15.28:0/1198830077 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fae341638c0 0x7fae3406c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:44.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.262+0000 7fae33400640 1 --2- 172.21.15.28:0/1198830077 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fae341638c0 0x7fae3406c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.262+0000 7fae33400640 1 -- 172.21.15.28:0/1198830077 >> v2:172.21.15.103:3300/0 conn(0x7fae3415f350 msgr2=0x7fae340d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:44.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.262+0000 7fae33400640 1 --2- 172.21.15.28:0/1198830077 >> v2:172.21.15.103:3300/0 conn(0x7fae3415f350 0x7fae340d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.262+0000 7fae33400640 1 -- 172.21.15.28:0/1198830077 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3415f730 con 0x7fae340cf8a0 2024-04-26T20:58:44.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.262+0000 7fae33e00640 1 --2- 172.21.15.28:0/1198830077 >> v2:172.21.15.103:3300/0 conn(0x7fae3415f350 0x7fae340d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:44.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.263+0000 7fae33400640 1 --2- 172.21.15.28:0/1198830077 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae340cf8a0 0x7fae340d1d90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fae3408daf0 tx=0x7fae1c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282ebd63086d1422 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:44.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.263+0000 7fae32a00640 1 -- 172.21.15.28:0/1198830077 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c037070 con 0x7fae340cf8a0 2024-04-26T20:58:44.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.263+0000 7fae32a00640 1 -- 172.21.15.28:0/1198830077 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae1c02ae00 con 0x7fae340cf8a0 2024-04-26T20:58:44.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.263+0000 7fae32a00640 1 -- 172.21.15.28:0/1198830077 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c032660 con 0x7fae340cf8a0 2024-04-26T20:58:44.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.264+0000 7fae3ae00640 1 -- 172.21.15.28:0/1198830077 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae340cf8a0 msgr2=0x7fae340d1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:44.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.264+0000 7fae3ae00640 1 --2- 172.21.15.28:0/1198830077 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae340cf8a0 0x7fae340d1d90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fae3408daf0 tx=0x7fae1c02a0b0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.264+0000 7fae3ae00640 1 -- 172.21.15.28:0/1198830077 shutdown_connections 2024-04-26T20:58:44.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.264+0000 7fae3ae00640 1 --2- 172.21.15.28:0/1198830077 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fae341638c0 0x7fae3406c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.264+0000 7fae3ae00640 1 --2- 172.21.15.28:0/1198830077 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae340cf8a0 0x7fae340d1d90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.264+0000 7fae3ae00640 1 --2- 172.21.15.28:0/1198830077 >> v2:172.21.15.103:3300/0 conn(0x7fae3415f350 0x7fae340d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.264+0000 7fae3ae00640 1 -- 172.21.15.28:0/1198830077 >> 172.21.15.28:0/1198830077 conn(0x7fae340b6d10 msgr2=0x7fae340b7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:44.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.264+0000 7fae3ae00640 1 -- 172.21.15.28:0/1198830077 shutdown_connections 2024-04-26T20:58:44.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.264+0000 7fae3ae00640 1 -- 172.21.15.28:0/1198830077 wait complete. 2024-04-26T20:58:44.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.266+0000 7fae3ae00640 1 Processor -- start 2024-04-26T20:58:44.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.266+0000 7fae3ae00640 1 -- start start 2024-04-26T20:58:44.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.267+0000 7fae3ae00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fae340cf8a0 0x7fae34162160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:44.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.267+0000 7fae3ae00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae3415f350 0x7fae340cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:44.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.267+0000 7fae3ae00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae341638c0 0x7fae340cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:44.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.267+0000 7fae3ae00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fae3406c890 con 0x7fae341638c0 2024-04-26T20:58:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.267+0000 7fae3ae00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fae340d0800 con 0x7fae3415f350 2024-04-26T20:58:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.267+0000 7fae3ae00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fae3406ca10 con 0x7fae340cf8a0 2024-04-26T20:58:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.267+0000 7fae38e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae341638c0 0x7fae340cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.268+0000 7fae38e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae341638c0 0x7fae340cc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53846/0 (socket says 172.21.15.28:53846) 2024-04-26T20:58:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.268+0000 7fae38e00640 1 -- 172.21.15.28:0/1258579880 learned_addr learned my addr 172.21.15.28:0/1258579880 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.268+0000 7fae33e00640 1 --2- 172.21.15.28:0/1258579880 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fae340cf8a0 0x7fae34162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.268+0000 7fae33400640 1 --2- 172.21.15.28:0/1258579880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae3415f350 0x7fae340cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.268+0000 7fae38e00640 1 -- 172.21.15.28:0/1258579880 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fae340cf8a0 msgr2=0x7fae34162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.268+0000 7fae38e00640 1 --2- 172.21.15.28:0/1258579880 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fae340cf8a0 0x7fae34162160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.268+0000 7fae38e00640 1 -- 172.21.15.28:0/1258579880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae3415f350 msgr2=0x7fae340cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.268+0000 7fae38e00640 1 --2- 172.21.15.28:0/1258579880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae3415f350 0x7fae340cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.268+0000 7fae38e00640 1 -- 172.21.15.28:0/1258579880 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3415ba10 con 0x7fae341638c0 2024-04-26T20:58:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.268+0000 7fae38e00640 1 --2- 172.21.15.28:0/1258579880 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae341638c0 0x7fae340cc650 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00c550 tx=0x7fae2c00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.269+0000 7fae2be00640 1 -- 172.21.15.28:0/1258579880 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c015070 con 0x7fae341638c0 2024-04-26T20:58:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.269+0000 7fae2be00640 1 -- 172.21.15.28:0/1258579880 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae2c004220 con 0x7fae341638c0 2024-04-26T20:58:44.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.269+0000 7fae2be00640 1 -- 172.21.15.28:0/1258579880 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2c003610 con 0x7fae341638c0 2024-04-26T20:58:44.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.269+0000 7fae3ae00640 1 -- 172.21.15.28:0/1258579880 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae340cdea0 con 0x7fae341638c0 2024-04-26T20:58:44.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.269+0000 7fae3ae00640 1 -- 172.21.15.28:0/1258579880 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae340ce3f0 con 0x7fae341638c0 2024-04-26T20:58:44.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.270+0000 7fae3ae00640 1 -- 172.21.15.28:0/1258579880 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae3406a400 con 0x7fae341638c0 2024-04-26T20:58:44.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.271+0000 7fae2be00640 1 -- 172.21.15.28:0/1258579880 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fae2c0037b0 con 0x7fae341638c0 2024-04-26T20:58:44.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.272+0000 7fae2be00640 1 --2- 172.21.15.28:0/1258579880 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fae1006c8f0 0x7fae1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.272+0000 7fae2be00640 1 -- 172.21.15.28:0/1258579880 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fae2c08b940 con 0x7fae341638c0 2024-04-26T20:58:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.273+0000 7fae33e00640 1 --2- 172.21.15.28:0/1258579880 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fae1006c8f0 0x7fae1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.273+0000 7fae33e00640 1 --2- 172.21.15.28:0/1258579880 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fae1006c8f0 0x7fae1006edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fae24007130 tx=0x7fae240079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:44.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.277+0000 7fae2be00640 1 -- 172.21.15.28:0/1258579880 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fae2c059cd0 con 0x7fae341638c0 2024-04-26T20:58:44.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.477+0000 7fae3ae00640 1 -- 172.21.15.28:0/1258579880 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae340d2d50 con 0x7fae341638c0 2024-04-26T20:58:44.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.479+0000 7fae2be00640 1 -- 172.21.15.28:0/1258579880 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16602 (secure 0 0 0) 0x7fae2c0595d0 con 0x7fae341638c0 2024-04-26T20:58:44.481 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:44.481 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":44,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:58:43.534414+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:58:41.306286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T20:58:42.497340+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:58:44.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.482+0000 7fae3ae00640 1 -- 172.21.15.28:0/1258579880 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fae1006c8f0 msgr2=0x7fae1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:44.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.482+0000 7fae3ae00640 1 --2- 172.21.15.28:0/1258579880 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fae1006c8f0 0x7fae1006edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fae24007130 tx=0x7fae240079c0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.482+0000 7fae3ae00640 1 -- 172.21.15.28:0/1258579880 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae341638c0 msgr2=0x7fae340cc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:44.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.482+0000 7fae3ae00640 1 --2- 172.21.15.28:0/1258579880 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae341638c0 0x7fae340cc650 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00c550 tx=0x7fae2c00ca10 comp rx=0 tx=0).stop 2024-04-26T20:58:44.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.482+0000 7fae3ae00640 1 -- 172.21.15.28:0/1258579880 shutdown_connections 2024-04-26T20:58:44.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.482+0000 7fae3ae00640 1 --2- 172.21.15.28:0/1258579880 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fae1006c8f0 0x7fae1006edb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.482+0000 7fae3ae00640 1 --2- 172.21.15.28:0/1258579880 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae341638c0 0x7fae340cc650 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.482+0000 7fae3ae00640 1 --2- 172.21.15.28:0/1258579880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fae3415f350 0x7fae340cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.482+0000 7fae3ae00640 1 --2- 172.21.15.28:0/1258579880 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fae340cf8a0 0x7fae34162160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.482+0000 7fae3ae00640 1 -- 172.21.15.28:0/1258579880 >> 172.21.15.28:0/1258579880 conn(0x7fae340b6d10 msgr2=0x7fae340d3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:44.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.482+0000 7fae3ae00640 1 -- 172.21.15.28:0/1258579880 shutdown_connections 2024-04-26T20:58:44.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.482+0000 7fae3ae00640 1 -- 172.21.15.28:0/1258579880 wait complete. 2024-04-26T20:58:44.502 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:58:44.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.701+0000 7f80faa00640 1 Processor -- start 2024-04-26T20:58:44.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.702+0000 7f80faa00640 1 -- start start 2024-04-26T20:58:44.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.703+0000 7f80faa00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f80f40d5540 0x7f80f40b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:44.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.703+0000 7f80faa00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80f40d0710 con 0x7f80f415f1b0 2024-04-26T20:58:44.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.703+0000 7f80faa00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80f40d4b40 con 0x7f80f40d4700 2024-04-26T20:58:44.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.703+0000 7f80faa00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f80f40cff00 con 0x7f80f40d5540 2024-04-26T20:58:44.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.703+0000 7f80f9600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f80f415f1b0 0x7f80f40cf700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52916/0 (socket says 172.21.15.28:52916) 2024-04-26T20:58:44.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.703+0000 7f80f9600640 1 -- 172.21.15.28:0/4291957988 learned_addr learned my addr 172.21.15.28:0/4291957988 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:44.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.704+0000 7f80fa000640 1 --2- 172.21.15.28:0/4291957988 >> v2:172.21.15.103:3300/0 conn(0x7f80f40d5540 0x7f80f40b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:44.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.704+0000 7f80f3e00640 1 -- 172.21.15.28:0/4291957988 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4159501052 0 0) 0x7f80e4001490 con 0x7f80f415f1b0 2024-04-26T20:58:44.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.704+0000 7f80f3e00640 1 -- 172.21.15.28:0/4291957988 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f80d8003520 con 0x7f80f415f1b0 2024-04-26T20:58:44.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.704+0000 7f80f3e00640 1 -- 172.21.15.28:0/4291957988 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3825854596 0 0) 0x7f80d8003520 con 0x7f80f415f1b0 2024-04-26T20:58:44.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.704+0000 7f80f3e00640 1 -- 172.21.15.28:0/4291957988 >> v2:172.21.15.103:3300/0 conn(0x7f80f40d5540 msgr2=0x7f80f40b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:44.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.704+0000 7f80f3e00640 1 --2- 172.21.15.28:0/4291957988 >> v2:172.21.15.103:3300/0 conn(0x7f80f40d5540 0x7f80f40b80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.704+0000 7f80f3e00640 1 -- 172.21.15.28:0/4291957988 >> v1:172.21.15.103:6789/0 conn(0x7f80f40d4700 legacy=0x7f80f40d4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:44.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.704+0000 7f80f3e00640 1 -- 172.21.15.28:0/4291957988 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f40d0150 con 0x7f80f415f1b0 2024-04-26T20:58:44.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.704+0000 7f80f3e00640 1 -- 172.21.15.28:0/4291957988 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f80e4001bc0 con 0x7f80f415f1b0 2024-04-26T20:58:44.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.704+0000 7f80f3e00640 1 -- 172.21.15.28:0/4291957988 >> v1:172.21.15.28:6789/0 conn(0x7f80f415f1b0 legacy=0x7f80f40cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:44.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.704+0000 7f80f3e00640 1 --2- 172.21.15.28:0/4291957988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80d80039b0 0x7f80d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:44.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.705+0000 7f80f3e00640 1 --2- 172.21.15.28:0/4291957988 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80d8004800 0x7f80d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:44.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.705+0000 7f80f3e00640 1 --2- 172.21.15.28:0/4291957988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80d80051c0 0x7f80d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:44.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.705+0000 7f80f3e00640 1 -- 172.21.15.28:0/4291957988 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f80f40cff00 con 0x7f80d80051c0 2024-04-26T20:58:44.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.705+0000 7f80f3e00640 1 -- 172.21.15.28:0/4291957988 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f80d800c070 con 0x7f80d80039b0 2024-04-26T20:58:44.707 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.705+0000 7f80f3e00640 1 -- 172.21.15.28:0/4291957988 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f80d800c270 con 0x7f80d8004800 2024-04-26T20:58:44.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.705+0000 7f80fa000640 1 --2- 172.21.15.28:0/4291957988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80d80051c0 0x7f80d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:44.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.705+0000 7f80fa000640 1 -- 172.21.15.28:0/4291957988 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80d8004800 msgr2=0x7f80d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:44.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.705+0000 7f80fa000640 1 --2- 172.21.15.28:0/4291957988 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80d8004800 0x7f80d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.705+0000 7f80f9600640 1 --2- 172.21.15.28:0/4291957988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80d80039b0 0x7f80d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:44.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.705+0000 7f80fa000640 1 -- 172.21.15.28:0/4291957988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80d80039b0 msgr2=0x7f80d8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:44.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.705+0000 7f80fa000640 1 --2- 172.21.15.28:0/4291957988 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80d80039b0 0x7f80d8003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.705+0000 7f80fa000640 1 -- 172.21.15.28:0/4291957988 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d8003520 con 0x7f80d80051c0 2024-04-26T20:58:44.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.706+0000 7f80fa000640 1 -- 172.21.15.28:0/4291957988 reap_dead start 2024-04-26T20:58:44.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.706+0000 7f80fa000640 1 --2- 172.21.15.28:0/4291957988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80d80051c0 0x7f80d800b9c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009ac0 tx=0x7f80d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07d547726771837 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:44.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.707+0000 7f80f3e00640 1 -- 172.21.15.28:0/4291957988 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec009690 con 0x7f80d80051c0 2024-04-26T20:58:44.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.707+0000 7f80f3e00640 1 -- 172.21.15.28:0/4291957988 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80ec004020 con 0x7f80d80051c0 2024-04-26T20:58:44.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.707+0000 7f80f3e00640 1 -- 172.21.15.28:0/4291957988 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec004410 con 0x7f80d80051c0 2024-04-26T20:58:44.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.707+0000 7f80faa00640 1 -- 172.21.15.28:0/4291957988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80d80051c0 msgr2=0x7f80d800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:44.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.707+0000 7f80faa00640 1 --2- 172.21.15.28:0/4291957988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80d80051c0 0x7f80d800b9c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f80ec009ac0 tx=0x7f80d8005830 comp rx=0 tx=0).stop 2024-04-26T20:58:44.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.707+0000 7f80faa00640 1 -- 172.21.15.28:0/4291957988 shutdown_connections 2024-04-26T20:58:44.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.707+0000 7f80faa00640 1 --2- 172.21.15.28:0/4291957988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80d80051c0 0x7f80d800b9c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.707+0000 7f80faa00640 1 -- 172.21.15.28:0/4291957988 >> 172.21.15.28:0/4291957988 conn(0x7f80f406c770 msgr2=0x7f80f406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:44.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.707+0000 7f80faa00640 1 -- 172.21.15.28:0/4291957988 shutdown_connections 2024-04-26T20:58:44.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.707+0000 7f80faa00640 1 -- 172.21.15.28:0/4291957988 wait complete. 2024-04-26T20:58:44.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.709+0000 7f80faa00640 1 Processor -- start 2024-04-26T20:58:44.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.709+0000 7f80faa00640 1 -- start start 2024-04-26T20:58:44.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.710+0000 7f80faa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80d80051c0 0x7f80f415cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:44.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.710+0000 7f80faa00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80f40ac4f0 0x7f80f40cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:44.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.710+0000 7f80faa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f40ab8a0 0x7f80f40abd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:44.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.710+0000 7f80faa00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f80f40d3f00 con 0x7f80d80051c0 2024-04-26T20:58:44.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.710+0000 7f80faa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f80f40d4270 con 0x7f80f40ab8a0 2024-04-26T20:58:44.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.710+0000 7f80faa00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f80f415f590 con 0x7f80f40ac4f0 2024-04-26T20:58:44.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.710+0000 7f80f9600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80d80051c0 0x7f80f415cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:44.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.710+0000 7f80f9600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80d80051c0 0x7f80f415cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53870/0 (socket says 172.21.15.28:53870) 2024-04-26T20:58:44.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.710+0000 7f80f9600640 1 -- 172.21.15.28:0/2750070569 learned_addr learned my addr 172.21.15.28:0/2750070569 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:44.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.711+0000 7f80f9600640 1 -- 172.21.15.28:0/2750070569 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80f40ac4f0 msgr2=0x7f80f40cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:44.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.711+0000 7f80f8c00640 1 --2- 172.21.15.28:0/2750070569 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80f40ac4f0 0x7f80f40cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:44.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.711+0000 7f80fa000640 1 --2- 172.21.15.28:0/2750070569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f40ab8a0 0x7f80f40abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:44.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.711+0000 7f80f9600640 1 --2- 172.21.15.28:0/2750070569 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80f40ac4f0 0x7f80f40cb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.711+0000 7f80f9600640 1 -- 172.21.15.28:0/2750070569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f40ab8a0 msgr2=0x7f80f40abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:44.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.711+0000 7f80f9600640 1 --2- 172.21.15.28:0/2750070569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f40ab8a0 0x7f80f40abd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.711+0000 7f80f9600640 1 -- 172.21.15.28:0/2750070569 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f40ad140 con 0x7f80d80051c0 2024-04-26T20:58:44.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.711+0000 7f80f9600640 1 --2- 172.21.15.28:0/2750070569 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80d80051c0 0x7f80f415cf70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f80e4001b80 tx=0x7f80e400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:44.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.711+0000 7f80f1600640 1 -- 172.21.15.28:0/2750070569 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4019070 con 0x7f80d80051c0 2024-04-26T20:58:44.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.711+0000 7f80f1600640 1 -- 172.21.15.28:0/2750070569 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e4004d60 con 0x7f80d80051c0 2024-04-26T20:58:44.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.711+0000 7f80f1600640 1 -- 172.21.15.28:0/2750070569 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e4007370 con 0x7f80d80051c0 2024-04-26T20:58:44.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.712+0000 7f80faa00640 1 -- 172.21.15.28:0/2750070569 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f40ad420 con 0x7f80d80051c0 2024-04-26T20:58:44.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.712+0000 7f80faa00640 1 -- 172.21.15.28:0/2750070569 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f40ad9e0 con 0x7f80d80051c0 2024-04-26T20:58:44.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.713+0000 7f80faa00640 1 -- 172.21.15.28:0/2750070569 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f406a360 con 0x7f80d80051c0 2024-04-26T20:58:44.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.714+0000 7f80f1600640 1 -- 172.21.15.28:0/2750070569 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80e4007510 con 0x7f80d80051c0 2024-04-26T20:58:44.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.715+0000 7f80f1600640 1 --2- 172.21.15.28:0/2750070569 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f80d406c920 0x7f80d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:44.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.715+0000 7f80f1600640 1 -- 172.21.15.28:0/2750070569 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f80e408bb50 con 0x7f80d80051c0 2024-04-26T20:58:44.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.715+0000 7f80f8c00640 1 --2- 172.21.15.28:0/2750070569 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f80d406c920 0x7f80d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:44.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.716+0000 7f80f8c00640 1 --2- 172.21.15.28:0/2750070569 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f80d406c920 0x7f80d406ede0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f80f40b6400 tx=0x7f80e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.720+0000 7f80f1600640 1 -- 172.21.15.28:0/2750070569 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f80e404fc00 con 0x7f80d80051c0 2024-04-26T20:58:44.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.919+0000 7f80faa00640 1 -- 172.21.15.28:0/2750070569 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80f4068bd0 con 0x7f80d80051c0 2024-04-26T20:58:44.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.922+0000 7f80f1600640 1 -- 172.21.15.28:0/2750070569 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16602 (secure 0 0 0) 0x7f80f40d0710 con 0x7f80d80051c0 2024-04-26T20:58:44.927 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:44.927 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":44,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:58:43.534414+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:58:41.306286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T20:58:42.497340+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:58:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.924+0000 7f80faa00640 1 -- 172.21.15.28:0/2750070569 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f80d406c920 msgr2=0x7f80d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.924+0000 7f80faa00640 1 --2- 172.21.15.28:0/2750070569 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f80d406c920 0x7f80d406ede0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f80f40b6400 tx=0x7f80e0009040 comp rx=0 tx=0).stop 2024-04-26T20:58:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.925+0000 7f80faa00640 1 -- 172.21.15.28:0/2750070569 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80d80051c0 msgr2=0x7f80f415cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.925+0000 7f80faa00640 1 --2- 172.21.15.28:0/2750070569 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80d80051c0 0x7f80f415cf70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f80e4001b80 tx=0x7f80e400b4a0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.925+0000 7f80faa00640 1 -- 172.21.15.28:0/2750070569 shutdown_connections 2024-04-26T20:58:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.925+0000 7f80faa00640 1 --2- 172.21.15.28:0/2750070569 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f80d406c920 0x7f80d406ede0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.925+0000 7f80faa00640 1 --2- 172.21.15.28:0/2750070569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80f40ab8a0 0x7f80f40abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.925+0000 7f80faa00640 1 --2- 172.21.15.28:0/2750070569 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80f40ac4f0 0x7f80f40cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.925+0000 7f80faa00640 1 --2- 172.21.15.28:0/2750070569 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80d80051c0 0x7f80f415cf70 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.925+0000 7f80faa00640 1 -- 172.21.15.28:0/2750070569 >> 172.21.15.28:0/2750070569 conn(0x7f80f406c770 msgr2=0x7f80f415fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.925+0000 7f80faa00640 1 -- 172.21.15.28:0/2750070569 shutdown_connections 2024-04-26T20:58:44.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:44.925+0000 7f80faa00640 1 -- 172.21.15.28:0/2750070569 wait complete. 2024-04-26T20:58:44.944 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T20:58:45.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.159+0000 7f5345000640 1 Processor -- start 2024-04-26T20:58:45.161 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.160+0000 7f5345000640 1 -- start start 2024-04-26T20:58:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.161+0000 7f5345000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f534015f680 0x7f53400d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.161+0000 7f5345000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f53400d1f90 0x7f53400d33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.161+0000 7f5345000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53400d4110 con 0x7f53400d3920 2024-04-26T20:58:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.161+0000 7f5345000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f53400d23f0 con 0x7f53400d1f90 2024-04-26T20:58:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.161+0000 7f5345000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f53400d43f0 con 0x7f534015f680 2024-04-26T20:58:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.161+0000 7f533d400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f53400d1f90 0x7f53400d33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.161+0000 7f533e800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f53400d3920 0x7f53400b6ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52920/0 (socket says 172.21.15.28:52920) 2024-04-26T20:58:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.161+0000 7f533d400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f53400d1f90 0x7f53400d33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53884/0 (socket says 172.21.15.28:53884) 2024-04-26T20:58:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.161+0000 7f533e800640 1 -- 172.21.15.28:0/3089812625 learned_addr learned my addr 172.21.15.28:0/3089812625 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:45.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.162+0000 7f533de00640 1 --2- 172.21.15.28:0/3089812625 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f534015f680 0x7f53400d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:45.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.162+0000 7f533d400640 1 -- 172.21.15.28:0/3089812625 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f534015f680 msgr2=0x7f53400d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:45.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.162+0000 7f533d400640 1 --2- 172.21.15.28:0/3089812625 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f534015f680 0x7f53400d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:45.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.162+0000 7f533d400640 1 -- 172.21.15.28:0/3089812625 >> v1:172.21.15.28:6789/0 conn(0x7f53400d3920 legacy=0x7f53400b6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:45.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.162+0000 7f533d400640 1 -- 172.21.15.28:0/3089812625 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53400d46d0 con 0x7f53400d1f90 2024-04-26T20:58:45.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.162+0000 7f533de00640 1 --2- 172.21.15.28:0/3089812625 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f534015f680 0x7f53400d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:45.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.163+0000 7f533d400640 1 --2- 172.21.15.28:0/3089812625 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53400d1f90 0x7f53400d33e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5328009790 tx=0x7f532802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c48a17ab7a1b1c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:45.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.163+0000 7f533ca00640 1 -- 172.21.15.28:0/3089812625 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328037070 con 0x7f53400d1f90 2024-04-26T20:58:45.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.163+0000 7f533ca00640 1 -- 172.21.15.28:0/3089812625 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5328032440 con 0x7f53400d1f90 2024-04-26T20:58:45.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.163+0000 7f533ca00640 1 -- 172.21.15.28:0/3089812625 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328032810 con 0x7f53400d1f90 2024-04-26T20:58:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.164+0000 7f5345000640 1 -- 172.21.15.28:0/3089812625 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53400d1f90 msgr2=0x7f53400d33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.164+0000 7f5345000640 1 --2- 172.21.15.28:0/3089812625 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53400d1f90 0x7f53400d33e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5328009790 tx=0x7f532802a180 comp rx=0 tx=0).stop 2024-04-26T20:58:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.164+0000 7f5345000640 1 -- 172.21.15.28:0/3089812625 shutdown_connections 2024-04-26T20:58:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.164+0000 7f5345000640 1 --2- 172.21.15.28:0/3089812625 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53400d1f90 0x7f53400d33e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.164+0000 7f5345000640 1 --2- 172.21.15.28:0/3089812625 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f534015f680 0x7f53400d1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.164+0000 7f5345000640 1 -- 172.21.15.28:0/3089812625 >> 172.21.15.28:0/3089812625 conn(0x7f534006ca10 msgr2=0x7f5340072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.164+0000 7f5345000640 1 -- 172.21.15.28:0/3089812625 shutdown_connections 2024-04-26T20:58:45.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.164+0000 7f5345000640 1 -- 172.21.15.28:0/3089812625 wait complete. 2024-04-26T20:58:45.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.166+0000 7f5345000640 1 Processor -- start 2024-04-26T20:58:45.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.166+0000 7f5345000640 1 -- start start 2024-04-26T20:58:45.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.167+0000 7f5345000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53400d1f90 0x7f5340162320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:45.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.167+0000 7f5345000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53400d3920 0x7f534014c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:45.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.167+0000 7f5345000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f534015f680 0x7f53400caf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:45.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.167+0000 7f5345000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f53400d3db0 con 0x7f53400d3920 2024-04-26T20:58:45.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.167+0000 7f5345000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f53400b8940 con 0x7f53400d1f90 2024-04-26T20:58:45.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.167+0000 7f5345000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f53400b9e70 con 0x7f534015f680 2024-04-26T20:58:45.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.167+0000 7f533d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53400d3920 0x7f534014c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:45.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.167+0000 7f533d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53400d3920 0x7f534014c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53892/0 (socket says 172.21.15.28:53892) 2024-04-26T20:58:45.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.167+0000 7f533d400640 1 -- 172.21.15.28:0/1316669750 learned_addr learned my addr 172.21.15.28:0/1316669750 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:45.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.168+0000 7f533e800640 1 --2- 172.21.15.28:0/1316669750 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f534015f680 0x7f53400caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:45.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.168+0000 7f533d400640 1 -- 172.21.15.28:0/1316669750 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f534015f680 msgr2=0x7f53400caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:45.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.168+0000 7f533d400640 1 --2- 172.21.15.28:0/1316669750 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f534015f680 0x7f53400caf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:45.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.168+0000 7f533d400640 1 -- 172.21.15.28:0/1316669750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53400d1f90 msgr2=0x7f5340162320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:45.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.168+0000 7f533d400640 1 --2- 172.21.15.28:0/1316669750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53400d1f90 0x7f5340162320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:45.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.168+0000 7f533d400640 1 -- 172.21.15.28:0/1316669750 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5340156c80 con 0x7f53400d3920 2024-04-26T20:58:45.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.168+0000 7f533d400640 1 --2- 172.21.15.28:0/1316669750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53400d3920 0x7f534014c2b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5328035040 tx=0x7f53280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:45.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.168+0000 7f5336000640 1 -- 172.21.15.28:0/1316669750 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328037040 con 0x7f53400d3920 2024-04-26T20:58:45.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.168+0000 7f5336000640 1 -- 172.21.15.28:0/1316669750 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5328030770 con 0x7f53400d3920 2024-04-26T20:58:45.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.168+0000 7f5336000640 1 -- 172.21.15.28:0/1316669750 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5328030b40 con 0x7f53400d3920 2024-04-26T20:58:45.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.168+0000 7f5345000640 1 -- 172.21.15.28:0/1316669750 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5340161b30 con 0x7f53400d3920 2024-04-26T20:58:45.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.169+0000 7f5345000640 1 -- 172.21.15.28:0/1316669750 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5340161290 con 0x7f53400d3920 2024-04-26T20:58:45.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.170+0000 7f5345000640 1 -- 172.21.15.28:0/1316669750 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f534006a360 con 0x7f53400d3920 2024-04-26T20:58:45.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.171+0000 7f5336000640 1 -- 172.21.15.28:0/1316669750 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5328044020 con 0x7f53400d3920 2024-04-26T20:58:45.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.172+0000 7f5336000640 1 --2- 172.21.15.28:0/1316669750 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f531c06c8f0 0x7f531c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:45.173 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.172+0000 7f5336000640 1 -- 172.21.15.28:0/1316669750 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f53280ac150 con 0x7f53400d3920 2024-04-26T20:58:45.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.173+0000 7f533de00640 1 --2- 172.21.15.28:0/1316669750 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f531c06c8f0 0x7f531c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:45.174 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.173+0000 7f533de00640 1 --2- 172.21.15.28:0/1316669750 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f531c06c8f0 0x7f531c06edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5330008e00 tx=0x7f5330008330 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:45.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.177+0000 7f5336000640 1 -- 172.21.15.28:0/1316669750 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5340156c80 con 0x7f53400d3920 2024-04-26T20:58:45.374 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-26T20:58:45.374 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-26T20:58:45.374 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-26T20:58:45.374 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-26T20:58:45.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:45.419+0000 7f5345000640 1 -- 172.21.15.28:0/1316669750 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5340162de0 con 0x7f53400d3920 2024-04-26T20:58:45.983 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:58:45.991 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:58:46.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.072+0000 7f5336000640 1 -- 172.21.15.28:0/1316669750 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f5328032e20 con 0x7f53400d3920 2024-04-26T20:58:46.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.075+0000 7f5345000640 1 -- 172.21.15.28:0/1316669750 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f531c06c8f0 msgr2=0x7f531c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:46.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.075+0000 7f5345000640 1 --2- 172.21.15.28:0/1316669750 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f531c06c8f0 0x7f531c06edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5330008e00 tx=0x7f5330008330 comp rx=0 tx=0).stop 2024-04-26T20:58:46.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.075+0000 7f5345000640 1 -- 172.21.15.28:0/1316669750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53400d3920 msgr2=0x7f534014c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:46.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.075+0000 7f5345000640 1 --2- 172.21.15.28:0/1316669750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53400d3920 0x7f534014c2b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5328035040 tx=0x7f53280046c0 comp rx=0 tx=0).stop 2024-04-26T20:58:46.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.075+0000 7f5345000640 1 -- 172.21.15.28:0/1316669750 shutdown_connections 2024-04-26T20:58:46.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.075+0000 7f5345000640 1 --2- 172.21.15.28:0/1316669750 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f531c06c8f0 0x7f531c06edb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:46.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.075+0000 7f5345000640 1 --2- 172.21.15.28:0/1316669750 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f534015f680 0x7f53400caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:46.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.075+0000 7f5345000640 1 --2- 172.21.15.28:0/1316669750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53400d3920 0x7f534014c2b0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:46.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.075+0000 7f5345000640 1 --2- 172.21.15.28:0/1316669750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53400d1f90 0x7f5340162320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:46.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.075+0000 7f5345000640 1 -- 172.21.15.28:0/1316669750 >> 172.21.15.28:0/1316669750 conn(0x7f534006ca10 msgr2=0x7f53400c9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:46.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.076+0000 7f5345000640 1 -- 172.21.15.28:0/1316669750 shutdown_connections 2024-04-26T20:58:46.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.076+0000 7f5345000640 1 -- 172.21.15.28:0/1316669750 wait complete. 2024-04-26T20:58:46.099 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T20:58:46.100 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T20:58:46.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.315+0000 7f7be5200640 1 Processor -- start 2024-04-26T20:58:46.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.316+0000 7f7be5200640 1 -- start start 2024-04-26T20:58:46.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.317+0000 7f7be5200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7be00bac00 0x7f7be0169cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:46.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.317+0000 7f7be5200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7be00ba350 con 0x7f7be015f360 2024-04-26T20:58:46.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.317+0000 7f7be5200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7be00b9ab0 con 0x7f7be00d1de0 2024-04-26T20:58:46.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.317+0000 7f7be5200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7be00b9d90 con 0x7f7be00bac00 2024-04-26T20:58:46.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.317+0000 7f7bd7e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7be015f360 0x7f7be00b7fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52932/0 (socket says 172.21.15.28:52932) 2024-04-26T20:58:46.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.317+0000 7f7bd7e00640 1 -- 172.21.15.28:0/871910998 learned_addr learned my addr 172.21.15.28:0/871910998 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:46.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.318+0000 7f7bdfe00640 1 --2- 172.21.15.28:0/871910998 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7be00bac00 0x7f7be0169cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:46.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.318+0000 7f7bdea00640 1 -- 172.21.15.28:0/871910998 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 424570194 0 0) 0x7f7be00ba350 con 0x7f7be015f360 2024-04-26T20:58:46.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.318+0000 7f7bdea00640 1 -- 172.21.15.28:0/871910998 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bc4003520 con 0x7f7be015f360 2024-04-26T20:58:46.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.318+0000 7f7bdea00640 1 -- 172.21.15.28:0/871910998 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 734991410 0 0) 0x7f7be00b9ab0 con 0x7f7be00d1de0 2024-04-26T20:58:46.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.318+0000 7f7bdea00640 1 -- 172.21.15.28:0/871910998 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7be00ba350 con 0x7f7be00d1de0 2024-04-26T20:58:46.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.318+0000 7f7bdea00640 1 -- 172.21.15.28:0/871910998 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3895418372 0 0) 0x7f7bc4003520 con 0x7f7be015f360 2024-04-26T20:58:46.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.318+0000 7f7bdea00640 1 -- 172.21.15.28:0/871910998 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7be00bac00 msgr2=0x7f7be0169cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:46.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.318+0000 7f7bdea00640 1 --2- 172.21.15.28:0/871910998 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7be00bac00 0x7f7be0169cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:46.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.318+0000 7f7bdea00640 1 -- 172.21.15.28:0/871910998 >> v1:172.21.15.103:6789/0 conn(0x7f7be00d1de0 legacy=0x7f7be00d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:46.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.318+0000 7f7bdea00640 1 -- 172.21.15.28:0/871910998 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be00ba070 con 0x7f7be015f360 2024-04-26T20:58:46.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.318+0000 7f7bdea00640 1 -- 172.21.15.28:0/871910998 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f7bc8001670 con 0x7f7be015f360 2024-04-26T20:58:46.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.318+0000 7f7bdea00640 1 -- 172.21.15.28:0/871910998 >> v1:172.21.15.28:6789/0 conn(0x7f7be015f360 legacy=0x7f7be00b7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:46.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.318+0000 7f7bdea00640 1 --2- 172.21.15.28:0/871910998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bc4003b70 0x7f7bc4003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:46.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.318+0000 7f7bdea00640 1 --2- 172.21.15.28:0/871910998 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7be00bac00 0x7f7bc4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:46.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.318+0000 7f7bdea00640 1 --2- 172.21.15.28:0/871910998 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7bc4004d40 0x7f7bc40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:46.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.318+0000 7f7bdea00640 1 -- 172.21.15.28:0/871910998 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7be00b9d90 con 0x7f7bc4004d40 2024-04-26T20:58:46.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.318+0000 7f7bdea00640 1 -- 172.21.15.28:0/871910998 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc400baa0 con 0x7f7bc4003b70 2024-04-26T20:58:46.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.318+0000 7f7bdea00640 1 -- 172.21.15.28:0/871910998 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc400bca0 con 0x7f7be00bac00 2024-04-26T20:58:46.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.319+0000 7f7bdfe00640 1 --2- 172.21.15.28:0/871910998 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7bc4004d40 0x7f7bc40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:46.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.319+0000 7f7bdf400640 1 --2- 172.21.15.28:0/871910998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bc4003b70 0x7f7bc4003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:46.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.319+0000 7f7bd7e00640 1 --2- 172.21.15.28:0/871910998 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7be00bac00 0x7f7bc4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:46.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.319+0000 7f7bdfe00640 1 -- 172.21.15.28:0/871910998 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7be00bac00 msgr2=0x7f7bc4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:46.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.319+0000 7f7bdfe00640 1 --2- 172.21.15.28:0/871910998 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7be00bac00 0x7f7bc4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:46.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.319+0000 7f7bdfe00640 1 -- 172.21.15.28:0/871910998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bc4003b70 msgr2=0x7f7bc4003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:46.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.319+0000 7f7bdfe00640 1 --2- 172.21.15.28:0/871910998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bc4003b70 0x7f7bc4003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:46.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.319+0000 7f7bdfe00640 1 -- 172.21.15.28:0/871910998 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc4003520 con 0x7f7bc4004d40 2024-04-26T20:58:46.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.319+0000 7f7bdf400640 1 --2- 172.21.15.28:0/871910998 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7bc4003b70 0x7f7bc4003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:58:46.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.320+0000 7f7bdfe00640 1 --2- 172.21.15.28:0/871910998 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7bc4004d40 0x7f7bc40051c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7be008e1a0 tx=0x7f7bc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49db2b158067c712 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:46.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.321+0000 7f7bdea00640 1 -- 172.21.15.28:0/871910998 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8037070 con 0x7f7bc4004d40 2024-04-26T20:58:46.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.321+0000 7f7bdea00640 1 -- 172.21.15.28:0/871910998 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd802e050 con 0x7f7bc4004d40 2024-04-26T20:58:46.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.321+0000 7f7bdea00640 1 -- 172.21.15.28:0/871910998 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8009c10 con 0x7f7bc4004d40 2024-04-26T20:58:46.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.321+0000 7f7be5200640 1 -- 172.21.15.28:0/871910998 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7bc4004d40 msgr2=0x7f7bc40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:46.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.321+0000 7f7be5200640 1 --2- 172.21.15.28:0/871910998 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7bc4004d40 0x7f7bc40051c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7be008e1a0 tx=0x7f7bc4005830 comp rx=0 tx=0).stop 2024-04-26T20:58:46.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.321+0000 7f7bdfe00640 1 -- 172.21.15.28:0/871910998 reap_dead start 2024-04-26T20:58:46.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.321+0000 7f7be5200640 1 -- 172.21.15.28:0/871910998 shutdown_connections 2024-04-26T20:58:46.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.321+0000 7f7be5200640 1 -- 172.21.15.28:0/871910998 >> 172.21.15.28:0/871910998 conn(0x7f7be006c770 msgr2=0x7f7be006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:46.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.321+0000 7f7be5200640 1 -- 172.21.15.28:0/871910998 shutdown_connections 2024-04-26T20:58:46.324 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.321+0000 7f7be5200640 1 -- 172.21.15.28:0/871910998 wait complete. 2024-04-26T20:58:46.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.323+0000 7f7be5200640 1 Processor -- start 2024-04-26T20:58:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.323+0000 7f7be5200640 1 -- start start 2024-04-26T20:58:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.324+0000 7f7be5200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7bc4004d40 0x7f7be014d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.324+0000 7f7be5200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7be00ce5f0 0x7f7be00b5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.324+0000 7f7be5200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7be00cea70 0x7f7be00b5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.324+0000 7f7be5200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7be00b9860 con 0x7f7be00cea70 2024-04-26T20:58:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.324+0000 7f7be5200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7be00b96e0 con 0x7f7be00ce5f0 2024-04-26T20:58:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.324+0000 7f7be5200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7be00d44a0 con 0x7f7bc4004d40 2024-04-26T20:58:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.325+0000 7f7bdfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7be00cea70 0x7f7be00b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.325+0000 7f7bdfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7be00cea70 0x7f7be00b5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53904/0 (socket says 172.21.15.28:53904) 2024-04-26T20:58:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.325+0000 7f7bdfe00640 1 -- 172.21.15.28:0/1698995362 learned_addr learned my addr 172.21.15.28:0/1698995362 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.325+0000 7f7bdf400640 1 --2- 172.21.15.28:0/1698995362 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7bc4004d40 0x7f7be014d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.325+0000 7f7bd7e00640 1 --2- 172.21.15.28:0/1698995362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7be00ce5f0 0x7f7be00b5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:46.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.325+0000 7f7bdfe00640 1 -- 172.21.15.28:0/1698995362 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7bc4004d40 msgr2=0x7f7be014d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:46.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.325+0000 7f7bdfe00640 1 --2- 172.21.15.28:0/1698995362 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7bc4004d40 0x7f7be014d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:46.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.325+0000 7f7bdfe00640 1 -- 172.21.15.28:0/1698995362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7be00ce5f0 msgr2=0x7f7be00b5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:46.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.325+0000 7f7bdfe00640 1 --2- 172.21.15.28:0/1698995362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7be00ce5f0 0x7f7be00b5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:46.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.325+0000 7f7bdfe00640 1 -- 172.21.15.28:0/1698995362 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7be014d7f0 con 0x7f7be00cea70 2024-04-26T20:58:46.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.326+0000 7f7bdfe00640 1 --2- 172.21.15.28:0/1698995362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7be00cea70 0x7f7be00b5880 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7bd800e230 tx=0x7f7bd8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:46.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.326+0000 7f7bd7400640 1 -- 172.21.15.28:0/1698995362 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8045070 con 0x7f7be00cea70 2024-04-26T20:58:46.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.326+0000 7f7bd7400640 1 -- 172.21.15.28:0/1698995362 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bd8003990 con 0x7f7be00cea70 2024-04-26T20:58:46.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.326+0000 7f7bd7400640 1 -- 172.21.15.28:0/1698995362 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bd8037040 con 0x7f7be00cea70 2024-04-26T20:58:46.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.326+0000 7f7be5200640 1 -- 172.21.15.28:0/1698995362 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7be015fb20 con 0x7f7be00cea70 2024-04-26T20:58:46.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.326+0000 7f7be5200640 1 -- 172.21.15.28:0/1698995362 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7be01600e0 con 0x7f7be00cea70 2024-04-26T20:58:46.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.328+0000 7f7be5200640 1 -- 172.21.15.28:0/1698995362 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7be00cf4d0 con 0x7f7be00cea70 2024-04-26T20:58:46.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.329+0000 7f7bd7400640 1 -- 172.21.15.28:0/1698995362 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7bd803a070 con 0x7f7be00cea70 2024-04-26T20:58:46.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.330+0000 7f7bd7400640 1 --2- 172.21.15.28:0/1698995362 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7bbc06c920 0x7f7bbc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:46.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.330+0000 7f7bd7400640 1 -- 172.21.15.28:0/1698995362 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f7bd80ab8d0 con 0x7f7be00cea70 2024-04-26T20:58:46.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.330+0000 7f7bdf400640 1 --2- 172.21.15.28:0/1698995362 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7bbc06c920 0x7f7bbc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:46.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.331+0000 7f7bdf400640 1 --2- 172.21.15.28:0/1698995362 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7bbc06c920 0x7f7bbc06ede0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7be00b6090 tx=0x7f7bd0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:46.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.334+0000 7f7bd7400640 1 -- 172.21.15.28:0/1698995362 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7be00cf4d0 con 0x7f7be00cea70 2024-04-26T20:58:46.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:46.579+0000 7f7be5200640 1 -- 172.21.15.28:0/1698995362 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f7be00d22f0 con 0x7f7be00cea70 2024-04-26T20:58:47.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.084+0000 7f7bd7400640 1 -- 172.21.15.28:0/1698995362 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f7be014d7f0 con 0x7f7be00cea70 2024-04-26T20:58:47.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.087+0000 7f7be5200640 1 -- 172.21.15.28:0/1698995362 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7bbc06c920 msgr2=0x7f7bbc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:47.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.087+0000 7f7be5200640 1 --2- 172.21.15.28:0/1698995362 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7bbc06c920 0x7f7bbc06ede0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7be00b6090 tx=0x7f7bd0009040 comp rx=0 tx=0).stop 2024-04-26T20:58:47.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.087+0000 7f7be5200640 1 -- 172.21.15.28:0/1698995362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7be00cea70 msgr2=0x7f7be00b5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:47.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.087+0000 7f7be5200640 1 --2- 172.21.15.28:0/1698995362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7be00cea70 0x7f7be00b5880 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7bd800e230 tx=0x7f7bd8009750 comp rx=0 tx=0).stop 2024-04-26T20:58:47.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.087+0000 7f7be5200640 1 -- 172.21.15.28:0/1698995362 shutdown_connections 2024-04-26T20:58:47.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.088+0000 7f7be5200640 1 --2- 172.21.15.28:0/1698995362 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7bbc06c920 0x7f7bbc06ede0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:47.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.088+0000 7f7be5200640 1 --2- 172.21.15.28:0/1698995362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7be00cea70 0x7f7be00b5880 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:47.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.088+0000 7f7be5200640 1 --2- 172.21.15.28:0/1698995362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7be00ce5f0 0x7f7be00b5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:47.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.088+0000 7f7be5200640 1 --2- 172.21.15.28:0/1698995362 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7bc4004d40 0x7f7be014d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:47.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.088+0000 7f7be5200640 1 -- 172.21.15.28:0/1698995362 >> 172.21.15.28:0/1698995362 conn(0x7f7be006c770 msgr2=0x7f7be00b8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:47.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.088+0000 7f7be5200640 1 -- 172.21.15.28:0/1698995362 shutdown_connections 2024-04-26T20:58:47.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.088+0000 7f7be5200640 1 -- 172.21.15.28:0/1698995362 wait complete. 2024-04-26T20:58:47.113 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T20:58:47.113 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:58:47.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.330+0000 7fe5ea000640 1 Processor -- start 2024-04-26T20:58:47.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.330+0000 7fe5ea000640 1 -- start start 2024-04-26T20:58:47.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.332+0000 7fe5ea000640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe5e40cb0a0 0x7fe5e406e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:47.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.332+0000 7fe5ea000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe5e40cb500 0x7fe5e406ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:47.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.332+0000 7fe5ea000640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5e4145a10 con 0x7fe5e40c9da0 2024-04-26T20:58:47.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.332+0000 7fe5ea000640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fe5e41603f0 con 0x7fe5e40cb0a0 2024-04-26T20:58:47.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.332+0000 7fe5ea000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe5e415f580 con 0x7fe5e40cb500 2024-04-26T20:58:47.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.333+0000 7fe5e8c00640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7fe5e40c9da0 0x7fe5e40ca180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.28:51598/0 (socket says 172.21.15.28:51598) 2024-04-26T20:58:47.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.333+0000 7fe5e3e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe5e40cb0a0 0x7fe5e406e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:47.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.333+0000 7fe5e8c00640 1 -- 172.21.15.28:0/3927551576 learned_addr learned my addr 172.21.15.28:0/3927551576 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:47.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.333+0000 7fe5e9600640 1 --2- 172.21.15.28:0/3927551576 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe5e40cb500 0x7fe5e406ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:47.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.334+0000 7fe5e3e00640 1 -- 172.21.15.28:0/3927551576 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe5e40cb500 msgr2=0x7fe5e406ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:47.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.334+0000 7fe5e3e00640 1 --2- 172.21.15.28:0/3927551576 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe5e40cb500 0x7fe5e406ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:47.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.334+0000 7fe5e3e00640 1 -- 172.21.15.28:0/3927551576 >> v1:172.21.15.103:6789/0 conn(0x7fe5e40c9da0 legacy=0x7fe5e40ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:47.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.334+0000 7fe5e3e00640 1 -- 172.21.15.28:0/3927551576 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e415f7e0 con 0x7fe5e40cb0a0 2024-04-26T20:58:47.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.335+0000 7fe5e3e00640 1 --2- 172.21.15.28:0/3927551576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5e40cb0a0 0x7fe5e406e9d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00097a0 tx=0x7fe5d002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22927c5af3fd375 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:47.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.335+0000 7fe5e3400640 1 -- 172.21.15.28:0/3927551576 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0037070 con 0x7fe5e40cb0a0 2024-04-26T20:58:47.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.335+0000 7fe5e3400640 1 -- 172.21.15.28:0/3927551576 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5d0032440 con 0x7fe5e40cb0a0 2024-04-26T20:58:47.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.335+0000 7fe5e3400640 1 -- 172.21.15.28:0/3927551576 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d0032810 con 0x7fe5e40cb0a0 2024-04-26T20:58:47.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.336+0000 7fe5ea000640 1 -- 172.21.15.28:0/3927551576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5e40cb0a0 msgr2=0x7fe5e406e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:47.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.336+0000 7fe5ea000640 1 --2- 172.21.15.28:0/3927551576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5e40cb0a0 0x7fe5e406e9d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00097a0 tx=0x7fe5d002a180 comp rx=0 tx=0).stop 2024-04-26T20:58:47.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.336+0000 7fe5ea000640 1 -- 172.21.15.28:0/3927551576 shutdown_connections 2024-04-26T20:58:47.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.336+0000 7fe5ea000640 1 --2- 172.21.15.28:0/3927551576 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe5e40cb500 0x7fe5e406ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:47.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.336+0000 7fe5ea000640 1 --2- 172.21.15.28:0/3927551576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5e40cb0a0 0x7fe5e406e9d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:47.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.336+0000 7fe5ea000640 1 -- 172.21.15.28:0/3927551576 >> 172.21.15.28:0/3927551576 conn(0x7fe5e40d0a50 msgr2=0x7fe5e40d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:47.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.336+0000 7fe5ea000640 1 -- 172.21.15.28:0/3927551576 shutdown_connections 2024-04-26T20:58:47.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.336+0000 7fe5ea000640 1 -- 172.21.15.28:0/3927551576 wait complete. 2024-04-26T20:58:47.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.338+0000 7fe5ea000640 1 Processor -- start 2024-04-26T20:58:47.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.338+0000 7fe5ea000640 1 -- start start 2024-04-26T20:58:47.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.339+0000 7fe5ea000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5e40c9da0 0x7fe5e415e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:47.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.339+0000 7fe5ea000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe5e40cb0a0 0x7fe5e41f8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:47.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.339+0000 7fe5ea000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5e40cb500 0x7fe5e41f8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:47.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.339+0000 7fe5ea000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e4160080 con 0x7fe5e40c9da0 2024-04-26T20:58:47.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.339+0000 7fe5ea000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e415fda0 con 0x7fe5e40cb500 2024-04-26T20:58:47.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.339+0000 7fe5ea000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe5e41489a0 con 0x7fe5e40cb0a0 2024-04-26T20:58:47.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.340+0000 7fe5e8c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5e40c9da0 0x7fe5e415e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:47.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.340+0000 7fe5e8c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5e40c9da0 0x7fe5e415e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53906/0 (socket says 172.21.15.28:53906) 2024-04-26T20:58:47.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.340+0000 7fe5e8c00640 1 -- 172.21.15.28:0/4182917671 learned_addr learned my addr 172.21.15.28:0/4182917671 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:47.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.340+0000 7fe5e3e00640 1 --2- 172.21.15.28:0/4182917671 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe5e40cb0a0 0x7fe5e41f8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:47.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.340+0000 7fe5e9600640 1 --2- 172.21.15.28:0/4182917671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5e40cb500 0x7fe5e41f8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:47.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.340+0000 7fe5e8c00640 1 -- 172.21.15.28:0/4182917671 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe5e40cb0a0 msgr2=0x7fe5e41f8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:47.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.340+0000 7fe5e8c00640 1 --2- 172.21.15.28:0/4182917671 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe5e40cb0a0 0x7fe5e41f8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:47.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.340+0000 7fe5e8c00640 1 -- 172.21.15.28:0/4182917671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5e40cb500 msgr2=0x7fe5e41f8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:47.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.340+0000 7fe5e8c00640 1 --2- 172.21.15.28:0/4182917671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5e40cb500 0x7fe5e41f8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:47.342 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.340+0000 7fe5e8c00640 1 -- 172.21.15.28:0/4182917671 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e4161410 con 0x7fe5e40c9da0 2024-04-26T20:58:47.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.341+0000 7fe5e8c00640 1 --2- 172.21.15.28:0/4182917671 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5e40c9da0 0x7fe5e415e540 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800c3f0 tx=0x7fe5d800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:47.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.341+0000 7fe5e0c00640 1 -- 172.21.15.28:0/4182917671 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8017070 con 0x7fe5e40c9da0 2024-04-26T20:58:47.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.341+0000 7fe5e0c00640 1 -- 172.21.15.28:0/4182917671 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5d8005ce0 con 0x7fe5e40c9da0 2024-04-26T20:58:47.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.341+0000 7fe5e0c00640 1 -- 172.21.15.28:0/4182917671 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d80072f0 con 0x7fe5e40c9da0 2024-04-26T20:58:47.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.341+0000 7fe5ea000640 1 -- 172.21.15.28:0/4182917671 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e41616f0 con 0x7fe5e40c9da0 2024-04-26T20:58:47.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.341+0000 7fe5ea000640 1 -- 172.21.15.28:0/4182917671 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e4161090 con 0x7fe5e40c9da0 2024-04-26T20:58:47.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.342+0000 7fe5ea000640 1 -- 172.21.15.28:0/4182917671 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e406a360 con 0x7fe5e40c9da0 2024-04-26T20:58:47.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.343+0000 7fe5e0c00640 1 -- 172.21.15.28:0/4182917671 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe5d801a070 con 0x7fe5e40c9da0 2024-04-26T20:58:47.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.344+0000 7fe5e0c00640 1 --2- 172.21.15.28:0/4182917671 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe5c406c8f0 0x7fe5c406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:47.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.344+0000 7fe5e0c00640 1 -- 172.21.15.28:0/4182917671 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fe5d808af70 con 0x7fe5e40c9da0 2024-04-26T20:58:47.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.345+0000 7fe5e3e00640 1 --2- 172.21.15.28:0/4182917671 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe5c406c8f0 0x7fe5c406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:47.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.345+0000 7fe5e3e00640 1 --2- 172.21.15.28:0/4182917671 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe5c406c8f0 0x7fe5c406edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0034000 tx=0x7fe5d002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:47.350 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.349+0000 7fe5e0c00640 1 -- 172.21.15.28:0/4182917671 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe5d8059300 con 0x7fe5e40c9da0 2024-04-26T20:58:47.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.592+0000 7fe5ea000640 1 -- 172.21.15.28:0/4182917671 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5e4074550 con 0x7fe5e40c9da0 2024-04-26T20:58:47.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.593+0000 7fe5e0c00640 1 -- 172.21.15.28:0/4182917671 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5184 (secure 0 0 0) 0x7fe5d8058c00 con 0x7fe5e40c9da0 2024-04-26T20:58:47.595 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:47.595 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/356870645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":356870645},{"type":"v1","addr":"172.21.15.28:6835","nonce":356870645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4829,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2957511478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":2957511478},{"type":"v1","addr":"172.21.15.103:6835","nonce":2957511478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/2060632176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2060632176},{"type":"v1","addr":"172.21.15.103:6834","nonce":2060632176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:58:43.534341+0000","modified":"2024-04-26T20:58:46.581816+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5571},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6837/2997047727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":2997047727},{"type":"v1","addr":"172.21.15.28:6837","nonce":2997047727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5571,"qdb_cluster":[5571]},"id":3}]} 2024-04-26T20:58:47.595 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 26 2024-04-26T20:58:47.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.596+0000 7fe5ea000640 1 -- 172.21.15.28:0/4182917671 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe5c406c8f0 msgr2=0x7fe5c406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:47.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.596+0000 7fe5ea000640 1 --2- 172.21.15.28:0/4182917671 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe5c406c8f0 0x7fe5c406edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0034000 tx=0x7fe5d002e000 comp rx=0 tx=0).stop 2024-04-26T20:58:47.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.596+0000 7fe5ea000640 1 -- 172.21.15.28:0/4182917671 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5e40c9da0 msgr2=0x7fe5e415e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:47.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.596+0000 7fe5ea000640 1 --2- 172.21.15.28:0/4182917671 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5e40c9da0 0x7fe5e415e540 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800c3f0 tx=0x7fe5d800c8b0 comp rx=0 tx=0).stop 2024-04-26T20:58:47.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.597+0000 7fe5ea000640 1 -- 172.21.15.28:0/4182917671 shutdown_connections 2024-04-26T20:58:47.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.597+0000 7fe5ea000640 1 --2- 172.21.15.28:0/4182917671 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe5c406c8f0 0x7fe5c406edb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:47.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.597+0000 7fe5ea000640 1 --2- 172.21.15.28:0/4182917671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5e40cb500 0x7fe5e41f8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:47.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.597+0000 7fe5ea000640 1 --2- 172.21.15.28:0/4182917671 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe5e40cb0a0 0x7fe5e41f8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:47.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.597+0000 7fe5ea000640 1 --2- 172.21.15.28:0/4182917671 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5e40c9da0 0x7fe5e415e540 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:47.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.597+0000 7fe5ea000640 1 -- 172.21.15.28:0/4182917671 >> 172.21.15.28:0/4182917671 conn(0x7fe5e40d0a50 msgr2=0x7fe5e40b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:47.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.597+0000 7fe5ea000640 1 -- 172.21.15.28:0/4182917671 shutdown_connections 2024-04-26T20:58:47.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.597+0000 7fe5ea000640 1 -- 172.21.15.28:0/4182917671 wait complete. 2024-04-26T20:58:47.616 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T20:58:47.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.828+0000 7f3093000640 1 Processor -- start 2024-04-26T20:58:47.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.828+0000 7f3093000640 1 -- start start 2024-04-26T20:58:47.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.830+0000 7f3093000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f308c146060 0x7f308c071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:47.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.830+0000 7f3093000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f308c146670 0x7f308c1609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:47.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.830+0000 7f3093000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f308c146440 con 0x7f308c160ef0 2024-04-26T20:58:47.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.830+0000 7f3093000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f308c06f510 con 0x7f308c146670 2024-04-26T20:58:47.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.830+0000 7f3093000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f308c06f010 con 0x7f308c146060 2024-04-26T20:58:47.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.830+0000 7f3091200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f308c146670 0x7f308c1609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:47.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.830+0000 7f3091200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f308c146670 0x7f308c1609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53918/0 (socket says 172.21.15.28:53918) 2024-04-26T20:58:47.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.830+0000 7f3091200640 1 -- 172.21.15.28:0/687201772 learned_addr learned my addr 172.21.15.28:0/687201772 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:47.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.831+0000 7f3091c00640 1 --2- 172.21.15.28:0/687201772 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f308c146060 0x7f308c071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:47.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.831+0000 7f3091200640 1 -- 172.21.15.28:0/687201772 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f308c146060 msgr2=0x7f308c071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:47.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.831+0000 7f3091200640 1 --2- 172.21.15.28:0/687201772 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f308c146060 0x7f308c071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:47.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.831+0000 7f3091200640 1 -- 172.21.15.28:0/687201772 >> v1:172.21.15.28:6789/0 conn(0x7f308c160ef0 legacy=0x7f308c06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:47.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.831+0000 7f3091200640 1 -- 172.21.15.28:0/687201772 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308c1617a0 con 0x7f308c146670 2024-04-26T20:58:47.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.832+0000 7f3091200640 1 --2- 172.21.15.28:0/687201772 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f308c146670 0x7f308c1609b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3078009870 tx=0x7f307802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c679826c8ce094f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:47.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.832+0000 7f308be00640 1 -- 172.21.15.28:0/687201772 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078037070 con 0x7f308c146670 2024-04-26T20:58:47.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.832+0000 7f308be00640 1 -- 172.21.15.28:0/687201772 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3078032440 con 0x7f308c146670 2024-04-26T20:58:47.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.832+0000 7f308be00640 1 -- 172.21.15.28:0/687201772 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078032810 con 0x7f308c146670 2024-04-26T20:58:47.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.833+0000 7f3093000640 1 -- 172.21.15.28:0/687201772 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f308c146670 msgr2=0x7f308c1609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:47.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.833+0000 7f3093000640 1 --2- 172.21.15.28:0/687201772 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f308c146670 0x7f308c1609b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3078009870 tx=0x7f307802a1c0 comp rx=0 tx=0).stop 2024-04-26T20:58:47.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.833+0000 7f3093000640 1 -- 172.21.15.28:0/687201772 shutdown_connections 2024-04-26T20:58:47.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.833+0000 7f3093000640 1 --2- 172.21.15.28:0/687201772 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f308c146670 0x7f308c1609b0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:47.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.833+0000 7f3093000640 1 --2- 172.21.15.28:0/687201772 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f308c146060 0x7f308c071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:47.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.833+0000 7f3093000640 1 -- 172.21.15.28:0/687201772 >> 172.21.15.28:0/687201772 conn(0x7f308c0d0a50 msgr2=0x7f308c0d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:47.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.833+0000 7f3093000640 1 -- 172.21.15.28:0/687201772 shutdown_connections 2024-04-26T20:58:47.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.833+0000 7f3093000640 1 -- 172.21.15.28:0/687201772 wait complete. 2024-04-26T20:58:47.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.835+0000 7f3093000640 1 Processor -- start 2024-04-26T20:58:47.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.835+0000 7f3093000640 1 -- start start 2024-04-26T20:58:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.836+0000 7f3093000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f308c146060 0x7f308c1e6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.836+0000 7f3093000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f308c146670 0x7f308c1e92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.836+0000 7f3093000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f308c160ef0 0x7f308c15d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.836+0000 7f3093000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f308c0cc0b0 con 0x7f308c160ef0 2024-04-26T20:58:47.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.836+0000 7f3093000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f308c06ff60 con 0x7f308c146060 2024-04-26T20:58:47.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.836+0000 7f3093000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f308c0cc230 con 0x7f308c146670 2024-04-26T20:58:47.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.836+0000 7f3092600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f308c160ef0 0x7f308c15d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:47.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.836+0000 7f3092600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f308c160ef0 0x7f308c15d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53934/0 (socket says 172.21.15.28:53934) 2024-04-26T20:58:47.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.836+0000 7f3092600640 1 -- 172.21.15.28:0/3841858009 learned_addr learned my addr 172.21.15.28:0/3841858009 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:47.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.837+0000 7f3091200640 1 --2- 172.21.15.28:0/3841858009 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f308c146670 0x7f308c1e92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:47.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.837+0000 7f3091c00640 1 --2- 172.21.15.28:0/3841858009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f308c146060 0x7f308c1e6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:47.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.837+0000 7f3092600640 1 -- 172.21.15.28:0/3841858009 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f308c146670 msgr2=0x7f308c1e92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:47.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.837+0000 7f3092600640 1 --2- 172.21.15.28:0/3841858009 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f308c146670 0x7f308c1e92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:47.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.837+0000 7f3092600640 1 -- 172.21.15.28:0/3841858009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f308c146060 msgr2=0x7f308c1e6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:47.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.837+0000 7f3092600640 1 --2- 172.21.15.28:0/3841858009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f308c146060 0x7f308c1e6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:47.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.837+0000 7f3092600640 1 -- 172.21.15.28:0/3841858009 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308c1e6ed0 con 0x7f308c160ef0 2024-04-26T20:58:47.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.837+0000 7f3092600640 1 --2- 172.21.15.28:0/3841858009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f308c160ef0 0x7f308c15d1f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f308400c3c0 tx=0x7f308400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:47.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.837+0000 7f3089600640 1 -- 172.21.15.28:0/3841858009 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084017070 con 0x7f308c160ef0 2024-04-26T20:58:47.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.837+0000 7f3089600640 1 -- 172.21.15.28:0/3841858009 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30840052d0 con 0x7f308c160ef0 2024-04-26T20:58:47.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.837+0000 7f3093000640 1 -- 172.21.15.28:0/3841858009 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f308c1e71b0 con 0x7f308c160ef0 2024-04-26T20:58:47.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.837+0000 7f3089600640 1 -- 172.21.15.28:0/3841858009 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084003810 con 0x7f308c160ef0 2024-04-26T20:58:47.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.838+0000 7f3093000640 1 -- 172.21.15.28:0/3841858009 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f308c1e7770 con 0x7f308c160ef0 2024-04-26T20:58:47.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.839+0000 7f3093000640 1 -- 172.21.15.28:0/3841858009 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f308c06a360 con 0x7f308c160ef0 2024-04-26T20:58:47.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.840+0000 7f3089600640 1 -- 172.21.15.28:0/3841858009 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f308401a070 con 0x7f308c160ef0 2024-04-26T20:58:47.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.841+0000 7f3089600640 1 --2- 172.21.15.28:0/3841858009 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f306c06c8f0 0x7f306c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:47.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.841+0000 7f3089600640 1 -- 172.21.15.28:0/3841858009 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f308408af40 con 0x7f308c160ef0 2024-04-26T20:58:47.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.841+0000 7f3091c00640 1 --2- 172.21.15.28:0/3841858009 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f306c06c8f0 0x7f306c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:47.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.842+0000 7f3091c00640 1 --2- 172.21.15.28:0/3841858009 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f306c06c8f0 0x7f306c06edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3080009e90 tx=0x7f308000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:47.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:47.846+0000 7f3089600640 1 -- 172.21.15.28:0/3841858009 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3084059240 con 0x7f308c160ef0 2024-04-26T20:58:48.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.052+0000 7f3093000640 1 -- 172.21.15.28:0/3841858009 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f308c1e6bc0 con 0x7f308c160ef0 2024-04-26T20:58:48.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.054+0000 7f3089600640 1 -- 172.21.15.28:0/3841858009 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16589 (secure 0 0 0) 0x7f3084058b40 con 0x7f308c160ef0 2024-04-26T20:58:48.055 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:48.055 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":45,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:58:45.050870+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:58:41.306286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T20:58:42.497340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T20:58:48.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.057+0000 7f3093000640 1 -- 172.21.15.28:0/3841858009 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f306c06c8f0 msgr2=0x7f306c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:48.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.057+0000 7f3093000640 1 --2- 172.21.15.28:0/3841858009 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f306c06c8f0 0x7f306c06edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3080009e90 tx=0x7f308000b040 comp rx=0 tx=0).stop 2024-04-26T20:58:48.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.057+0000 7f3093000640 1 -- 172.21.15.28:0/3841858009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f308c160ef0 msgr2=0x7f308c15d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:48.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.057+0000 7f3093000640 1 --2- 172.21.15.28:0/3841858009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f308c160ef0 0x7f308c15d1f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f308400c3c0 tx=0x7f308400c880 comp rx=0 tx=0).stop 2024-04-26T20:58:48.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.057+0000 7f3093000640 1 -- 172.21.15.28:0/3841858009 shutdown_connections 2024-04-26T20:58:48.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.057+0000 7f3093000640 1 --2- 172.21.15.28:0/3841858009 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f306c06c8f0 0x7f306c06edb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:48.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.057+0000 7f3093000640 1 --2- 172.21.15.28:0/3841858009 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f308c160ef0 0x7f308c15d1f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:48.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.057+0000 7f3093000640 1 --2- 172.21.15.28:0/3841858009 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f308c146670 0x7f308c1e92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:48.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.057+0000 7f3093000640 1 --2- 172.21.15.28:0/3841858009 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f308c146060 0x7f308c1e6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:48.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.057+0000 7f3093000640 1 -- 172.21.15.28:0/3841858009 >> 172.21.15.28:0/3841858009 conn(0x7f308c0d0a50 msgr2=0x7f308c0cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:48.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.058+0000 7f3093000640 1 -- 172.21.15.28:0/3841858009 shutdown_connections 2024-04-26T20:58:48.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.058+0000 7f3093000640 1 -- 172.21.15.28:0/3841858009 wait complete. 2024-04-26T20:58:48.078 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T20:58:48.078 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:58:48.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.290+0000 7f534ac00640 1 Processor -- start 2024-04-26T20:58:48.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.291+0000 7f534ac00640 1 -- start start 2024-04-26T20:58:48.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.292+0000 7f534ac00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f534415f1b0 0x7f53440d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:48.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.292+0000 7f534ac00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f53440cf700 0x7f53440d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:48.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.292+0000 7f534ac00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53440d0710 con 0x7f53440d5370 2024-04-26T20:58:48.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.292+0000 7f534ac00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f53440d57e0 con 0x7f534415f1b0 2024-04-26T20:58:48.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.292+0000 7f534ac00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f53440cff00 con 0x7f53440cf700 2024-04-26T20:58:48.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.292+0000 7f5349800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f534415f1b0 0x7f53440d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:48.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.293+0000 7f5349800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f534415f1b0 0x7f53440d4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53944/0 (socket says 172.21.15.28:53944) 2024-04-26T20:58:48.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.293+0000 7f5349800640 1 -- 172.21.15.28:0/2062996403 learned_addr learned my addr 172.21.15.28:0/2062996403 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:48.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.293+0000 7f5349800640 1 -- 172.21.15.28:0/2062996403 >> v2:172.21.15.103:3300/0 conn(0x7f53440cf700 msgr2=0x7f53440d4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:48.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.293+0000 7f5349800640 1 --2- 172.21.15.28:0/2062996403 >> v2:172.21.15.103:3300/0 conn(0x7f53440cf700 0x7f53440d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:48.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.293+0000 7f5349800640 1 -- 172.21.15.28:0/2062996403 >> v1:172.21.15.28:6789/0 conn(0x7f53440d5370 legacy=0x7f53440b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:48.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.293+0000 7f5349800640 1 -- 172.21.15.28:0/2062996403 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53440d0150 con 0x7f534415f1b0 2024-04-26T20:58:48.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.294+0000 7f5349800640 1 --2- 172.21.15.28:0/2062996403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f534415f1b0 0x7f53440d4700 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f53440d0080 tx=0x7f533402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b469f838623a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:48.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.294+0000 7f5343e00640 1 -- 172.21.15.28:0/2062996403 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5334037070 con 0x7f534415f1b0 2024-04-26T20:58:48.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.295+0000 7f5343e00640 1 -- 172.21.15.28:0/2062996403 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f533402ae50 con 0x7f534415f1b0 2024-04-26T20:58:48.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.295+0000 7f5343e00640 1 -- 172.21.15.28:0/2062996403 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5334032860 con 0x7f534415f1b0 2024-04-26T20:58:48.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.295+0000 7f534ac00640 1 -- 172.21.15.28:0/2062996403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f534415f1b0 msgr2=0x7f53440d4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:48.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.295+0000 7f534ac00640 1 --2- 172.21.15.28:0/2062996403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f534415f1b0 0x7f53440d4700 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f53440d0080 tx=0x7f533402a300 comp rx=0 tx=0).stop 2024-04-26T20:58:48.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.295+0000 7f534ac00640 1 -- 172.21.15.28:0/2062996403 shutdown_connections 2024-04-26T20:58:48.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.295+0000 7f534ac00640 1 --2- 172.21.15.28:0/2062996403 >> v2:172.21.15.103:3300/0 conn(0x7f53440cf700 0x7f53440d4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:48.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.295+0000 7f534ac00640 1 --2- 172.21.15.28:0/2062996403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f534415f1b0 0x7f53440d4700 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:48.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.295+0000 7f534ac00640 1 -- 172.21.15.28:0/2062996403 >> 172.21.15.28:0/2062996403 conn(0x7f534406c770 msgr2=0x7f534406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:48.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.295+0000 7f534ac00640 1 -- 172.21.15.28:0/2062996403 shutdown_connections 2024-04-26T20:58:48.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.295+0000 7f534ac00640 1 -- 172.21.15.28:0/2062996403 wait complete. 2024-04-26T20:58:48.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.297+0000 7f534ac00640 1 Processor -- start 2024-04-26T20:58:48.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.297+0000 7f534ac00640 1 -- start start 2024-04-26T20:58:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.298+0000 7f534ac00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53440cf700 0x7f534414e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.298+0000 7f534ac00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53440d5370 0x7f53440cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.298+0000 7f534ac00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f534415f1b0 0x7f53440aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.298+0000 7f534ac00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f53440d3f00 con 0x7f53440cf700 2024-04-26T20:58:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.298+0000 7f534ac00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f53440d4270 con 0x7f53440d5370 2024-04-26T20:58:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.298+0000 7f534ac00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f534415f590 con 0x7f534415f1b0 2024-04-26T20:58:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.298+0000 7f5349800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53440cf700 0x7f534414e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.299+0000 7f5349800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53440cf700 0x7f534414e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53946/0 (socket says 172.21.15.28:53946) 2024-04-26T20:58:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.299+0000 7f5349800640 1 -- 172.21.15.28:0/2619836404 learned_addr learned my addr 172.21.15.28:0/2619836404 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:48.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.299+0000 7f5348e00640 1 --2- 172.21.15.28:0/2619836404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53440d5370 0x7f53440cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:48.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.299+0000 7f5349800640 1 -- 172.21.15.28:0/2619836404 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f534415f1b0 msgr2=0x7f53440aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:48.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.299+0000 7f534a200640 1 --2- 172.21.15.28:0/2619836404 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f534415f1b0 0x7f53440aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:48.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.299+0000 7f5349800640 1 --2- 172.21.15.28:0/2619836404 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f534415f1b0 0x7f53440aea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:48.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.299+0000 7f5349800640 1 -- 172.21.15.28:0/2619836404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53440d5370 msgr2=0x7f53440cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:48.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.299+0000 7f5349800640 1 --2- 172.21.15.28:0/2619836404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53440d5370 0x7f53440cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:48.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.299+0000 7f5349800640 1 -- 172.21.15.28:0/2619836404 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534414ec80 con 0x7f53440cf700 2024-04-26T20:58:48.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.299+0000 7f5348e00640 1 --2- 172.21.15.28:0/2619836404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53440d5370 0x7f53440cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:48.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.299+0000 7f5349800640 1 --2- 172.21.15.28:0/2619836404 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53440cf700 0x7f534414e6a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5334000c00 tx=0x7f53340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:48.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.300+0000 7f5341600640 1 -- 172.21.15.28:0/2619836404 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5334041070 con 0x7f53440cf700 2024-04-26T20:58:48.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.300+0000 7f5341600640 1 -- 172.21.15.28:0/2619836404 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f533403c870 con 0x7f53440cf700 2024-04-26T20:58:48.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.300+0000 7f5341600640 1 -- 172.21.15.28:0/2619836404 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5334037040 con 0x7f53440cf700 2024-04-26T20:58:48.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.300+0000 7f534ac00640 1 -- 172.21.15.28:0/2619836404 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f534414ef60 con 0x7f53440cf700 2024-04-26T20:58:48.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.300+0000 7f534ac00640 1 -- 172.21.15.28:0/2619836404 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f534414f800 con 0x7f53440cf700 2024-04-26T20:58:48.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.301+0000 7f534ac00640 1 -- 172.21.15.28:0/2619836404 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f534414f130 con 0x7f53440cf700 2024-04-26T20:58:48.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.302+0000 7f5341600640 1 -- 172.21.15.28:0/2619836404 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5334044070 con 0x7f53440cf700 2024-04-26T20:58:48.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.303+0000 7f5341600640 1 --2- 172.21.15.28:0/2619836404 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f532406c940 0x7f532406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:48.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.303+0000 7f5341600640 1 -- 172.21.15.28:0/2619836404 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f53340ac6b0 con 0x7f53440cf700 2024-04-26T20:58:48.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.303+0000 7f5348e00640 1 --2- 172.21.15.28:0/2619836404 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f532406c940 0x7f532406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:48.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.304+0000 7f5348e00640 1 --2- 172.21.15.28:0/2619836404 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f532406c940 0x7f532406ee00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f534415e920 tx=0x7f5330008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:48.308 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.308+0000 7f5341600640 1 -- 172.21.15.28:0/2619836404 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f533407abe0 con 0x7f53440cf700 2024-04-26T20:58:48.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.508+0000 7f534ac00640 1 -- 172.21.15.28:0/2619836404 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f534414f360 con 0x7f532406c940 2024-04-26T20:58:48.517 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.516+0000 7f5341600640 1 -- 172.21.15.28:0/2619836404 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246636 (secure 0 0 0) 0x7f533000c7d0 con 0x7f532406c940 2024-04-26T20:58:48.521 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:48.523 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T20:58:48.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.522+0000 7f534ac00640 1 -- 172.21.15.28:0/2619836404 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f532406c940 msgr2=0x7f532406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:48.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.522+0000 7f534ac00640 1 --2- 172.21.15.28:0/2619836404 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f532406c940 0x7f532406ee00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f534415e920 tx=0x7f5330008040 comp rx=0 tx=0).stop 2024-04-26T20:58:48.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.522+0000 7f534ac00640 1 -- 172.21.15.28:0/2619836404 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53440cf700 msgr2=0x7f534414e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:48.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.522+0000 7f534ac00640 1 --2- 172.21.15.28:0/2619836404 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53440cf700 0x7f534414e6a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5334000c00 tx=0x7f53340046c0 comp rx=0 tx=0).stop 2024-04-26T20:58:48.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.523+0000 7f534ac00640 1 -- 172.21.15.28:0/2619836404 shutdown_connections 2024-04-26T20:58:48.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.523+0000 7f534ac00640 1 --2- 172.21.15.28:0/2619836404 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f532406c940 0x7f532406ee00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:48.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.523+0000 7f534ac00640 1 --2- 172.21.15.28:0/2619836404 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f534415f1b0 0x7f53440aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:48.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.523+0000 7f534ac00640 1 --2- 172.21.15.28:0/2619836404 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f53440d5370 0x7f53440cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:48.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.523+0000 7f534ac00640 1 --2- 172.21.15.28:0/2619836404 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53440cf700 0x7f534414e6a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:48.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.523+0000 7f534ac00640 1 -- 172.21.15.28:0/2619836404 >> 172.21.15.28:0/2619836404 conn(0x7f534406c770 msgr2=0x7f53440cdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:48.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.523+0000 7f534ac00640 1 -- 172.21.15.28:0/2619836404 shutdown_connections 2024-04-26T20:58:48.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.523+0000 7f534ac00640 1 -- 172.21.15.28:0/2619836404 wait complete. 2024-04-26T20:58:48.544 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-04-26T20:58:46.741270+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237876,"kb_used_data":5060,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736844,"statfs":{"total":773094113280,"available":772850528256,"internally_reserved":0,"allocated":5181440,"data_stored":2773167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.724338"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055951+0000","last_change":"2024-04-26T20:58:43.055951+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055951+0000","last_undegraded":"2024-04-26T20:58:43.055951+0000","last_fullsized":"2024-04-26T20:58:43.055951+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.052235+0000","last_change":"2024-04-26T20:58:43.052235+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.052235+0000","last_undegraded":"2024-04-26T20:58:43.052235+0000","last_fullsized":"2024-04-26T20:58:43.052235+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.048628+0000","last_change":"2024-04-26T20:58:43.048628+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.048628+0000","last_undegraded":"2024-04-26T20:58:43.048628+0000","last_fullsized":"2024-04-26T20:58:43.048628+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056479+0000","last_change":"2024-04-26T20:58:43.056479+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056479+0000","last_undegraded":"2024-04-26T20:58:43.056479+0000","last_fullsized":"2024-04-26T20:58:43.056479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056437+0000","last_change":"2024-04-26T20:58:43.056437+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056437+0000","last_undegraded":"2024-04-26T20:58:43.056437+0000","last_fullsized":"2024-04-26T20:58:43.056437+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.048264+0000","last_change":"2024-04-26T20:58:43.048264+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.048264+0000","last_undegraded":"2024-04-26T20:58:43.048264+0000","last_fullsized":"2024-04-26T20:58:43.048264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.049255+0000","last_change":"2024-04-26T20:58:43.049255+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.049255+0000","last_undegraded":"2024-04-26T20:58:43.049255+0000","last_fullsized":"2024-04-26T20:58:43.049255+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.051206+0000","last_change":"2024-04-26T20:58:43.051206+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.051206+0000","last_undegraded":"2024-04-26T20:58:43.051206+0000","last_fullsized":"2024-04-26T20:58:43.051206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.053178+0000","last_change":"2024-04-26T20:58:43.053178+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.053178+0000","last_undegraded":"2024-04-26T20:58:43.053178+0000","last_fullsized":"2024-04-26T20:58:43.053178+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056034+0000","last_change":"2024-04-26T20:58:43.056034+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056034+0000","last_undegraded":"2024-04-26T20:58:43.056034+0000","last_fullsized":"2024-04-26T20:58:43.056034+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.052135+0000","last_change":"2024-04-26T20:58:43.052135+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.052135+0000","last_undegraded":"2024-04-26T20:58:43.052135+0000","last_fullsized":"2024-04-26T20:58:43.052135+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.060146+0000","last_change":"2024-04-26T20:58:43.060146+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.060146+0000","last_undegraded":"2024-04-26T20:58:43.060146+0000","last_fullsized":"2024-04-26T20:58:43.060146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.038802+0000","last_change":"2024-04-26T20:58:43.038802+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.038802+0000","last_undegraded":"2024-04-26T20:58:43.038802+0000","last_fullsized":"2024-04-26T20:58:43.038802+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.051120+0000","last_change":"2024-04-26T20:58:43.051120+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.051120+0000","last_undegraded":"2024-04-26T20:58:43.051120+0000","last_fullsized":"2024-04-26T20:58:43.051120+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.049339+0000","last_change":"2024-04-26T20:58:43.049339+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.049339+0000","last_undegraded":"2024-04-26T20:58:43.049339+0000","last_fullsized":"2024-04-26T20:58:43.049339+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.061785+0000","last_change":"2024-04-26T20:58:43.061785+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.061785+0000","last_undegraded":"2024-04-26T20:58:43.061785+0000","last_fullsized":"2024-04-26T20:58:43.061785+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.049592+0000","last_change":"2024-04-26T20:58:43.049592+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.049592+0000","last_undegraded":"2024-04-26T20:58:43.049592+0000","last_fullsized":"2024-04-26T20:58:43.049592+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.051921+0000","last_change":"2024-04-26T20:58:43.051921+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.051921+0000","last_undegraded":"2024-04-26T20:58:43.051921+0000","last_fullsized":"2024-04-26T20:58:43.051921+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056057+0000","last_change":"2024-04-26T20:58:43.056057+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056057+0000","last_undegraded":"2024-04-26T20:58:43.056057+0000","last_fullsized":"2024-04-26T20:58:43.056057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.039248+0000","last_change":"2024-04-26T20:58:43.039248+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.039248+0000","last_undegraded":"2024-04-26T20:58:43.039248+0000","last_fullsized":"2024-04-26T20:58:43.039248+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.047349+0000","last_change":"2024-04-26T20:58:43.047349+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.047349+0000","last_undegraded":"2024-04-26T20:58:43.047349+0000","last_fullsized":"2024-04-26T20:58:43.047349+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.046758+0000","last_change":"2024-04-26T20:58:43.046758+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.046758+0000","last_undegraded":"2024-04-26T20:58:43.046758+0000","last_fullsized":"2024-04-26T20:58:43.046758+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.059090+0000","last_change":"2024-04-26T20:58:43.059090+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.059090+0000","last_undegraded":"2024-04-26T20:58:43.059090+0000","last_fullsized":"2024-04-26T20:58:43.059090+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.060755+0000","last_change":"2024-04-26T20:58:43.060755+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.060755+0000","last_undegraded":"2024-04-26T20:58:43.060755+0000","last_fullsized":"2024-04-26T20:58:43.060755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.064938+0000","last_change":"2024-04-26T20:58:43.064938+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.064938+0000","last_undegraded":"2024-04-26T20:58:43.064938+0000","last_fullsized":"2024-04-26T20:58:43.064938+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.040818+0000","last_change":"2024-04-26T20:58:43.040818+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.040818+0000","last_undegraded":"2024-04-26T20:58:43.040818+0000","last_fullsized":"2024-04-26T20:58:43.040818+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.046134+0000","last_change":"2024-04-26T20:58:43.046134+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.046134+0000","last_undegraded":"2024-04-26T20:58:43.046134+0000","last_fullsized":"2024-04-26T20:58:43.046134+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.058202+0000","last_change":"2024-04-26T20:58:43.058202+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.058202+0000","last_undegraded":"2024-04-26T20:58:43.058202+0000","last_fullsized":"2024-04-26T20:58:43.058202+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056143+0000","last_change":"2024-04-26T20:58:43.056143+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056143+0000","last_undegraded":"2024-04-26T20:58:43.056143+0000","last_fullsized":"2024-04-26T20:58:43.056143+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.047071+0000","last_change":"2024-04-26T20:58:43.047071+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.047071+0000","last_undegraded":"2024-04-26T20:58:43.047071+0000","last_fullsized":"2024-04-26T20:58:43.047071+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.047975+0000","last_change":"2024-04-26T20:58:43.047975+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.047975+0000","last_undegraded":"2024-04-26T20:58:43.047975+0000","last_fullsized":"2024-04-26T20:58:43.047975+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.040702+0000","last_change":"2024-04-26T20:58:43.040702+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.040702+0000","last_undegraded":"2024-04-26T20:58:43.040702+0000","last_fullsized":"2024-04-26T20:58:43.040702+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.053161+0000","last_change":"2024-04-26T20:58:43.053161+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.053161+0000","last_undegraded":"2024-04-26T20:58:43.053161+0000","last_fullsized":"2024-04-26T20:58:43.053161+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.072931+0000","last_change":"2024-04-26T20:58:43.070204+0000","last_active":"2024-04-26T20:58:43.072931+0000","last_peered":"2024-04-26T20:58:43.072931+0000","last_clean":"2024-04-26T20:58:43.072931+0000","last_became_active":"2024-04-26T20:58:43.069791+0000","last_became_peered":"2024-04-26T20:58:43.069791+0000","last_unstale":"2024-04-26T20:58:43.072931+0000","last_undegraded":"2024-04-26T20:58:43.072931+0000","last_fullsized":"2024-04-26T20:58:43.072931+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:21.679330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.064417+0000","last_change":"2024-04-26T20:58:43.064417+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.064417+0000","last_undegraded":"2024-04-26T20:58:43.064417+0000","last_fullsized":"2024-04-26T20:58:43.064417+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.076328+0000","last_change":"2024-04-26T20:58:43.073833+0000","last_active":"2024-04-26T20:58:43.076328+0000","last_peered":"2024-04-26T20:58:43.076328+0000","last_clean":"2024-04-26T20:58:43.076328+0000","last_became_active":"2024-04-26T20:58:43.070958+0000","last_became_peered":"2024-04-26T20:58:43.070958+0000","last_unstale":"2024-04-26T20:58:43.076328+0000","last_undegraded":"2024-04-26T20:58:43.076328+0000","last_fullsized":"2024-04-26T20:58:43.076328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:23.656127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.057716+0000","last_change":"2024-04-26T20:58:43.057716+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.057716+0000","last_undegraded":"2024-04-26T20:58:43.057716+0000","last_fullsized":"2024-04-26T20:58:43.057716+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074245+0000","last_change":"2024-04-26T20:58:43.072680+0000","last_active":"2024-04-26T20:58:43.074245+0000","last_peered":"2024-04-26T20:58:43.074245+0000","last_clean":"2024-04-26T20:58:43.074245+0000","last_became_active":"2024-04-26T20:58:43.072369+0000","last_became_peered":"2024-04-26T20:58:43.072369+0000","last_unstale":"2024-04-26T20:58:43.074245+0000","last_undegraded":"2024-04-26T20:58:43.074245+0000","last_fullsized":"2024-04-26T20:58:43.074245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:30.432055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055248+0000","last_change":"2024-04-26T20:58:43.055248+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055248+0000","last_undegraded":"2024-04-26T20:58:43.055248+0000","last_fullsized":"2024-04-26T20:58:43.055248+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074948+0000","last_change":"2024-04-26T20:58:43.070599+0000","last_active":"2024-04-26T20:58:43.074948+0000","last_peered":"2024-04-26T20:58:43.074948+0000","last_clean":"2024-04-26T20:58:43.074948+0000","last_became_active":"2024-04-26T20:58:43.070312+0000","last_became_peered":"2024-04-26T20:58:43.070312+0000","last_unstale":"2024-04-26T20:58:43.074948+0000","last_undegraded":"2024-04-26T20:58:43.074948+0000","last_fullsized":"2024-04-26T20:58:43.074948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:41.312188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.045079+0000","last_change":"2024-04-26T20:58:43.045079+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.045079+0000","last_undegraded":"2024-04-26T20:58:43.045079+0000","last_fullsized":"2024-04-26T20:58:43.045079+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.068273+0000","last_change":"2024-04-26T20:58:43.066495+0000","last_active":"2024-04-26T20:58:43.068273+0000","last_peered":"2024-04-26T20:58:43.068273+0000","last_clean":"2024-04-26T20:58:43.068273+0000","last_became_active":"2024-04-26T20:58:43.066214+0000","last_became_peered":"2024-04-26T20:58:43.066214+0000","last_unstale":"2024-04-26T20:58:43.068273+0000","last_undegraded":"2024-04-26T20:58:43.068273+0000","last_fullsized":"2024-04-26T20:58:43.068273+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:02.546662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055253+0000","last_change":"2024-04-26T20:58:43.055253+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055253+0000","last_undegraded":"2024-04-26T20:58:43.055253+0000","last_fullsized":"2024-04-26T20:58:43.055253+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.071565+0000","last_change":"2024-04-26T20:58:43.067727+0000","last_active":"2024-04-26T20:58:43.071565+0000","last_peered":"2024-04-26T20:58:43.071565+0000","last_clean":"2024-04-26T20:58:43.071565+0000","last_became_active":"2024-04-26T20:58:43.066679+0000","last_became_peered":"2024-04-26T20:58:43.066679+0000","last_unstale":"2024-04-26T20:58:43.071565+0000","last_undegraded":"2024-04-26T20:58:43.071565+0000","last_fullsized":"2024-04-26T20:58:43.071565+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:28.654953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.050736+0000","last_change":"2024-04-26T20:58:43.050736+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.050736+0000","last_undegraded":"2024-04-26T20:58:43.050736+0000","last_fullsized":"2024-04-26T20:58:43.050736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.066307+0000","last_change":"2024-04-26T20:58:43.057010+0000","last_active":"2024-04-26T20:58:43.066307+0000","last_peered":"2024-04-26T20:58:43.066307+0000","last_clean":"2024-04-26T20:58:43.066307+0000","last_became_active":"2024-04-26T20:58:43.056562+0000","last_became_peered":"2024-04-26T20:58:43.056562+0000","last_unstale":"2024-04-26T20:58:43.066307+0000","last_undegraded":"2024-04-26T20:58:43.066307+0000","last_fullsized":"2024-04-26T20:58:43.066307+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:46.360000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.059706+0000","last_change":"2024-04-26T20:58:43.059706+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.059706+0000","last_undegraded":"2024-04-26T20:58:43.059706+0000","last_fullsized":"2024-04-26T20:58:43.059706+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.068377+0000","last_change":"2024-04-26T20:58:43.066688+0000","last_active":"2024-04-26T20:58:43.068377+0000","last_peered":"2024-04-26T20:58:43.068377+0000","last_clean":"2024-04-26T20:58:43.068377+0000","last_became_active":"2024-04-26T20:58:43.066372+0000","last_became_peered":"2024-04-26T20:58:43.066372+0000","last_unstale":"2024-04-26T20:58:43.068377+0000","last_undegraded":"2024-04-26T20:58:43.068377+0000","last_fullsized":"2024-04-26T20:58:43.068377+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:32.800607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.050460+0000","last_change":"2024-04-26T20:58:43.050460+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.050460+0000","last_undegraded":"2024-04-26T20:58:43.050460+0000","last_fullsized":"2024-04-26T20:58:43.050460+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.064945+0000","last_change":"2024-04-26T20:58:43.061818+0000","last_active":"2024-04-26T20:58:43.064945+0000","last_peered":"2024-04-26T20:58:43.064945+0000","last_clean":"2024-04-26T20:58:43.064945+0000","last_became_active":"2024-04-26T20:58:43.059103+0000","last_became_peered":"2024-04-26T20:58:43.059103+0000","last_unstale":"2024-04-26T20:58:43.064945+0000","last_undegraded":"2024-04-26T20:58:43.064945+0000","last_fullsized":"2024-04-26T20:58:43.064945+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:31.981808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.047092+0000","last_change":"2024-04-26T20:58:43.047092+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.047092+0000","last_undegraded":"2024-04-26T20:58:43.047092+0000","last_fullsized":"2024-04-26T20:58:43.047092+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.073123+0000","last_change":"2024-04-26T20:58:43.071259+0000","last_active":"2024-04-26T20:58:43.073123+0000","last_peered":"2024-04-26T20:58:43.073123+0000","last_clean":"2024-04-26T20:58:43.073123+0000","last_became_active":"2024-04-26T20:58:43.070952+0000","last_became_peered":"2024-04-26T20:58:43.070952+0000","last_unstale":"2024-04-26T20:58:43.073123+0000","last_undegraded":"2024-04-26T20:58:43.073123+0000","last_fullsized":"2024-04-26T20:58:43.073123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:39.191097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.046401+0000","last_change":"2024-04-26T20:58:43.046401+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.046401+0000","last_undegraded":"2024-04-26T20:58:43.046401+0000","last_fullsized":"2024-04-26T20:58:43.046401+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.076213+0000","last_change":"2024-04-26T20:58:43.072767+0000","last_active":"2024-04-26T20:58:43.076213+0000","last_peered":"2024-04-26T20:58:43.076213+0000","last_clean":"2024-04-26T20:58:43.076213+0000","last_became_active":"2024-04-26T20:58:43.072408+0000","last_became_peered":"2024-04-26T20:58:43.072408+0000","last_unstale":"2024-04-26T20:58:43.076213+0000","last_undegraded":"2024-04-26T20:58:43.076213+0000","last_fullsized":"2024-04-26T20:58:43.076213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:46.360000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.049406+0000","last_change":"2024-04-26T20:58:43.049406+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.049406+0000","last_undegraded":"2024-04-26T20:58:43.049406+0000","last_fullsized":"2024-04-26T20:58:43.049406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.060882+0000","last_change":"2024-04-26T20:58:43.059043+0000","last_active":"2024-04-26T20:58:43.060882+0000","last_peered":"2024-04-26T20:58:43.060882+0000","last_clean":"2024-04-26T20:58:43.060882+0000","last_became_active":"2024-04-26T20:58:43.058718+0000","last_became_peered":"2024-04-26T20:58:43.058718+0000","last_unstale":"2024-04-26T20:58:43.060882+0000","last_undegraded":"2024-04-26T20:58:43.060882+0000","last_fullsized":"2024-04-26T20:58:43.060882+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:57.286045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055956+0000","last_change":"2024-04-26T20:58:43.055956+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055956+0000","last_undegraded":"2024-04-26T20:58:43.055956+0000","last_fullsized":"2024-04-26T20:58:43.055956+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.062506+0000","last_change":"2024-04-26T20:58:43.057592+0000","last_active":"2024-04-26T20:58:43.062506+0000","last_peered":"2024-04-26T20:58:43.062506+0000","last_clean":"2024-04-26T20:58:43.062506+0000","last_became_active":"2024-04-26T20:58:43.057285+0000","last_became_peered":"2024-04-26T20:58:43.057285+0000","last_unstale":"2024-04-26T20:58:43.062506+0000","last_undegraded":"2024-04-26T20:58:43.062506+0000","last_fullsized":"2024-04-26T20:58:43.062506+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:47.781660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.054665+0000","last_change":"2024-04-26T20:58:43.054665+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.054665+0000","last_undegraded":"2024-04-26T20:58:43.054665+0000","last_fullsized":"2024-04-26T20:58:43.054665+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074564+0000","last_change":"2024-04-26T20:58:43.068260+0000","last_active":"2024-04-26T20:58:43.074564+0000","last_peered":"2024-04-26T20:58:43.074564+0000","last_clean":"2024-04-26T20:58:43.074564+0000","last_became_active":"2024-04-26T20:58:43.067852+0000","last_became_peered":"2024-04-26T20:58:43.067852+0000","last_unstale":"2024-04-26T20:58:43.074564+0000","last_undegraded":"2024-04-26T20:58:43.074564+0000","last_fullsized":"2024-04-26T20:58:43.074564+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:15.110999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.045954+0000","last_change":"2024-04-26T20:58:43.045954+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.045954+0000","last_undegraded":"2024-04-26T20:58:43.045954+0000","last_fullsized":"2024-04-26T20:58:43.045954+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.061318+0000","last_change":"2024-04-26T20:58:43.059009+0000","last_active":"2024-04-26T20:58:43.061318+0000","last_peered":"2024-04-26T20:58:43.061318+0000","last_clean":"2024-04-26T20:58:43.061318+0000","last_became_active":"2024-04-26T20:58:43.058725+0000","last_became_peered":"2024-04-26T20:58:43.058725+0000","last_unstale":"2024-04-26T20:58:43.061318+0000","last_undegraded":"2024-04-26T20:58:43.061318+0000","last_fullsized":"2024-04-26T20:58:43.061318+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:51.552587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.048146+0000","last_change":"2024-04-26T20:58:43.048146+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.048146+0000","last_undegraded":"2024-04-26T20:58:43.048146+0000","last_fullsized":"2024-04-26T20:58:43.048146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.072981+0000","last_change":"2024-04-26T20:58:43.071075+0000","last_active":"2024-04-26T20:58:43.072981+0000","last_peered":"2024-04-26T20:58:43.072981+0000","last_clean":"2024-04-26T20:58:43.072981+0000","last_became_active":"2024-04-26T20:58:43.070762+0000","last_became_peered":"2024-04-26T20:58:43.070762+0000","last_unstale":"2024-04-26T20:58:43.072981+0000","last_undegraded":"2024-04-26T20:58:43.072981+0000","last_fullsized":"2024-04-26T20:58:43.072981+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:31.981808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.039364+0000","last_change":"2024-04-26T20:58:43.039364+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.039364+0000","last_undegraded":"2024-04-26T20:58:43.039364+0000","last_fullsized":"2024-04-26T20:58:43.039364+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.053458+0000","last_change":"2024-04-26T20:58:43.048980+0000","last_active":"2024-04-26T20:58:43.053458+0000","last_peered":"2024-04-26T20:58:43.053458+0000","last_clean":"2024-04-26T20:58:43.053458+0000","last_became_active":"2024-04-26T20:58:43.048625+0000","last_became_peered":"2024-04-26T20:58:43.048625+0000","last_unstale":"2024-04-26T20:58:43.053458+0000","last_undegraded":"2024-04-26T20:58:43.053458+0000","last_fullsized":"2024-04-26T20:58:43.053458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:49:57.621037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.051926+0000","last_change":"2024-04-26T20:58:43.051926+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.051926+0000","last_undegraded":"2024-04-26T20:58:43.051926+0000","last_fullsized":"2024-04-26T20:58:43.051926+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.066737+0000","last_change":"2024-04-26T20:58:43.063230+0000","last_active":"2024-04-26T20:58:43.066737+0000","last_peered":"2024-04-26T20:58:43.066737+0000","last_clean":"2024-04-26T20:58:43.066737+0000","last_became_active":"2024-04-26T20:58:43.062868+0000","last_became_peered":"2024-04-26T20:58:43.062868+0000","last_unstale":"2024-04-26T20:58:43.066737+0000","last_undegraded":"2024-04-26T20:58:43.066737+0000","last_fullsized":"2024-04-26T20:58:43.066737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:39.191097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.064001+0000","last_change":"2024-04-26T20:58:43.064001+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.064001+0000","last_undegraded":"2024-04-26T20:58:43.064001+0000","last_fullsized":"2024-04-26T20:58:43.064001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074548+0000","last_change":"2024-04-26T20:58:43.070226+0000","last_active":"2024-04-26T20:58:43.074548+0000","last_peered":"2024-04-26T20:58:43.074548+0000","last_clean":"2024-04-26T20:58:43.074548+0000","last_became_active":"2024-04-26T20:58:43.067146+0000","last_became_peered":"2024-04-26T20:58:43.067146+0000","last_unstale":"2024-04-26T20:58:43.074548+0000","last_undegraded":"2024-04-26T20:58:43.074548+0000","last_fullsized":"2024-04-26T20:58:43.074548+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:44.310082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056614+0000","last_change":"2024-04-26T20:58:43.056614+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056614+0000","last_undegraded":"2024-04-26T20:58:43.056614+0000","last_fullsized":"2024-04-26T20:58:43.056614+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.072821+0000","last_change":"2024-04-26T20:58:43.070073+0000","last_active":"2024-04-26T20:58:43.072821+0000","last_peered":"2024-04-26T20:58:43.072821+0000","last_clean":"2024-04-26T20:58:43.072821+0000","last_became_active":"2024-04-26T20:58:43.069554+0000","last_became_peered":"2024-04-26T20:58:43.069554+0000","last_unstale":"2024-04-26T20:58:43.072821+0000","last_undegraded":"2024-04-26T20:58:43.072821+0000","last_fullsized":"2024-04-26T20:58:43.072821+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:14.194412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.045725+0000","last_change":"2024-04-26T20:58:43.045725+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.045725+0000","last_undegraded":"2024-04-26T20:58:43.045725+0000","last_fullsized":"2024-04-26T20:58:43.045725+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.068193+0000","last_change":"2024-04-26T20:58:43.066409+0000","last_active":"2024-04-26T20:58:43.068193+0000","last_peered":"2024-04-26T20:58:43.068193+0000","last_clean":"2024-04-26T20:58:43.068193+0000","last_became_active":"2024-04-26T20:58:43.065775+0000","last_became_peered":"2024-04-26T20:58:43.065775+0000","last_unstale":"2024-04-26T20:58:43.068193+0000","last_undegraded":"2024-04-26T20:58:43.068193+0000","last_fullsized":"2024-04-26T20:58:43.068193+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:41.335687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.050764+0000","last_change":"2024-04-26T20:58:43.050764+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.050764+0000","last_undegraded":"2024-04-26T20:58:43.050764+0000","last_fullsized":"2024-04-26T20:58:43.050764+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.075472+0000","last_change":"2024-04-26T20:58:43.072643+0000","last_active":"2024-04-26T20:58:43.075472+0000","last_peered":"2024-04-26T20:58:43.075472+0000","last_clean":"2024-04-26T20:58:43.075472+0000","last_became_active":"2024-04-26T20:58:43.072358+0000","last_became_peered":"2024-04-26T20:58:43.072358+0000","last_unstale":"2024-04-26T20:58:43.075472+0000","last_undegraded":"2024-04-26T20:58:43.075472+0000","last_fullsized":"2024-04-26T20:58:43.075472+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:44.341099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.038929+0000","last_change":"2024-04-26T20:58:43.038929+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.038929+0000","last_undegraded":"2024-04-26T20:58:43.038929+0000","last_fullsized":"2024-04-26T20:58:43.038929+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.072986+0000","last_change":"2024-04-26T20:58:43.070983+0000","last_active":"2024-04-26T20:58:43.072986+0000","last_peered":"2024-04-26T20:58:43.072986+0000","last_clean":"2024-04-26T20:58:43.072986+0000","last_became_active":"2024-04-26T20:58:43.070724+0000","last_became_peered":"2024-04-26T20:58:43.070724+0000","last_unstale":"2024-04-26T20:58:43.072986+0000","last_undegraded":"2024-04-26T20:58:43.072986+0000","last_fullsized":"2024-04-26T20:58:43.072986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:36.534833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.057380+0000","last_change":"2024-04-26T20:58:43.057380+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.057380+0000","last_undegraded":"2024-04-26T20:58:43.057380+0000","last_fullsized":"2024-04-26T20:58:43.057380+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074913+0000","last_change":"2024-04-26T20:58:43.069835+0000","last_active":"2024-04-26T20:58:43.074913+0000","last_peered":"2024-04-26T20:58:43.074913+0000","last_clean":"2024-04-26T20:58:43.074913+0000","last_became_active":"2024-04-26T20:58:43.069535+0000","last_became_peered":"2024-04-26T20:58:43.069535+0000","last_unstale":"2024-04-26T20:58:43.074913+0000","last_undegraded":"2024-04-26T20:58:43.074913+0000","last_fullsized":"2024-04-26T20:58:43.074913+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:45.716150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.063901+0000","last_change":"2024-04-26T20:58:43.063901+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.063901+0000","last_undegraded":"2024-04-26T20:58:43.063901+0000","last_fullsized":"2024-04-26T20:58:43.063901+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.064585+0000","last_change":"2024-04-26T20:58:43.060904+0000","last_active":"2024-04-26T20:58:43.064585+0000","last_peered":"2024-04-26T20:58:43.064585+0000","last_clean":"2024-04-26T20:58:43.064585+0000","last_became_active":"2024-04-26T20:58:43.060511+0000","last_became_peered":"2024-04-26T20:58:43.060511+0000","last_unstale":"2024-04-26T20:58:43.064585+0000","last_undegraded":"2024-04-26T20:58:43.064585+0000","last_fullsized":"2024-04-26T20:58:43.064585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:21.497125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055227+0000","last_change":"2024-04-26T20:58:43.055227+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055227+0000","last_undegraded":"2024-04-26T20:58:43.055227+0000","last_fullsized":"2024-04-26T20:58:43.055227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.079030+0000","last_change":"2024-04-26T20:58:43.075337+0000","last_active":"2024-04-26T20:58:43.079030+0000","last_peered":"2024-04-26T20:58:43.079030+0000","last_clean":"2024-04-26T20:58:43.079030+0000","last_became_active":"2024-04-26T20:58:43.075019+0000","last_became_peered":"2024-04-26T20:58:43.075019+0000","last_unstale":"2024-04-26T20:58:43.079030+0000","last_undegraded":"2024-04-26T20:58:43.079030+0000","last_fullsized":"2024-04-26T20:58:43.079030+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:44.310082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.047448+0000","last_change":"2024-04-26T20:58:43.047448+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.047448+0000","last_undegraded":"2024-04-26T20:58:43.047448+0000","last_fullsized":"2024-04-26T20:58:43.047448+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074179+0000","last_change":"2024-04-26T20:58:43.071692+0000","last_active":"2024-04-26T20:58:43.074179+0000","last_peered":"2024-04-26T20:58:43.074179+0000","last_clean":"2024-04-26T20:58:43.074179+0000","last_became_active":"2024-04-26T20:58:43.069775+0000","last_became_peered":"2024-04-26T20:58:43.069775+0000","last_unstale":"2024-04-26T20:58:43.074179+0000","last_undegraded":"2024-04-26T20:58:43.074179+0000","last_fullsized":"2024-04-26T20:58:43.074179+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.491913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.045175+0000","last_change":"2024-04-26T20:58:43.045175+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.045175+0000","last_undegraded":"2024-04-26T20:58:43.045175+0000","last_fullsized":"2024-04-26T20:58:43.045175+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.079032+0000","last_change":"2024-04-26T20:58:43.075348+0000","last_active":"2024-04-26T20:58:43.079032+0000","last_peered":"2024-04-26T20:58:43.079032+0000","last_clean":"2024-04-26T20:58:43.079032+0000","last_became_active":"2024-04-26T20:58:43.075012+0000","last_became_peered":"2024-04-26T20:58:43.075012+0000","last_unstale":"2024-04-26T20:58:43.079032+0000","last_undegraded":"2024-04-26T20:58:43.079032+0000","last_fullsized":"2024-04-26T20:58:43.079032+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:44.341099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.045208+0000","last_change":"2024-04-26T20:58:43.045208+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.045208+0000","last_undegraded":"2024-04-26T20:58:43.045208+0000","last_fullsized":"2024-04-26T20:58:43.045208+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.064494+0000","last_change":"2024-04-26T20:58:43.054765+0000","last_active":"2024-04-26T20:58:43.064494+0000","last_peered":"2024-04-26T20:58:43.064494+0000","last_clean":"2024-04-26T20:58:43.064494+0000","last_became_active":"2024-04-26T20:58:43.054455+0000","last_became_peered":"2024-04-26T20:58:43.054455+0000","last_unstale":"2024-04-26T20:58:43.064494+0000","last_undegraded":"2024-04-26T20:58:43.064494+0000","last_fullsized":"2024-04-26T20:58:43.064494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:36.534833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.062096+0000","last_change":"2024-04-26T20:58:43.062096+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.062096+0000","last_undegraded":"2024-04-26T20:58:43.062096+0000","last_fullsized":"2024-04-26T20:58:43.062096+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.067885+0000","last_change":"2024-04-26T20:58:43.065790+0000","last_active":"2024-04-26T20:58:43.067885+0000","last_peered":"2024-04-26T20:58:43.067885+0000","last_clean":"2024-04-26T20:58:43.067885+0000","last_became_active":"2024-04-26T20:58:43.065469+0000","last_became_peered":"2024-04-26T20:58:43.065469+0000","last_unstale":"2024-04-26T20:58:43.067885+0000","last_undegraded":"2024-04-26T20:58:43.067885+0000","last_fullsized":"2024-04-26T20:58:43.067885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:28.519261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.044116+0000","last_change":"2024-04-26T20:58:43.044116+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.044116+0000","last_undegraded":"2024-04-26T20:58:43.044116+0000","last_fullsized":"2024-04-26T20:58:43.044116+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.059577+0000","last_change":"2024-04-26T20:58:43.056930+0000","last_active":"2024-04-26T20:58:43.059577+0000","last_peered":"2024-04-26T20:58:43.059577+0000","last_clean":"2024-04-26T20:58:43.059577+0000","last_became_active":"2024-04-26T20:58:43.055764+0000","last_became_peered":"2024-04-26T20:58:43.055764+0000","last_unstale":"2024-04-26T20:58:43.059577+0000","last_undegraded":"2024-04-26T20:58:43.059577+0000","last_fullsized":"2024-04-26T20:58:43.059577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:14.346315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055239+0000","last_change":"2024-04-26T20:58:43.055239+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055239+0000","last_undegraded":"2024-04-26T20:58:43.055239+0000","last_fullsized":"2024-04-26T20:58:43.055239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.076095+0000","last_change":"2024-04-26T20:58:43.074219+0000","last_active":"2024-04-26T20:58:43.076095+0000","last_peered":"2024-04-26T20:58:43.076095+0000","last_clean":"2024-04-26T20:58:43.076095+0000","last_became_active":"2024-04-26T20:58:43.072029+0000","last_became_peered":"2024-04-26T20:58:43.072029+0000","last_unstale":"2024-04-26T20:58:43.076095+0000","last_undegraded":"2024-04-26T20:58:43.076095+0000","last_fullsized":"2024-04-26T20:58:43.076095+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:58.207157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":211,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.058660+0000","last_change":"2024-04-26T20:56:46.848603+0000","last_active":"2024-04-26T20:58:43.058660+0000","last_peered":"2024-04-26T20:58:43.058660+0000","last_clean":"2024-04-26T20:58:43.058660+0000","last_became_active":"2024-04-26T20:56:46.846091+0000","last_became_peered":"2024-04-26T20:56:46.846091+0000","last_unstale":"2024-04-26T20:58:43.058660+0000","last_undegraded":"2024-04-26T20:58:43.058660+0000","last_fullsized":"2024-04-26T20:58:43.058660+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_clean_scrub_stamp":"2024-04-26T20:56:45.594041+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:00.232387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":785505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":201404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":201404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":201404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":201404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":791772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:58:48.548 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:58:48.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.800+0000 7fc16de00640 1 Processor -- start 2024-04-26T20:58:48.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.800+0000 7fc16de00640 1 -- start start 2024-04-26T20:58:48.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.801+0000 7fc16de00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc1680d4700 0x7fc1680d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:48.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.801+0000 7fc16de00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1680d0710 con 0x7fc1680d5370 2024-04-26T20:58:48.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.801+0000 7fc16de00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1680d4b40 con 0x7fc16815f1b0 2024-04-26T20:58:48.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.801+0000 7fc16de00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc1680cff00 con 0x7fc1680d4700 2024-04-26T20:58:48.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.802+0000 7fc16d400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc1680d5370 0x7fc1680b80b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52956/0 (socket says 172.21.15.28:52956) 2024-04-26T20:58:48.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.802+0000 7fc16d400640 1 -- 172.21.15.28:0/2375267326 learned_addr learned my addr 172.21.15.28:0/2375267326 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:48.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.802+0000 7fc167e00640 1 --2- 172.21.15.28:0/2375267326 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc1680d4700 0x7fc1680d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:48.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.802+0000 7fc167400640 1 -- 172.21.15.28:0/2375267326 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3409378016 0 0) 0x7fc160001490 con 0x7fc1680d5370 2024-04-26T20:58:48.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.803+0000 7fc167400640 1 -- 172.21.15.28:0/2375267326 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc150003520 con 0x7fc1680d5370 2024-04-26T20:58:48.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.803+0000 7fc167400640 1 -- 172.21.15.28:0/2375267326 <== mon.1 v1:172.21.15.103:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4061314403 0 0) 0x7fc15c001110 con 0x7fc16815f1b0 2024-04-26T20:58:48.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.803+0000 7fc167400640 1 -- 172.21.15.28:0/2375267326 --> v1:172.21.15.103:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc160001490 con 0x7fc16815f1b0 2024-04-26T20:58:48.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.803+0000 7fc167400640 1 -- 172.21.15.28:0/2375267326 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4174951825 0 0) 0x7fc150003520 con 0x7fc1680d5370 2024-04-26T20:58:48.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.803+0000 7fc167400640 1 -- 172.21.15.28:0/2375267326 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc1680d4700 msgr2=0x7fc1680d4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:48.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.803+0000 7fc167400640 1 --2- 172.21.15.28:0/2375267326 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc1680d4700 0x7fc1680d4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:48.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.803+0000 7fc167400640 1 -- 172.21.15.28:0/2375267326 >> v1:172.21.15.103:6789/0 conn(0x7fc16815f1b0 legacy=0x7fc1680cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:48.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.803+0000 7fc167400640 1 -- 172.21.15.28:0/2375267326 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1680d0150 con 0x7fc1680d5370 2024-04-26T20:58:48.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.803+0000 7fc167400640 1 -- 172.21.15.28:0/2375267326 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fc160001bc0 con 0x7fc1680d5370 2024-04-26T20:58:48.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.803+0000 7fc167400640 1 -- 172.21.15.28:0/2375267326 >> v1:172.21.15.28:6789/0 conn(0x7fc1680d5370 legacy=0x7fc1680b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:48.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.803+0000 7fc167400640 1 --2- 172.21.15.28:0/2375267326 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc1680d4700 0x7fc150003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:48.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.803+0000 7fc167400640 1 --2- 172.21.15.28:0/2375267326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc150004120 0x7fc150004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:48.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.803+0000 7fc167400640 1 --2- 172.21.15.28:0/2375267326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc150004d40 0x7fc1500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:48.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.803+0000 7fc167400640 1 -- 172.21.15.28:0/2375267326 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc1680cff00 con 0x7fc150004d40 2024-04-26T20:58:48.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.803+0000 7fc167400640 1 -- 172.21.15.28:0/2375267326 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc15000bb00 con 0x7fc150004120 2024-04-26T20:58:48.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.803+0000 7fc167400640 1 -- 172.21.15.28:0/2375267326 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc15000bd00 con 0x7fc1680d4700 2024-04-26T20:58:48.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.803+0000 7fc16d400640 1 --2- 172.21.15.28:0/2375267326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc150004d40 0x7fc1500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:48.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.804+0000 7fc167e00640 1 --2- 172.21.15.28:0/2375267326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc150004120 0x7fc150004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:48.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.804+0000 7fc16ca00640 1 --2- 172.21.15.28:0/2375267326 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc1680d4700 0x7fc150003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:48.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.804+0000 7fc16d400640 1 -- 172.21.15.28:0/2375267326 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc1680d4700 msgr2=0x7fc150003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:48.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.804+0000 7fc16d400640 1 --2- 172.21.15.28:0/2375267326 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc1680d4700 0x7fc150003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:48.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.804+0000 7fc16d400640 1 -- 172.21.15.28:0/2375267326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc150004120 msgr2=0x7fc150004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:48.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.804+0000 7fc16d400640 1 --2- 172.21.15.28:0/2375267326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc150004120 0x7fc150004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:48.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.804+0000 7fc16d400640 1 -- 172.21.15.28:0/2375267326 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc150003520 con 0x7fc150004d40 2024-04-26T20:58:48.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.805+0000 7fc16d400640 1 --2- 172.21.15.28:0/2375267326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc150004d40 0x7fc1500051c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc16000a930 tx=0x7fc150005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0000ba6a5fcf779 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:48.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.805+0000 7fc167400640 1 -- 172.21.15.28:0/2375267326 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16000a550 con 0x7fc150004d40 2024-04-26T20:58:48.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.805+0000 7fc167400640 1 -- 172.21.15.28:0/2375267326 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc16002f070 con 0x7fc150004d40 2024-04-26T20:58:48.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.805+0000 7fc167400640 1 -- 172.21.15.28:0/2375267326 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16002bd90 con 0x7fc150004d40 2024-04-26T20:58:48.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.806+0000 7fc16de00640 1 -- 172.21.15.28:0/2375267326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc150004d40 msgr2=0x7fc1500051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:48.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.806+0000 7fc16de00640 1 --2- 172.21.15.28:0/2375267326 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc150004d40 0x7fc1500051c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc16000a930 tx=0x7fc150005830 comp rx=0 tx=0).stop 2024-04-26T20:58:48.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.806+0000 7fc16d400640 1 -- 172.21.15.28:0/2375267326 reap_dead start 2024-04-26T20:58:48.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.806+0000 7fc16de00640 1 -- 172.21.15.28:0/2375267326 shutdown_connections 2024-04-26T20:58:48.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.806+0000 7fc16de00640 1 -- 172.21.15.28:0/2375267326 >> 172.21.15.28:0/2375267326 conn(0x7fc16806c770 msgr2=0x7fc16806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:48.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.806+0000 7fc16de00640 1 -- 172.21.15.28:0/2375267326 shutdown_connections 2024-04-26T20:58:48.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.806+0000 7fc16de00640 1 -- 172.21.15.28:0/2375267326 wait complete. 2024-04-26T20:58:48.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.808+0000 7fc16de00640 1 Processor -- start 2024-04-26T20:58:48.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.808+0000 7fc16de00640 1 -- start start 2024-04-26T20:58:48.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.809+0000 7fc16de00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc150004d40 0x7fc1680c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:48.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.809+0000 7fc16de00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc1681f4dd0 0x7fc1681573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:48.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.809+0000 7fc16de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1680c9480 0x7fc16814a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:48.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.809+0000 7fc16de00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc1680d3f00 con 0x7fc1680c9480 2024-04-26T20:58:48.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.809+0000 7fc16de00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc1680d4270 con 0x7fc150004d40 2024-04-26T20:58:48.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.809+0000 7fc16de00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc16815f590 con 0x7fc1681f4dd0 2024-04-26T20:58:48.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.810+0000 7fc16d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1680c9480 0x7fc16814a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:48.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.810+0000 7fc16d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1680c9480 0x7fc16814a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53962/0 (socket says 172.21.15.28:53962) 2024-04-26T20:58:48.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.810+0000 7fc16d400640 1 -- 172.21.15.28:0/1415276951 learned_addr learned my addr 172.21.15.28:0/1415276951 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:48.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.810+0000 7fc16ca00640 1 --2- 172.21.15.28:0/1415276951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc150004d40 0x7fc1680c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:48.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.810+0000 7fc167e00640 1 --2- 172.21.15.28:0/1415276951 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc1681f4dd0 0x7fc1681573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:48.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.810+0000 7fc16d400640 1 -- 172.21.15.28:0/1415276951 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc1681f4dd0 msgr2=0x7fc1681573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:48.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.810+0000 7fc16d400640 1 --2- 172.21.15.28:0/1415276951 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc1681f4dd0 0x7fc1681573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:48.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.810+0000 7fc16d400640 1 -- 172.21.15.28:0/1415276951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc150004d40 msgr2=0x7fc1680c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:48.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.810+0000 7fc16d400640 1 --2- 172.21.15.28:0/1415276951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc150004d40 0x7fc1680c8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:48.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.810+0000 7fc16d400640 1 -- 172.21.15.28:0/1415276951 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc168157900 con 0x7fc1680c9480 2024-04-26T20:58:48.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.811+0000 7fc16d400640 1 --2- 172.21.15.28:0/1415276951 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1680c9480 0x7fc16814a120 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc16002b5c0 tx=0x7fc160001ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:48.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.811+0000 7fc164c00640 1 -- 172.21.15.28:0/1415276951 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc160044070 con 0x7fc1680c9480 2024-04-26T20:58:48.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.811+0000 7fc164c00640 1 -- 172.21.15.28:0/1415276951 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc160001c30 con 0x7fc1680c9480 2024-04-26T20:58:48.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.811+0000 7fc164c00640 1 -- 172.21.15.28:0/1415276951 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc16000a550 con 0x7fc1680c9480 2024-04-26T20:58:48.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.811+0000 7fc16de00640 1 -- 172.21.15.28:0/1415276951 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc16815a010 con 0x7fc1680c9480 2024-04-26T20:58:48.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.811+0000 7fc16de00640 1 -- 172.21.15.28:0/1415276951 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc16815a2b0 con 0x7fc1680c9480 2024-04-26T20:58:48.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.812+0000 7fc16de00640 1 -- 172.21.15.28:0/1415276951 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc16806a360 con 0x7fc1680c9480 2024-04-26T20:58:48.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.813+0000 7fc164c00640 1 -- 172.21.15.28:0/1415276951 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc160047070 con 0x7fc1680c9480 2024-04-26T20:58:48.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.814+0000 7fc164c00640 1 --2- 172.21.15.28:0/1415276951 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc14806c8f0 0x7fc14806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:48.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.814+0000 7fc164c00640 1 -- 172.21.15.28:0/1415276951 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fc1600abe10 con 0x7fc1680c9480 2024-04-26T20:58:48.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.815+0000 7fc16ca00640 1 --2- 172.21.15.28:0/1415276951 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc14806c8f0 0x7fc14806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:48.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.815+0000 7fc16ca00640 1 --2- 172.21.15.28:0/1415276951 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc14806c8f0 0x7fc14806edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0012b0 tx=0x7fc15c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:48.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:48.819+0000 7fc164c00640 1 -- 172.21.15.28:0/1415276951 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc1600af030 con 0x7fc1680c9480 2024-04-26T20:58:49.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.019+0000 7fc16de00640 1 -- 172.21.15.28:0/1415276951 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc1680b4170 con 0x7fc14806c8f0 2024-04-26T20:58:49.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.036+0000 7fc164c00640 1 -- 172.21.15.28:0/1415276951 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246636 (secure 0 0 0) 0x7fc1680b4170 con 0x7fc14806c8f0 2024-04-26T20:58:49.041 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:49.042 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T20:58:49.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.042+0000 7fc16de00640 1 -- 172.21.15.28:0/1415276951 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc14806c8f0 msgr2=0x7fc14806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:49.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.042+0000 7fc16de00640 1 --2- 172.21.15.28:0/1415276951 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc14806c8f0 0x7fc14806edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc15c0012b0 tx=0x7fc15c009040 comp rx=0 tx=0).stop 2024-04-26T20:58:49.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.042+0000 7fc16de00640 1 -- 172.21.15.28:0/1415276951 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1680c9480 msgr2=0x7fc16814a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:49.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.042+0000 7fc16de00640 1 --2- 172.21.15.28:0/1415276951 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1680c9480 0x7fc16814a120 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc16002b5c0 tx=0x7fc160001ec0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.042+0000 7fc16de00640 1 -- 172.21.15.28:0/1415276951 shutdown_connections 2024-04-26T20:58:49.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.042+0000 7fc16de00640 1 --2- 172.21.15.28:0/1415276951 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc14806c8f0 0x7fc14806edb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.042+0000 7fc16de00640 1 --2- 172.21.15.28:0/1415276951 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1680c9480 0x7fc16814a120 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.042+0000 7fc16de00640 1 --2- 172.21.15.28:0/1415276951 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc1681f4dd0 0x7fc1681573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.042+0000 7fc16de00640 1 --2- 172.21.15.28:0/1415276951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc150004d40 0x7fc1680c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.042+0000 7fc16de00640 1 -- 172.21.15.28:0/1415276951 >> 172.21.15.28:0/1415276951 conn(0x7fc16806c770 msgr2=0x7fc16815f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:49.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.043+0000 7fc16de00640 1 -- 172.21.15.28:0/1415276951 shutdown_connections 2024-04-26T20:58:49.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.043+0000 7fc16de00640 1 -- 172.21.15.28:0/1415276951 wait complete. 2024-04-26T20:58:49.063 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-04-26T20:58:48.741891+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237876,"kb_used_data":5060,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736844,"statfs":{"total":773094113280,"available":772850528256,"internally_reserved":0,"allocated":5181440,"data_stored":2773167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001436"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055951+0000","last_change":"2024-04-26T20:58:43.055951+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055951+0000","last_undegraded":"2024-04-26T20:58:43.055951+0000","last_fullsized":"2024-04-26T20:58:43.055951+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.052235+0000","last_change":"2024-04-26T20:58:43.052235+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.052235+0000","last_undegraded":"2024-04-26T20:58:43.052235+0000","last_fullsized":"2024-04-26T20:58:43.052235+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.048628+0000","last_change":"2024-04-26T20:58:43.048628+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.048628+0000","last_undegraded":"2024-04-26T20:58:43.048628+0000","last_fullsized":"2024-04-26T20:58:43.048628+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056479+0000","last_change":"2024-04-26T20:58:43.056479+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056479+0000","last_undegraded":"2024-04-26T20:58:43.056479+0000","last_fullsized":"2024-04-26T20:58:43.056479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056437+0000","last_change":"2024-04-26T20:58:43.056437+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056437+0000","last_undegraded":"2024-04-26T20:58:43.056437+0000","last_fullsized":"2024-04-26T20:58:43.056437+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.048264+0000","last_change":"2024-04-26T20:58:43.048264+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.048264+0000","last_undegraded":"2024-04-26T20:58:43.048264+0000","last_fullsized":"2024-04-26T20:58:43.048264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.049255+0000","last_change":"2024-04-26T20:58:43.049255+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.049255+0000","last_undegraded":"2024-04-26T20:58:43.049255+0000","last_fullsized":"2024-04-26T20:58:43.049255+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.051206+0000","last_change":"2024-04-26T20:58:43.051206+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.051206+0000","last_undegraded":"2024-04-26T20:58:43.051206+0000","last_fullsized":"2024-04-26T20:58:43.051206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.053178+0000","last_change":"2024-04-26T20:58:43.053178+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.053178+0000","last_undegraded":"2024-04-26T20:58:43.053178+0000","last_fullsized":"2024-04-26T20:58:43.053178+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056034+0000","last_change":"2024-04-26T20:58:43.056034+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056034+0000","last_undegraded":"2024-04-26T20:58:43.056034+0000","last_fullsized":"2024-04-26T20:58:43.056034+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.052135+0000","last_change":"2024-04-26T20:58:43.052135+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.052135+0000","last_undegraded":"2024-04-26T20:58:43.052135+0000","last_fullsized":"2024-04-26T20:58:43.052135+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.060146+0000","last_change":"2024-04-26T20:58:43.060146+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.060146+0000","last_undegraded":"2024-04-26T20:58:43.060146+0000","last_fullsized":"2024-04-26T20:58:43.060146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.038802+0000","last_change":"2024-04-26T20:58:43.038802+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.038802+0000","last_undegraded":"2024-04-26T20:58:43.038802+0000","last_fullsized":"2024-04-26T20:58:43.038802+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.051120+0000","last_change":"2024-04-26T20:58:43.051120+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.051120+0000","last_undegraded":"2024-04-26T20:58:43.051120+0000","last_fullsized":"2024-04-26T20:58:43.051120+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.049339+0000","last_change":"2024-04-26T20:58:43.049339+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.049339+0000","last_undegraded":"2024-04-26T20:58:43.049339+0000","last_fullsized":"2024-04-26T20:58:43.049339+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.061785+0000","last_change":"2024-04-26T20:58:43.061785+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.061785+0000","last_undegraded":"2024-04-26T20:58:43.061785+0000","last_fullsized":"2024-04-26T20:58:43.061785+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.049592+0000","last_change":"2024-04-26T20:58:43.049592+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.049592+0000","last_undegraded":"2024-04-26T20:58:43.049592+0000","last_fullsized":"2024-04-26T20:58:43.049592+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.051921+0000","last_change":"2024-04-26T20:58:43.051921+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.051921+0000","last_undegraded":"2024-04-26T20:58:43.051921+0000","last_fullsized":"2024-04-26T20:58:43.051921+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056057+0000","last_change":"2024-04-26T20:58:43.056057+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056057+0000","last_undegraded":"2024-04-26T20:58:43.056057+0000","last_fullsized":"2024-04-26T20:58:43.056057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.039248+0000","last_change":"2024-04-26T20:58:43.039248+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.039248+0000","last_undegraded":"2024-04-26T20:58:43.039248+0000","last_fullsized":"2024-04-26T20:58:43.039248+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.047349+0000","last_change":"2024-04-26T20:58:43.047349+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.047349+0000","last_undegraded":"2024-04-26T20:58:43.047349+0000","last_fullsized":"2024-04-26T20:58:43.047349+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.046758+0000","last_change":"2024-04-26T20:58:43.046758+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.046758+0000","last_undegraded":"2024-04-26T20:58:43.046758+0000","last_fullsized":"2024-04-26T20:58:43.046758+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.059090+0000","last_change":"2024-04-26T20:58:43.059090+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.059090+0000","last_undegraded":"2024-04-26T20:58:43.059090+0000","last_fullsized":"2024-04-26T20:58:43.059090+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.060755+0000","last_change":"2024-04-26T20:58:43.060755+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.060755+0000","last_undegraded":"2024-04-26T20:58:43.060755+0000","last_fullsized":"2024-04-26T20:58:43.060755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.064938+0000","last_change":"2024-04-26T20:58:43.064938+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.064938+0000","last_undegraded":"2024-04-26T20:58:43.064938+0000","last_fullsized":"2024-04-26T20:58:43.064938+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.040818+0000","last_change":"2024-04-26T20:58:43.040818+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.040818+0000","last_undegraded":"2024-04-26T20:58:43.040818+0000","last_fullsized":"2024-04-26T20:58:43.040818+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.046134+0000","last_change":"2024-04-26T20:58:43.046134+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.046134+0000","last_undegraded":"2024-04-26T20:58:43.046134+0000","last_fullsized":"2024-04-26T20:58:43.046134+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.058202+0000","last_change":"2024-04-26T20:58:43.058202+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.058202+0000","last_undegraded":"2024-04-26T20:58:43.058202+0000","last_fullsized":"2024-04-26T20:58:43.058202+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056143+0000","last_change":"2024-04-26T20:58:43.056143+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056143+0000","last_undegraded":"2024-04-26T20:58:43.056143+0000","last_fullsized":"2024-04-26T20:58:43.056143+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.047071+0000","last_change":"2024-04-26T20:58:43.047071+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.047071+0000","last_undegraded":"2024-04-26T20:58:43.047071+0000","last_fullsized":"2024-04-26T20:58:43.047071+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.047975+0000","last_change":"2024-04-26T20:58:43.047975+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.047975+0000","last_undegraded":"2024-04-26T20:58:43.047975+0000","last_fullsized":"2024-04-26T20:58:43.047975+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.040702+0000","last_change":"2024-04-26T20:58:43.040702+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.040702+0000","last_undegraded":"2024-04-26T20:58:43.040702+0000","last_fullsized":"2024-04-26T20:58:43.040702+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.053161+0000","last_change":"2024-04-26T20:58:43.053161+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.053161+0000","last_undegraded":"2024-04-26T20:58:43.053161+0000","last_fullsized":"2024-04-26T20:58:43.053161+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.072931+0000","last_change":"2024-04-26T20:58:43.070204+0000","last_active":"2024-04-26T20:58:43.072931+0000","last_peered":"2024-04-26T20:58:43.072931+0000","last_clean":"2024-04-26T20:58:43.072931+0000","last_became_active":"2024-04-26T20:58:43.069791+0000","last_became_peered":"2024-04-26T20:58:43.069791+0000","last_unstale":"2024-04-26T20:58:43.072931+0000","last_undegraded":"2024-04-26T20:58:43.072931+0000","last_fullsized":"2024-04-26T20:58:43.072931+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:21.679330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.064417+0000","last_change":"2024-04-26T20:58:43.064417+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.064417+0000","last_undegraded":"2024-04-26T20:58:43.064417+0000","last_fullsized":"2024-04-26T20:58:43.064417+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.076328+0000","last_change":"2024-04-26T20:58:43.073833+0000","last_active":"2024-04-26T20:58:43.076328+0000","last_peered":"2024-04-26T20:58:43.076328+0000","last_clean":"2024-04-26T20:58:43.076328+0000","last_became_active":"2024-04-26T20:58:43.070958+0000","last_became_peered":"2024-04-26T20:58:43.070958+0000","last_unstale":"2024-04-26T20:58:43.076328+0000","last_undegraded":"2024-04-26T20:58:43.076328+0000","last_fullsized":"2024-04-26T20:58:43.076328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:23.656127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.057716+0000","last_change":"2024-04-26T20:58:43.057716+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.057716+0000","last_undegraded":"2024-04-26T20:58:43.057716+0000","last_fullsized":"2024-04-26T20:58:43.057716+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074245+0000","last_change":"2024-04-26T20:58:43.072680+0000","last_active":"2024-04-26T20:58:43.074245+0000","last_peered":"2024-04-26T20:58:43.074245+0000","last_clean":"2024-04-26T20:58:43.074245+0000","last_became_active":"2024-04-26T20:58:43.072369+0000","last_became_peered":"2024-04-26T20:58:43.072369+0000","last_unstale":"2024-04-26T20:58:43.074245+0000","last_undegraded":"2024-04-26T20:58:43.074245+0000","last_fullsized":"2024-04-26T20:58:43.074245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:30.432055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055248+0000","last_change":"2024-04-26T20:58:43.055248+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055248+0000","last_undegraded":"2024-04-26T20:58:43.055248+0000","last_fullsized":"2024-04-26T20:58:43.055248+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074948+0000","last_change":"2024-04-26T20:58:43.070599+0000","last_active":"2024-04-26T20:58:43.074948+0000","last_peered":"2024-04-26T20:58:43.074948+0000","last_clean":"2024-04-26T20:58:43.074948+0000","last_became_active":"2024-04-26T20:58:43.070312+0000","last_became_peered":"2024-04-26T20:58:43.070312+0000","last_unstale":"2024-04-26T20:58:43.074948+0000","last_undegraded":"2024-04-26T20:58:43.074948+0000","last_fullsized":"2024-04-26T20:58:43.074948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:41.312188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.045079+0000","last_change":"2024-04-26T20:58:43.045079+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.045079+0000","last_undegraded":"2024-04-26T20:58:43.045079+0000","last_fullsized":"2024-04-26T20:58:43.045079+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.068273+0000","last_change":"2024-04-26T20:58:43.066495+0000","last_active":"2024-04-26T20:58:43.068273+0000","last_peered":"2024-04-26T20:58:43.068273+0000","last_clean":"2024-04-26T20:58:43.068273+0000","last_became_active":"2024-04-26T20:58:43.066214+0000","last_became_peered":"2024-04-26T20:58:43.066214+0000","last_unstale":"2024-04-26T20:58:43.068273+0000","last_undegraded":"2024-04-26T20:58:43.068273+0000","last_fullsized":"2024-04-26T20:58:43.068273+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:02.546662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055253+0000","last_change":"2024-04-26T20:58:43.055253+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055253+0000","last_undegraded":"2024-04-26T20:58:43.055253+0000","last_fullsized":"2024-04-26T20:58:43.055253+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.071565+0000","last_change":"2024-04-26T20:58:43.067727+0000","last_active":"2024-04-26T20:58:43.071565+0000","last_peered":"2024-04-26T20:58:43.071565+0000","last_clean":"2024-04-26T20:58:43.071565+0000","last_became_active":"2024-04-26T20:58:43.066679+0000","last_became_peered":"2024-04-26T20:58:43.066679+0000","last_unstale":"2024-04-26T20:58:43.071565+0000","last_undegraded":"2024-04-26T20:58:43.071565+0000","last_fullsized":"2024-04-26T20:58:43.071565+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:28.654953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.050736+0000","last_change":"2024-04-26T20:58:43.050736+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.050736+0000","last_undegraded":"2024-04-26T20:58:43.050736+0000","last_fullsized":"2024-04-26T20:58:43.050736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.066307+0000","last_change":"2024-04-26T20:58:43.057010+0000","last_active":"2024-04-26T20:58:43.066307+0000","last_peered":"2024-04-26T20:58:43.066307+0000","last_clean":"2024-04-26T20:58:43.066307+0000","last_became_active":"2024-04-26T20:58:43.056562+0000","last_became_peered":"2024-04-26T20:58:43.056562+0000","last_unstale":"2024-04-26T20:58:43.066307+0000","last_undegraded":"2024-04-26T20:58:43.066307+0000","last_fullsized":"2024-04-26T20:58:43.066307+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:46.360000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.059706+0000","last_change":"2024-04-26T20:58:43.059706+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.059706+0000","last_undegraded":"2024-04-26T20:58:43.059706+0000","last_fullsized":"2024-04-26T20:58:43.059706+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.068377+0000","last_change":"2024-04-26T20:58:43.066688+0000","last_active":"2024-04-26T20:58:43.068377+0000","last_peered":"2024-04-26T20:58:43.068377+0000","last_clean":"2024-04-26T20:58:43.068377+0000","last_became_active":"2024-04-26T20:58:43.066372+0000","last_became_peered":"2024-04-26T20:58:43.066372+0000","last_unstale":"2024-04-26T20:58:43.068377+0000","last_undegraded":"2024-04-26T20:58:43.068377+0000","last_fullsized":"2024-04-26T20:58:43.068377+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:32.800607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.050460+0000","last_change":"2024-04-26T20:58:43.050460+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.050460+0000","last_undegraded":"2024-04-26T20:58:43.050460+0000","last_fullsized":"2024-04-26T20:58:43.050460+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.064945+0000","last_change":"2024-04-26T20:58:43.061818+0000","last_active":"2024-04-26T20:58:43.064945+0000","last_peered":"2024-04-26T20:58:43.064945+0000","last_clean":"2024-04-26T20:58:43.064945+0000","last_became_active":"2024-04-26T20:58:43.059103+0000","last_became_peered":"2024-04-26T20:58:43.059103+0000","last_unstale":"2024-04-26T20:58:43.064945+0000","last_undegraded":"2024-04-26T20:58:43.064945+0000","last_fullsized":"2024-04-26T20:58:43.064945+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:31.981808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.047092+0000","last_change":"2024-04-26T20:58:43.047092+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.047092+0000","last_undegraded":"2024-04-26T20:58:43.047092+0000","last_fullsized":"2024-04-26T20:58:43.047092+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.073123+0000","last_change":"2024-04-26T20:58:43.071259+0000","last_active":"2024-04-26T20:58:43.073123+0000","last_peered":"2024-04-26T20:58:43.073123+0000","last_clean":"2024-04-26T20:58:43.073123+0000","last_became_active":"2024-04-26T20:58:43.070952+0000","last_became_peered":"2024-04-26T20:58:43.070952+0000","last_unstale":"2024-04-26T20:58:43.073123+0000","last_undegraded":"2024-04-26T20:58:43.073123+0000","last_fullsized":"2024-04-26T20:58:43.073123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:39.191097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.046401+0000","last_change":"2024-04-26T20:58:43.046401+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.046401+0000","last_undegraded":"2024-04-26T20:58:43.046401+0000","last_fullsized":"2024-04-26T20:58:43.046401+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.076213+0000","last_change":"2024-04-26T20:58:43.072767+0000","last_active":"2024-04-26T20:58:43.076213+0000","last_peered":"2024-04-26T20:58:43.076213+0000","last_clean":"2024-04-26T20:58:43.076213+0000","last_became_active":"2024-04-26T20:58:43.072408+0000","last_became_peered":"2024-04-26T20:58:43.072408+0000","last_unstale":"2024-04-26T20:58:43.076213+0000","last_undegraded":"2024-04-26T20:58:43.076213+0000","last_fullsized":"2024-04-26T20:58:43.076213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:46.360000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.049406+0000","last_change":"2024-04-26T20:58:43.049406+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.049406+0000","last_undegraded":"2024-04-26T20:58:43.049406+0000","last_fullsized":"2024-04-26T20:58:43.049406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.060882+0000","last_change":"2024-04-26T20:58:43.059043+0000","last_active":"2024-04-26T20:58:43.060882+0000","last_peered":"2024-04-26T20:58:43.060882+0000","last_clean":"2024-04-26T20:58:43.060882+0000","last_became_active":"2024-04-26T20:58:43.058718+0000","last_became_peered":"2024-04-26T20:58:43.058718+0000","last_unstale":"2024-04-26T20:58:43.060882+0000","last_undegraded":"2024-04-26T20:58:43.060882+0000","last_fullsized":"2024-04-26T20:58:43.060882+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:57.286045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055956+0000","last_change":"2024-04-26T20:58:43.055956+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055956+0000","last_undegraded":"2024-04-26T20:58:43.055956+0000","last_fullsized":"2024-04-26T20:58:43.055956+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.062506+0000","last_change":"2024-04-26T20:58:43.057592+0000","last_active":"2024-04-26T20:58:43.062506+0000","last_peered":"2024-04-26T20:58:43.062506+0000","last_clean":"2024-04-26T20:58:43.062506+0000","last_became_active":"2024-04-26T20:58:43.057285+0000","last_became_peered":"2024-04-26T20:58:43.057285+0000","last_unstale":"2024-04-26T20:58:43.062506+0000","last_undegraded":"2024-04-26T20:58:43.062506+0000","last_fullsized":"2024-04-26T20:58:43.062506+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:47.781660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.054665+0000","last_change":"2024-04-26T20:58:43.054665+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.054665+0000","last_undegraded":"2024-04-26T20:58:43.054665+0000","last_fullsized":"2024-04-26T20:58:43.054665+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074564+0000","last_change":"2024-04-26T20:58:43.068260+0000","last_active":"2024-04-26T20:58:43.074564+0000","last_peered":"2024-04-26T20:58:43.074564+0000","last_clean":"2024-04-26T20:58:43.074564+0000","last_became_active":"2024-04-26T20:58:43.067852+0000","last_became_peered":"2024-04-26T20:58:43.067852+0000","last_unstale":"2024-04-26T20:58:43.074564+0000","last_undegraded":"2024-04-26T20:58:43.074564+0000","last_fullsized":"2024-04-26T20:58:43.074564+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:15.110999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.045954+0000","last_change":"2024-04-26T20:58:43.045954+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.045954+0000","last_undegraded":"2024-04-26T20:58:43.045954+0000","last_fullsized":"2024-04-26T20:58:43.045954+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.061318+0000","last_change":"2024-04-26T20:58:43.059009+0000","last_active":"2024-04-26T20:58:43.061318+0000","last_peered":"2024-04-26T20:58:43.061318+0000","last_clean":"2024-04-26T20:58:43.061318+0000","last_became_active":"2024-04-26T20:58:43.058725+0000","last_became_peered":"2024-04-26T20:58:43.058725+0000","last_unstale":"2024-04-26T20:58:43.061318+0000","last_undegraded":"2024-04-26T20:58:43.061318+0000","last_fullsized":"2024-04-26T20:58:43.061318+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:51.552587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.048146+0000","last_change":"2024-04-26T20:58:43.048146+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.048146+0000","last_undegraded":"2024-04-26T20:58:43.048146+0000","last_fullsized":"2024-04-26T20:58:43.048146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.072981+0000","last_change":"2024-04-26T20:58:43.071075+0000","last_active":"2024-04-26T20:58:43.072981+0000","last_peered":"2024-04-26T20:58:43.072981+0000","last_clean":"2024-04-26T20:58:43.072981+0000","last_became_active":"2024-04-26T20:58:43.070762+0000","last_became_peered":"2024-04-26T20:58:43.070762+0000","last_unstale":"2024-04-26T20:58:43.072981+0000","last_undegraded":"2024-04-26T20:58:43.072981+0000","last_fullsized":"2024-04-26T20:58:43.072981+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:31.981808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.039364+0000","last_change":"2024-04-26T20:58:43.039364+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.039364+0000","last_undegraded":"2024-04-26T20:58:43.039364+0000","last_fullsized":"2024-04-26T20:58:43.039364+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.053458+0000","last_change":"2024-04-26T20:58:43.048980+0000","last_active":"2024-04-26T20:58:43.053458+0000","last_peered":"2024-04-26T20:58:43.053458+0000","last_clean":"2024-04-26T20:58:43.053458+0000","last_became_active":"2024-04-26T20:58:43.048625+0000","last_became_peered":"2024-04-26T20:58:43.048625+0000","last_unstale":"2024-04-26T20:58:43.053458+0000","last_undegraded":"2024-04-26T20:58:43.053458+0000","last_fullsized":"2024-04-26T20:58:43.053458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:49:57.621037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.051926+0000","last_change":"2024-04-26T20:58:43.051926+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.051926+0000","last_undegraded":"2024-04-26T20:58:43.051926+0000","last_fullsized":"2024-04-26T20:58:43.051926+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.066737+0000","last_change":"2024-04-26T20:58:43.063230+0000","last_active":"2024-04-26T20:58:43.066737+0000","last_peered":"2024-04-26T20:58:43.066737+0000","last_clean":"2024-04-26T20:58:43.066737+0000","last_became_active":"2024-04-26T20:58:43.062868+0000","last_became_peered":"2024-04-26T20:58:43.062868+0000","last_unstale":"2024-04-26T20:58:43.066737+0000","last_undegraded":"2024-04-26T20:58:43.066737+0000","last_fullsized":"2024-04-26T20:58:43.066737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:39.191097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.064001+0000","last_change":"2024-04-26T20:58:43.064001+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.064001+0000","last_undegraded":"2024-04-26T20:58:43.064001+0000","last_fullsized":"2024-04-26T20:58:43.064001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074548+0000","last_change":"2024-04-26T20:58:43.070226+0000","last_active":"2024-04-26T20:58:43.074548+0000","last_peered":"2024-04-26T20:58:43.074548+0000","last_clean":"2024-04-26T20:58:43.074548+0000","last_became_active":"2024-04-26T20:58:43.067146+0000","last_became_peered":"2024-04-26T20:58:43.067146+0000","last_unstale":"2024-04-26T20:58:43.074548+0000","last_undegraded":"2024-04-26T20:58:43.074548+0000","last_fullsized":"2024-04-26T20:58:43.074548+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:44.310082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056614+0000","last_change":"2024-04-26T20:58:43.056614+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056614+0000","last_undegraded":"2024-04-26T20:58:43.056614+0000","last_fullsized":"2024-04-26T20:58:43.056614+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.072821+0000","last_change":"2024-04-26T20:58:43.070073+0000","last_active":"2024-04-26T20:58:43.072821+0000","last_peered":"2024-04-26T20:58:43.072821+0000","last_clean":"2024-04-26T20:58:43.072821+0000","last_became_active":"2024-04-26T20:58:43.069554+0000","last_became_peered":"2024-04-26T20:58:43.069554+0000","last_unstale":"2024-04-26T20:58:43.072821+0000","last_undegraded":"2024-04-26T20:58:43.072821+0000","last_fullsized":"2024-04-26T20:58:43.072821+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:14.194412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.045725+0000","last_change":"2024-04-26T20:58:43.045725+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.045725+0000","last_undegraded":"2024-04-26T20:58:43.045725+0000","last_fullsized":"2024-04-26T20:58:43.045725+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.068193+0000","last_change":"2024-04-26T20:58:43.066409+0000","last_active":"2024-04-26T20:58:43.068193+0000","last_peered":"2024-04-26T20:58:43.068193+0000","last_clean":"2024-04-26T20:58:43.068193+0000","last_became_active":"2024-04-26T20:58:43.065775+0000","last_became_peered":"2024-04-26T20:58:43.065775+0000","last_unstale":"2024-04-26T20:58:43.068193+0000","last_undegraded":"2024-04-26T20:58:43.068193+0000","last_fullsized":"2024-04-26T20:58:43.068193+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:41.335687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.050764+0000","last_change":"2024-04-26T20:58:43.050764+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.050764+0000","last_undegraded":"2024-04-26T20:58:43.050764+0000","last_fullsized":"2024-04-26T20:58:43.050764+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.075472+0000","last_change":"2024-04-26T20:58:43.072643+0000","last_active":"2024-04-26T20:58:43.075472+0000","last_peered":"2024-04-26T20:58:43.075472+0000","last_clean":"2024-04-26T20:58:43.075472+0000","last_became_active":"2024-04-26T20:58:43.072358+0000","last_became_peered":"2024-04-26T20:58:43.072358+0000","last_unstale":"2024-04-26T20:58:43.075472+0000","last_undegraded":"2024-04-26T20:58:43.075472+0000","last_fullsized":"2024-04-26T20:58:43.075472+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:44.341099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.038929+0000","last_change":"2024-04-26T20:58:43.038929+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.038929+0000","last_undegraded":"2024-04-26T20:58:43.038929+0000","last_fullsized":"2024-04-26T20:58:43.038929+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.072986+0000","last_change":"2024-04-26T20:58:43.070983+0000","last_active":"2024-04-26T20:58:43.072986+0000","last_peered":"2024-04-26T20:58:43.072986+0000","last_clean":"2024-04-26T20:58:43.072986+0000","last_became_active":"2024-04-26T20:58:43.070724+0000","last_became_peered":"2024-04-26T20:58:43.070724+0000","last_unstale":"2024-04-26T20:58:43.072986+0000","last_undegraded":"2024-04-26T20:58:43.072986+0000","last_fullsized":"2024-04-26T20:58:43.072986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:36.534833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.057380+0000","last_change":"2024-04-26T20:58:43.057380+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.057380+0000","last_undegraded":"2024-04-26T20:58:43.057380+0000","last_fullsized":"2024-04-26T20:58:43.057380+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074913+0000","last_change":"2024-04-26T20:58:43.069835+0000","last_active":"2024-04-26T20:58:43.074913+0000","last_peered":"2024-04-26T20:58:43.074913+0000","last_clean":"2024-04-26T20:58:43.074913+0000","last_became_active":"2024-04-26T20:58:43.069535+0000","last_became_peered":"2024-04-26T20:58:43.069535+0000","last_unstale":"2024-04-26T20:58:43.074913+0000","last_undegraded":"2024-04-26T20:58:43.074913+0000","last_fullsized":"2024-04-26T20:58:43.074913+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:45.716150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.063901+0000","last_change":"2024-04-26T20:58:43.063901+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.063901+0000","last_undegraded":"2024-04-26T20:58:43.063901+0000","last_fullsized":"2024-04-26T20:58:43.063901+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.064585+0000","last_change":"2024-04-26T20:58:43.060904+0000","last_active":"2024-04-26T20:58:43.064585+0000","last_peered":"2024-04-26T20:58:43.064585+0000","last_clean":"2024-04-26T20:58:43.064585+0000","last_became_active":"2024-04-26T20:58:43.060511+0000","last_became_peered":"2024-04-26T20:58:43.060511+0000","last_unstale":"2024-04-26T20:58:43.064585+0000","last_undegraded":"2024-04-26T20:58:43.064585+0000","last_fullsized":"2024-04-26T20:58:43.064585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:21.497125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055227+0000","last_change":"2024-04-26T20:58:43.055227+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055227+0000","last_undegraded":"2024-04-26T20:58:43.055227+0000","last_fullsized":"2024-04-26T20:58:43.055227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.079030+0000","last_change":"2024-04-26T20:58:43.075337+0000","last_active":"2024-04-26T20:58:43.079030+0000","last_peered":"2024-04-26T20:58:43.079030+0000","last_clean":"2024-04-26T20:58:43.079030+0000","last_became_active":"2024-04-26T20:58:43.075019+0000","last_became_peered":"2024-04-26T20:58:43.075019+0000","last_unstale":"2024-04-26T20:58:43.079030+0000","last_undegraded":"2024-04-26T20:58:43.079030+0000","last_fullsized":"2024-04-26T20:58:43.079030+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:44.310082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.047448+0000","last_change":"2024-04-26T20:58:43.047448+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.047448+0000","last_undegraded":"2024-04-26T20:58:43.047448+0000","last_fullsized":"2024-04-26T20:58:43.047448+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074179+0000","last_change":"2024-04-26T20:58:43.071692+0000","last_active":"2024-04-26T20:58:43.074179+0000","last_peered":"2024-04-26T20:58:43.074179+0000","last_clean":"2024-04-26T20:58:43.074179+0000","last_became_active":"2024-04-26T20:58:43.069775+0000","last_became_peered":"2024-04-26T20:58:43.069775+0000","last_unstale":"2024-04-26T20:58:43.074179+0000","last_undegraded":"2024-04-26T20:58:43.074179+0000","last_fullsized":"2024-04-26T20:58:43.074179+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.491913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.045175+0000","last_change":"2024-04-26T20:58:43.045175+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.045175+0000","last_undegraded":"2024-04-26T20:58:43.045175+0000","last_fullsized":"2024-04-26T20:58:43.045175+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.079032+0000","last_change":"2024-04-26T20:58:43.075348+0000","last_active":"2024-04-26T20:58:43.079032+0000","last_peered":"2024-04-26T20:58:43.079032+0000","last_clean":"2024-04-26T20:58:43.079032+0000","last_became_active":"2024-04-26T20:58:43.075012+0000","last_became_peered":"2024-04-26T20:58:43.075012+0000","last_unstale":"2024-04-26T20:58:43.079032+0000","last_undegraded":"2024-04-26T20:58:43.079032+0000","last_fullsized":"2024-04-26T20:58:43.079032+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:44.341099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.045208+0000","last_change":"2024-04-26T20:58:43.045208+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.045208+0000","last_undegraded":"2024-04-26T20:58:43.045208+0000","last_fullsized":"2024-04-26T20:58:43.045208+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.064494+0000","last_change":"2024-04-26T20:58:43.054765+0000","last_active":"2024-04-26T20:58:43.064494+0000","last_peered":"2024-04-26T20:58:43.064494+0000","last_clean":"2024-04-26T20:58:43.064494+0000","last_became_active":"2024-04-26T20:58:43.054455+0000","last_became_peered":"2024-04-26T20:58:43.054455+0000","last_unstale":"2024-04-26T20:58:43.064494+0000","last_undegraded":"2024-04-26T20:58:43.064494+0000","last_fullsized":"2024-04-26T20:58:43.064494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:36.534833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.062096+0000","last_change":"2024-04-26T20:58:43.062096+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.062096+0000","last_undegraded":"2024-04-26T20:58:43.062096+0000","last_fullsized":"2024-04-26T20:58:43.062096+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.067885+0000","last_change":"2024-04-26T20:58:43.065790+0000","last_active":"2024-04-26T20:58:43.067885+0000","last_peered":"2024-04-26T20:58:43.067885+0000","last_clean":"2024-04-26T20:58:43.067885+0000","last_became_active":"2024-04-26T20:58:43.065469+0000","last_became_peered":"2024-04-26T20:58:43.065469+0000","last_unstale":"2024-04-26T20:58:43.067885+0000","last_undegraded":"2024-04-26T20:58:43.067885+0000","last_fullsized":"2024-04-26T20:58:43.067885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:28.519261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.044116+0000","last_change":"2024-04-26T20:58:43.044116+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.044116+0000","last_undegraded":"2024-04-26T20:58:43.044116+0000","last_fullsized":"2024-04-26T20:58:43.044116+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.059577+0000","last_change":"2024-04-26T20:58:43.056930+0000","last_active":"2024-04-26T20:58:43.059577+0000","last_peered":"2024-04-26T20:58:43.059577+0000","last_clean":"2024-04-26T20:58:43.059577+0000","last_became_active":"2024-04-26T20:58:43.055764+0000","last_became_peered":"2024-04-26T20:58:43.055764+0000","last_unstale":"2024-04-26T20:58:43.059577+0000","last_undegraded":"2024-04-26T20:58:43.059577+0000","last_fullsized":"2024-04-26T20:58:43.059577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:14.346315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055239+0000","last_change":"2024-04-26T20:58:43.055239+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055239+0000","last_undegraded":"2024-04-26T20:58:43.055239+0000","last_fullsized":"2024-04-26T20:58:43.055239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.076095+0000","last_change":"2024-04-26T20:58:43.074219+0000","last_active":"2024-04-26T20:58:43.076095+0000","last_peered":"2024-04-26T20:58:43.076095+0000","last_clean":"2024-04-26T20:58:43.076095+0000","last_became_active":"2024-04-26T20:58:43.072029+0000","last_became_peered":"2024-04-26T20:58:43.072029+0000","last_unstale":"2024-04-26T20:58:43.076095+0000","last_undegraded":"2024-04-26T20:58:43.076095+0000","last_fullsized":"2024-04-26T20:58:43.076095+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:58.207157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":211,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.058660+0000","last_change":"2024-04-26T20:56:46.848603+0000","last_active":"2024-04-26T20:58:43.058660+0000","last_peered":"2024-04-26T20:58:43.058660+0000","last_clean":"2024-04-26T20:58:43.058660+0000","last_became_active":"2024-04-26T20:56:46.846091+0000","last_became_peered":"2024-04-26T20:56:46.846091+0000","last_unstale":"2024-04-26T20:58:43.058660+0000","last_undegraded":"2024-04-26T20:58:43.058660+0000","last_fullsized":"2024-04-26T20:58:43.058660+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_clean_scrub_stamp":"2024-04-26T20:56:45.594041+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:00.232387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":785505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":201404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":201404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":201404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":201404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":791772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:58:49.067 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:58:49.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.322+0000 7f481ec00640 1 Processor -- start 2024-04-26T20:58:49.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.322+0000 7f481ec00640 1 -- start start 2024-04-26T20:58:49.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.323+0000 7f481ec00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f481815f000 0x7f48180baad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:49.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.323+0000 7f481ec00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f481815f3e0 0x7f4818162d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:49.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.323+0000 7f481ec00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4818163250 con 0x7f48180cfd00 2024-04-26T20:58:49.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.324+0000 7f481ec00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f48180d3330 con 0x7f481815f000 2024-04-26T20:58:49.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.324+0000 7f481ec00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f48180d3610 con 0x7f481815f3e0 2024-04-26T20:58:49.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.324+0000 7f481d800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f481815f000 0x7f48180baad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:49.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.324+0000 7f481e200640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f48180cfd00 0x7f48180b69b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52968/0 (socket says 172.21.15.28:52968) 2024-04-26T20:58:49.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.324+0000 7f481e200640 1 -- 172.21.15.28:0/1839639535 learned_addr learned my addr 172.21.15.28:0/1839639535 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:49.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.324+0000 7f481ce00640 1 --2- 172.21.15.28:0/1839639535 >> v2:172.21.15.103:3300/0 conn(0x7f481815f3e0 0x7f4818162d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:49.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.324+0000 7f4817e00640 1 -- 172.21.15.28:0/1839639535 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2155971579 0 0) 0x7f4818163250 con 0x7f48180cfd00 2024-04-26T20:58:49.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.324+0000 7f4817e00640 1 -- 172.21.15.28:0/1839639535 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47fc003520 con 0x7f48180cfd00 2024-04-26T20:58:49.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.324+0000 7f481d800640 1 -- 172.21.15.28:0/1839639535 >> v2:172.21.15.103:3300/0 conn(0x7f481815f3e0 msgr2=0x7f4818162d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:49.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.324+0000 7f481d800640 1 --2- 172.21.15.28:0/1839639535 >> v2:172.21.15.103:3300/0 conn(0x7f481815f3e0 0x7f4818162d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.324+0000 7f481d800640 1 -- 172.21.15.28:0/1839639535 >> v1:172.21.15.28:6789/0 conn(0x7f48180cfd00 legacy=0x7f48180b69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:49.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.324+0000 7f481d800640 1 -- 172.21.15.28:0/1839639535 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48181634f0 con 0x7f481815f000 2024-04-26T20:58:49.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.324+0000 7f481ce00640 1 --2- 172.21.15.28:0/1839639535 >> v2:172.21.15.103:3300/0 conn(0x7f481815f3e0 0x7f4818162d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:49.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.326+0000 7f481d800640 1 --2- 172.21.15.28:0/1839639535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f481815f000 0x7f48180baad0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f48180d2a30 tx=0x7f481002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801000be3503bfd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:49.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.326+0000 7f4817e00640 1 -- 172.21.15.28:0/1839639535 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810007740 con 0x7f481815f000 2024-04-26T20:58:49.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.326+0000 7f4817e00640 1 -- 172.21.15.28:0/1839639535 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48100078e0 con 0x7f481815f000 2024-04-26T20:58:49.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.326+0000 7f4817e00640 1 -- 172.21.15.28:0/1839639535 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810007c90 con 0x7f481815f000 2024-04-26T20:58:49.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.326+0000 7f481ec00640 1 -- 172.21.15.28:0/1839639535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f481815f000 msgr2=0x7f48180baad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:49.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.326+0000 7f481ec00640 1 --2- 172.21.15.28:0/1839639535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f481815f000 0x7f48180baad0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f48180d2a30 tx=0x7f481002a5f0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.327+0000 7f481ec00640 1 -- 172.21.15.28:0/1839639535 shutdown_connections 2024-04-26T20:58:49.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.327+0000 7f481ec00640 1 --2- 172.21.15.28:0/1839639535 >> v2:172.21.15.103:3300/0 conn(0x7f481815f3e0 0x7f4818162d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.327+0000 7f481ec00640 1 --2- 172.21.15.28:0/1839639535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f481815f000 0x7f48180baad0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.327+0000 7f481ec00640 1 -- 172.21.15.28:0/1839639535 >> 172.21.15.28:0/1839639535 conn(0x7f4818072b50 msgr2=0x7f4818072f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:49.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.327+0000 7f481ec00640 1 -- 172.21.15.28:0/1839639535 shutdown_connections 2024-04-26T20:58:49.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.327+0000 7f481ec00640 1 -- 172.21.15.28:0/1839639535 wait complete. 2024-04-26T20:58:49.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.328+0000 7f481ec00640 1 Processor -- start 2024-04-26T20:58:49.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.329+0000 7f481ec00640 1 -- start start 2024-04-26T20:58:49.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.330+0000 7f481ec00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48180cfd00 0x7f48181564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:49.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.330+0000 7f481ec00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f481815f000 0x7f48180b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:49.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.330+0000 7f481ec00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f481815f3e0 0x7f481815ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:49.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.330+0000 7f481ec00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f48180d4ab0 con 0x7f48180cfd00 2024-04-26T20:58:49.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.330+0000 7f481ec00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4818163ab0 con 0x7f481815f000 2024-04-26T20:58:49.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.330+0000 7f481ec00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f48180d4c30 con 0x7f481815f3e0 2024-04-26T20:58:49.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.330+0000 7f481d800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48180cfd00 0x7f48181564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:49.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.330+0000 7f481d800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48180cfd00 0x7f48181564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:53984/0 (socket says 172.21.15.28:53984) 2024-04-26T20:58:49.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.330+0000 7f481d800640 1 -- 172.21.15.28:0/547044757 learned_addr learned my addr 172.21.15.28:0/547044757 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:49.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.330+0000 7f481d800640 1 -- 172.21.15.28:0/547044757 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f481815f3e0 msgr2=0x7f481815ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:49.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.330+0000 7f481d800640 1 --2- 172.21.15.28:0/547044757 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f481815f3e0 0x7f481815ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.330+0000 7f481d800640 1 -- 172.21.15.28:0/547044757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f481815f000 msgr2=0x7f48180b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:49.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.330+0000 7f481d800640 1 --2- 172.21.15.28:0/547044757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f481815f000 0x7f48180b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.330+0000 7f481d800640 1 -- 172.21.15.28:0/547044757 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4818160a90 con 0x7f48180cfd00 2024-04-26T20:58:49.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.331+0000 7f481d800640 1 --2- 172.21.15.28:0/547044757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48180cfd00 0x7f48181564e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4810009e80 tx=0x7f4810004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:49.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.331+0000 7f4815600640 1 -- 172.21.15.28:0/547044757 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810009450 con 0x7f48180cfd00 2024-04-26T20:58:49.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.331+0000 7f4815600640 1 -- 172.21.15.28:0/547044757 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4810004260 con 0x7f48180cfd00 2024-04-26T20:58:49.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.331+0000 7f481ec00640 1 -- 172.21.15.28:0/547044757 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4818160d70 con 0x7f48180cfd00 2024-04-26T20:58:49.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.331+0000 7f4815600640 1 -- 172.21.15.28:0/547044757 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810004650 con 0x7f48180cfd00 2024-04-26T20:58:49.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.331+0000 7f481ec00640 1 -- 172.21.15.28:0/547044757 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4818161330 con 0x7f48180cfd00 2024-04-26T20:58:49.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.332+0000 7f481ec00640 1 -- 172.21.15.28:0/547044757 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4818161530 con 0x7f48180cfd00 2024-04-26T20:58:49.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.333+0000 7f4815600640 1 -- 172.21.15.28:0/547044757 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4810039040 con 0x7f48180cfd00 2024-04-26T20:58:49.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.334+0000 7f4815600640 1 --2- 172.21.15.28:0/547044757 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f47f806c970 0x7f47f806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:49.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.334+0000 7f4815600640 1 -- 172.21.15.28:0/547044757 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f48100ad150 con 0x7f48180cfd00 2024-04-26T20:58:49.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.334+0000 7f481ce00640 1 --2- 172.21.15.28:0/547044757 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f47f806c970 0x7f47f806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:49.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.335+0000 7f481ce00640 1 --2- 172.21.15.28:0/547044757 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f47f806c970 0x7f47f806ee30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4804006fd0 tx=0x7f4804008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:49.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.339+0000 7f4815600640 1 -- 172.21.15.28:0/547044757 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4818161530 con 0x7f48180cfd00 2024-04-26T20:58:49.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.539+0000 7f481ec00640 1 -- 172.21.15.28:0/547044757 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4818160840 con 0x7f47f806c970 2024-04-26T20:58:49.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.549+0000 7f4815600640 1 -- 172.21.15.28:0/547044757 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246636 (secure 0 0 0) 0x7f4818160840 con 0x7f47f806c970 2024-04-26T20:58:49.555 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:49.557 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T20:58:49.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.554+0000 7f481ec00640 1 -- 172.21.15.28:0/547044757 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f47f806c970 msgr2=0x7f47f806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:49.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.554+0000 7f481ec00640 1 --2- 172.21.15.28:0/547044757 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f47f806c970 0x7f47f806ee30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4804006fd0 tx=0x7f4804008040 comp rx=0 tx=0).stop 2024-04-26T20:58:49.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.554+0000 7f481ec00640 1 -- 172.21.15.28:0/547044757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48180cfd00 msgr2=0x7f48181564e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:49.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.554+0000 7f481ec00640 1 --2- 172.21.15.28:0/547044757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48180cfd00 0x7f48181564e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4810009e80 tx=0x7f4810004a40 comp rx=0 tx=0).stop 2024-04-26T20:58:49.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.554+0000 7f481ec00640 1 -- 172.21.15.28:0/547044757 shutdown_connections 2024-04-26T20:58:49.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.554+0000 7f481ec00640 1 --2- 172.21.15.28:0/547044757 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f47f806c970 0x7f47f806ee30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.554+0000 7f481ec00640 1 --2- 172.21.15.28:0/547044757 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f481815f3e0 0x7f481815ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.554+0000 7f481ec00640 1 --2- 172.21.15.28:0/547044757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f481815f000 0x7f48180b53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.554+0000 7f481ec00640 1 --2- 172.21.15.28:0/547044757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f48180cfd00 0x7f48181564e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.554+0000 7f481ec00640 1 -- 172.21.15.28:0/547044757 >> 172.21.15.28:0/547044757 conn(0x7f4818072b50 msgr2=0x7f48180d4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:49.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.554+0000 7f481ec00640 1 -- 172.21.15.28:0/547044757 shutdown_connections 2024-04-26T20:58:49.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.554+0000 7f481ec00640 1 -- 172.21.15.28:0/547044757 wait complete. 2024-04-26T20:58:49.576 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-04-26T20:58:48.741891+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237876,"kb_used_data":5060,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736844,"statfs":{"total":773094113280,"available":772850528256,"internally_reserved":0,"allocated":5181440,"data_stored":2773167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001436"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055951+0000","last_change":"2024-04-26T20:58:43.055951+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055951+0000","last_undegraded":"2024-04-26T20:58:43.055951+0000","last_fullsized":"2024-04-26T20:58:43.055951+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.052235+0000","last_change":"2024-04-26T20:58:43.052235+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.052235+0000","last_undegraded":"2024-04-26T20:58:43.052235+0000","last_fullsized":"2024-04-26T20:58:43.052235+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.048628+0000","last_change":"2024-04-26T20:58:43.048628+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.048628+0000","last_undegraded":"2024-04-26T20:58:43.048628+0000","last_fullsized":"2024-04-26T20:58:43.048628+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056479+0000","last_change":"2024-04-26T20:58:43.056479+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056479+0000","last_undegraded":"2024-04-26T20:58:43.056479+0000","last_fullsized":"2024-04-26T20:58:43.056479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056437+0000","last_change":"2024-04-26T20:58:43.056437+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056437+0000","last_undegraded":"2024-04-26T20:58:43.056437+0000","last_fullsized":"2024-04-26T20:58:43.056437+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.048264+0000","last_change":"2024-04-26T20:58:43.048264+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.048264+0000","last_undegraded":"2024-04-26T20:58:43.048264+0000","last_fullsized":"2024-04-26T20:58:43.048264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.049255+0000","last_change":"2024-04-26T20:58:43.049255+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.049255+0000","last_undegraded":"2024-04-26T20:58:43.049255+0000","last_fullsized":"2024-04-26T20:58:43.049255+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.051206+0000","last_change":"2024-04-26T20:58:43.051206+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.051206+0000","last_undegraded":"2024-04-26T20:58:43.051206+0000","last_fullsized":"2024-04-26T20:58:43.051206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.053178+0000","last_change":"2024-04-26T20:58:43.053178+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.053178+0000","last_undegraded":"2024-04-26T20:58:43.053178+0000","last_fullsized":"2024-04-26T20:58:43.053178+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056034+0000","last_change":"2024-04-26T20:58:43.056034+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056034+0000","last_undegraded":"2024-04-26T20:58:43.056034+0000","last_fullsized":"2024-04-26T20:58:43.056034+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.052135+0000","last_change":"2024-04-26T20:58:43.052135+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.052135+0000","last_undegraded":"2024-04-26T20:58:43.052135+0000","last_fullsized":"2024-04-26T20:58:43.052135+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.060146+0000","last_change":"2024-04-26T20:58:43.060146+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.060146+0000","last_undegraded":"2024-04-26T20:58:43.060146+0000","last_fullsized":"2024-04-26T20:58:43.060146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.038802+0000","last_change":"2024-04-26T20:58:43.038802+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.038802+0000","last_undegraded":"2024-04-26T20:58:43.038802+0000","last_fullsized":"2024-04-26T20:58:43.038802+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.051120+0000","last_change":"2024-04-26T20:58:43.051120+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.051120+0000","last_undegraded":"2024-04-26T20:58:43.051120+0000","last_fullsized":"2024-04-26T20:58:43.051120+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.049339+0000","last_change":"2024-04-26T20:58:43.049339+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.049339+0000","last_undegraded":"2024-04-26T20:58:43.049339+0000","last_fullsized":"2024-04-26T20:58:43.049339+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.061785+0000","last_change":"2024-04-26T20:58:43.061785+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.061785+0000","last_undegraded":"2024-04-26T20:58:43.061785+0000","last_fullsized":"2024-04-26T20:58:43.061785+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.049592+0000","last_change":"2024-04-26T20:58:43.049592+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.049592+0000","last_undegraded":"2024-04-26T20:58:43.049592+0000","last_fullsized":"2024-04-26T20:58:43.049592+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.051921+0000","last_change":"2024-04-26T20:58:43.051921+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.051921+0000","last_undegraded":"2024-04-26T20:58:43.051921+0000","last_fullsized":"2024-04-26T20:58:43.051921+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056057+0000","last_change":"2024-04-26T20:58:43.056057+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056057+0000","last_undegraded":"2024-04-26T20:58:43.056057+0000","last_fullsized":"2024-04-26T20:58:43.056057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.039248+0000","last_change":"2024-04-26T20:58:43.039248+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.039248+0000","last_undegraded":"2024-04-26T20:58:43.039248+0000","last_fullsized":"2024-04-26T20:58:43.039248+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.047349+0000","last_change":"2024-04-26T20:58:43.047349+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.047349+0000","last_undegraded":"2024-04-26T20:58:43.047349+0000","last_fullsized":"2024-04-26T20:58:43.047349+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.046758+0000","last_change":"2024-04-26T20:58:43.046758+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.046758+0000","last_undegraded":"2024-04-26T20:58:43.046758+0000","last_fullsized":"2024-04-26T20:58:43.046758+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.059090+0000","last_change":"2024-04-26T20:58:43.059090+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.059090+0000","last_undegraded":"2024-04-26T20:58:43.059090+0000","last_fullsized":"2024-04-26T20:58:43.059090+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.060755+0000","last_change":"2024-04-26T20:58:43.060755+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.060755+0000","last_undegraded":"2024-04-26T20:58:43.060755+0000","last_fullsized":"2024-04-26T20:58:43.060755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.064938+0000","last_change":"2024-04-26T20:58:43.064938+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.064938+0000","last_undegraded":"2024-04-26T20:58:43.064938+0000","last_fullsized":"2024-04-26T20:58:43.064938+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.040818+0000","last_change":"2024-04-26T20:58:43.040818+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.040818+0000","last_undegraded":"2024-04-26T20:58:43.040818+0000","last_fullsized":"2024-04-26T20:58:43.040818+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.046134+0000","last_change":"2024-04-26T20:58:43.046134+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.046134+0000","last_undegraded":"2024-04-26T20:58:43.046134+0000","last_fullsized":"2024-04-26T20:58:43.046134+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.058202+0000","last_change":"2024-04-26T20:58:43.058202+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.058202+0000","last_undegraded":"2024-04-26T20:58:43.058202+0000","last_fullsized":"2024-04-26T20:58:43.058202+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056143+0000","last_change":"2024-04-26T20:58:43.056143+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056143+0000","last_undegraded":"2024-04-26T20:58:43.056143+0000","last_fullsized":"2024-04-26T20:58:43.056143+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.047071+0000","last_change":"2024-04-26T20:58:43.047071+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.047071+0000","last_undegraded":"2024-04-26T20:58:43.047071+0000","last_fullsized":"2024-04-26T20:58:43.047071+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.047975+0000","last_change":"2024-04-26T20:58:43.047975+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.047975+0000","last_undegraded":"2024-04-26T20:58:43.047975+0000","last_fullsized":"2024-04-26T20:58:43.047975+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.040702+0000","last_change":"2024-04-26T20:58:43.040702+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.040702+0000","last_undegraded":"2024-04-26T20:58:43.040702+0000","last_fullsized":"2024-04-26T20:58:43.040702+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.053161+0000","last_change":"2024-04-26T20:58:43.053161+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.053161+0000","last_undegraded":"2024-04-26T20:58:43.053161+0000","last_fullsized":"2024-04-26T20:58:43.053161+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.072931+0000","last_change":"2024-04-26T20:58:43.070204+0000","last_active":"2024-04-26T20:58:43.072931+0000","last_peered":"2024-04-26T20:58:43.072931+0000","last_clean":"2024-04-26T20:58:43.072931+0000","last_became_active":"2024-04-26T20:58:43.069791+0000","last_became_peered":"2024-04-26T20:58:43.069791+0000","last_unstale":"2024-04-26T20:58:43.072931+0000","last_undegraded":"2024-04-26T20:58:43.072931+0000","last_fullsized":"2024-04-26T20:58:43.072931+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:21.679330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.064417+0000","last_change":"2024-04-26T20:58:43.064417+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.064417+0000","last_undegraded":"2024-04-26T20:58:43.064417+0000","last_fullsized":"2024-04-26T20:58:43.064417+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.076328+0000","last_change":"2024-04-26T20:58:43.073833+0000","last_active":"2024-04-26T20:58:43.076328+0000","last_peered":"2024-04-26T20:58:43.076328+0000","last_clean":"2024-04-26T20:58:43.076328+0000","last_became_active":"2024-04-26T20:58:43.070958+0000","last_became_peered":"2024-04-26T20:58:43.070958+0000","last_unstale":"2024-04-26T20:58:43.076328+0000","last_undegraded":"2024-04-26T20:58:43.076328+0000","last_fullsized":"2024-04-26T20:58:43.076328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:23.656127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.057716+0000","last_change":"2024-04-26T20:58:43.057716+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.057716+0000","last_undegraded":"2024-04-26T20:58:43.057716+0000","last_fullsized":"2024-04-26T20:58:43.057716+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074245+0000","last_change":"2024-04-26T20:58:43.072680+0000","last_active":"2024-04-26T20:58:43.074245+0000","last_peered":"2024-04-26T20:58:43.074245+0000","last_clean":"2024-04-26T20:58:43.074245+0000","last_became_active":"2024-04-26T20:58:43.072369+0000","last_became_peered":"2024-04-26T20:58:43.072369+0000","last_unstale":"2024-04-26T20:58:43.074245+0000","last_undegraded":"2024-04-26T20:58:43.074245+0000","last_fullsized":"2024-04-26T20:58:43.074245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:30.432055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055248+0000","last_change":"2024-04-26T20:58:43.055248+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055248+0000","last_undegraded":"2024-04-26T20:58:43.055248+0000","last_fullsized":"2024-04-26T20:58:43.055248+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074948+0000","last_change":"2024-04-26T20:58:43.070599+0000","last_active":"2024-04-26T20:58:43.074948+0000","last_peered":"2024-04-26T20:58:43.074948+0000","last_clean":"2024-04-26T20:58:43.074948+0000","last_became_active":"2024-04-26T20:58:43.070312+0000","last_became_peered":"2024-04-26T20:58:43.070312+0000","last_unstale":"2024-04-26T20:58:43.074948+0000","last_undegraded":"2024-04-26T20:58:43.074948+0000","last_fullsized":"2024-04-26T20:58:43.074948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:41.312188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.045079+0000","last_change":"2024-04-26T20:58:43.045079+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.045079+0000","last_undegraded":"2024-04-26T20:58:43.045079+0000","last_fullsized":"2024-04-26T20:58:43.045079+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.068273+0000","last_change":"2024-04-26T20:58:43.066495+0000","last_active":"2024-04-26T20:58:43.068273+0000","last_peered":"2024-04-26T20:58:43.068273+0000","last_clean":"2024-04-26T20:58:43.068273+0000","last_became_active":"2024-04-26T20:58:43.066214+0000","last_became_peered":"2024-04-26T20:58:43.066214+0000","last_unstale":"2024-04-26T20:58:43.068273+0000","last_undegraded":"2024-04-26T20:58:43.068273+0000","last_fullsized":"2024-04-26T20:58:43.068273+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:02.546662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055253+0000","last_change":"2024-04-26T20:58:43.055253+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055253+0000","last_undegraded":"2024-04-26T20:58:43.055253+0000","last_fullsized":"2024-04-26T20:58:43.055253+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.071565+0000","last_change":"2024-04-26T20:58:43.067727+0000","last_active":"2024-04-26T20:58:43.071565+0000","last_peered":"2024-04-26T20:58:43.071565+0000","last_clean":"2024-04-26T20:58:43.071565+0000","last_became_active":"2024-04-26T20:58:43.066679+0000","last_became_peered":"2024-04-26T20:58:43.066679+0000","last_unstale":"2024-04-26T20:58:43.071565+0000","last_undegraded":"2024-04-26T20:58:43.071565+0000","last_fullsized":"2024-04-26T20:58:43.071565+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:28.654953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.050736+0000","last_change":"2024-04-26T20:58:43.050736+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.050736+0000","last_undegraded":"2024-04-26T20:58:43.050736+0000","last_fullsized":"2024-04-26T20:58:43.050736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.066307+0000","last_change":"2024-04-26T20:58:43.057010+0000","last_active":"2024-04-26T20:58:43.066307+0000","last_peered":"2024-04-26T20:58:43.066307+0000","last_clean":"2024-04-26T20:58:43.066307+0000","last_became_active":"2024-04-26T20:58:43.056562+0000","last_became_peered":"2024-04-26T20:58:43.056562+0000","last_unstale":"2024-04-26T20:58:43.066307+0000","last_undegraded":"2024-04-26T20:58:43.066307+0000","last_fullsized":"2024-04-26T20:58:43.066307+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:46.360000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.059706+0000","last_change":"2024-04-26T20:58:43.059706+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.059706+0000","last_undegraded":"2024-04-26T20:58:43.059706+0000","last_fullsized":"2024-04-26T20:58:43.059706+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.068377+0000","last_change":"2024-04-26T20:58:43.066688+0000","last_active":"2024-04-26T20:58:43.068377+0000","last_peered":"2024-04-26T20:58:43.068377+0000","last_clean":"2024-04-26T20:58:43.068377+0000","last_became_active":"2024-04-26T20:58:43.066372+0000","last_became_peered":"2024-04-26T20:58:43.066372+0000","last_unstale":"2024-04-26T20:58:43.068377+0000","last_undegraded":"2024-04-26T20:58:43.068377+0000","last_fullsized":"2024-04-26T20:58:43.068377+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:32.800607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.050460+0000","last_change":"2024-04-26T20:58:43.050460+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.050460+0000","last_undegraded":"2024-04-26T20:58:43.050460+0000","last_fullsized":"2024-04-26T20:58:43.050460+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.064945+0000","last_change":"2024-04-26T20:58:43.061818+0000","last_active":"2024-04-26T20:58:43.064945+0000","last_peered":"2024-04-26T20:58:43.064945+0000","last_clean":"2024-04-26T20:58:43.064945+0000","last_became_active":"2024-04-26T20:58:43.059103+0000","last_became_peered":"2024-04-26T20:58:43.059103+0000","last_unstale":"2024-04-26T20:58:43.064945+0000","last_undegraded":"2024-04-26T20:58:43.064945+0000","last_fullsized":"2024-04-26T20:58:43.064945+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:31.981808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.047092+0000","last_change":"2024-04-26T20:58:43.047092+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.047092+0000","last_undegraded":"2024-04-26T20:58:43.047092+0000","last_fullsized":"2024-04-26T20:58:43.047092+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.073123+0000","last_change":"2024-04-26T20:58:43.071259+0000","last_active":"2024-04-26T20:58:43.073123+0000","last_peered":"2024-04-26T20:58:43.073123+0000","last_clean":"2024-04-26T20:58:43.073123+0000","last_became_active":"2024-04-26T20:58:43.070952+0000","last_became_peered":"2024-04-26T20:58:43.070952+0000","last_unstale":"2024-04-26T20:58:43.073123+0000","last_undegraded":"2024-04-26T20:58:43.073123+0000","last_fullsized":"2024-04-26T20:58:43.073123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:39.191097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.046401+0000","last_change":"2024-04-26T20:58:43.046401+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.046401+0000","last_undegraded":"2024-04-26T20:58:43.046401+0000","last_fullsized":"2024-04-26T20:58:43.046401+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.076213+0000","last_change":"2024-04-26T20:58:43.072767+0000","last_active":"2024-04-26T20:58:43.076213+0000","last_peered":"2024-04-26T20:58:43.076213+0000","last_clean":"2024-04-26T20:58:43.076213+0000","last_became_active":"2024-04-26T20:58:43.072408+0000","last_became_peered":"2024-04-26T20:58:43.072408+0000","last_unstale":"2024-04-26T20:58:43.076213+0000","last_undegraded":"2024-04-26T20:58:43.076213+0000","last_fullsized":"2024-04-26T20:58:43.076213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:46.360000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.049406+0000","last_change":"2024-04-26T20:58:43.049406+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.049406+0000","last_undegraded":"2024-04-26T20:58:43.049406+0000","last_fullsized":"2024-04-26T20:58:43.049406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.060882+0000","last_change":"2024-04-26T20:58:43.059043+0000","last_active":"2024-04-26T20:58:43.060882+0000","last_peered":"2024-04-26T20:58:43.060882+0000","last_clean":"2024-04-26T20:58:43.060882+0000","last_became_active":"2024-04-26T20:58:43.058718+0000","last_became_peered":"2024-04-26T20:58:43.058718+0000","last_unstale":"2024-04-26T20:58:43.060882+0000","last_undegraded":"2024-04-26T20:58:43.060882+0000","last_fullsized":"2024-04-26T20:58:43.060882+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:57.286045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055956+0000","last_change":"2024-04-26T20:58:43.055956+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055956+0000","last_undegraded":"2024-04-26T20:58:43.055956+0000","last_fullsized":"2024-04-26T20:58:43.055956+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.062506+0000","last_change":"2024-04-26T20:58:43.057592+0000","last_active":"2024-04-26T20:58:43.062506+0000","last_peered":"2024-04-26T20:58:43.062506+0000","last_clean":"2024-04-26T20:58:43.062506+0000","last_became_active":"2024-04-26T20:58:43.057285+0000","last_became_peered":"2024-04-26T20:58:43.057285+0000","last_unstale":"2024-04-26T20:58:43.062506+0000","last_undegraded":"2024-04-26T20:58:43.062506+0000","last_fullsized":"2024-04-26T20:58:43.062506+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:47.781660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.054665+0000","last_change":"2024-04-26T20:58:43.054665+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.054665+0000","last_undegraded":"2024-04-26T20:58:43.054665+0000","last_fullsized":"2024-04-26T20:58:43.054665+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074564+0000","last_change":"2024-04-26T20:58:43.068260+0000","last_active":"2024-04-26T20:58:43.074564+0000","last_peered":"2024-04-26T20:58:43.074564+0000","last_clean":"2024-04-26T20:58:43.074564+0000","last_became_active":"2024-04-26T20:58:43.067852+0000","last_became_peered":"2024-04-26T20:58:43.067852+0000","last_unstale":"2024-04-26T20:58:43.074564+0000","last_undegraded":"2024-04-26T20:58:43.074564+0000","last_fullsized":"2024-04-26T20:58:43.074564+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:15.110999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.045954+0000","last_change":"2024-04-26T20:58:43.045954+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.045954+0000","last_undegraded":"2024-04-26T20:58:43.045954+0000","last_fullsized":"2024-04-26T20:58:43.045954+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.061318+0000","last_change":"2024-04-26T20:58:43.059009+0000","last_active":"2024-04-26T20:58:43.061318+0000","last_peered":"2024-04-26T20:58:43.061318+0000","last_clean":"2024-04-26T20:58:43.061318+0000","last_became_active":"2024-04-26T20:58:43.058725+0000","last_became_peered":"2024-04-26T20:58:43.058725+0000","last_unstale":"2024-04-26T20:58:43.061318+0000","last_undegraded":"2024-04-26T20:58:43.061318+0000","last_fullsized":"2024-04-26T20:58:43.061318+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:51.552587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.048146+0000","last_change":"2024-04-26T20:58:43.048146+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.048146+0000","last_undegraded":"2024-04-26T20:58:43.048146+0000","last_fullsized":"2024-04-26T20:58:43.048146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.072981+0000","last_change":"2024-04-26T20:58:43.071075+0000","last_active":"2024-04-26T20:58:43.072981+0000","last_peered":"2024-04-26T20:58:43.072981+0000","last_clean":"2024-04-26T20:58:43.072981+0000","last_became_active":"2024-04-26T20:58:43.070762+0000","last_became_peered":"2024-04-26T20:58:43.070762+0000","last_unstale":"2024-04-26T20:58:43.072981+0000","last_undegraded":"2024-04-26T20:58:43.072981+0000","last_fullsized":"2024-04-26T20:58:43.072981+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:31.981808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.039364+0000","last_change":"2024-04-26T20:58:43.039364+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.039364+0000","last_undegraded":"2024-04-26T20:58:43.039364+0000","last_fullsized":"2024-04-26T20:58:43.039364+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.053458+0000","last_change":"2024-04-26T20:58:43.048980+0000","last_active":"2024-04-26T20:58:43.053458+0000","last_peered":"2024-04-26T20:58:43.053458+0000","last_clean":"2024-04-26T20:58:43.053458+0000","last_became_active":"2024-04-26T20:58:43.048625+0000","last_became_peered":"2024-04-26T20:58:43.048625+0000","last_unstale":"2024-04-26T20:58:43.053458+0000","last_undegraded":"2024-04-26T20:58:43.053458+0000","last_fullsized":"2024-04-26T20:58:43.053458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:49:57.621037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.051926+0000","last_change":"2024-04-26T20:58:43.051926+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.051926+0000","last_undegraded":"2024-04-26T20:58:43.051926+0000","last_fullsized":"2024-04-26T20:58:43.051926+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.066737+0000","last_change":"2024-04-26T20:58:43.063230+0000","last_active":"2024-04-26T20:58:43.066737+0000","last_peered":"2024-04-26T20:58:43.066737+0000","last_clean":"2024-04-26T20:58:43.066737+0000","last_became_active":"2024-04-26T20:58:43.062868+0000","last_became_peered":"2024-04-26T20:58:43.062868+0000","last_unstale":"2024-04-26T20:58:43.066737+0000","last_undegraded":"2024-04-26T20:58:43.066737+0000","last_fullsized":"2024-04-26T20:58:43.066737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:39.191097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.064001+0000","last_change":"2024-04-26T20:58:43.064001+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.064001+0000","last_undegraded":"2024-04-26T20:58:43.064001+0000","last_fullsized":"2024-04-26T20:58:43.064001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074548+0000","last_change":"2024-04-26T20:58:43.070226+0000","last_active":"2024-04-26T20:58:43.074548+0000","last_peered":"2024-04-26T20:58:43.074548+0000","last_clean":"2024-04-26T20:58:43.074548+0000","last_became_active":"2024-04-26T20:58:43.067146+0000","last_became_peered":"2024-04-26T20:58:43.067146+0000","last_unstale":"2024-04-26T20:58:43.074548+0000","last_undegraded":"2024-04-26T20:58:43.074548+0000","last_fullsized":"2024-04-26T20:58:43.074548+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:44.310082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056614+0000","last_change":"2024-04-26T20:58:43.056614+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056614+0000","last_undegraded":"2024-04-26T20:58:43.056614+0000","last_fullsized":"2024-04-26T20:58:43.056614+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.072821+0000","last_change":"2024-04-26T20:58:43.070073+0000","last_active":"2024-04-26T20:58:43.072821+0000","last_peered":"2024-04-26T20:58:43.072821+0000","last_clean":"2024-04-26T20:58:43.072821+0000","last_became_active":"2024-04-26T20:58:43.069554+0000","last_became_peered":"2024-04-26T20:58:43.069554+0000","last_unstale":"2024-04-26T20:58:43.072821+0000","last_undegraded":"2024-04-26T20:58:43.072821+0000","last_fullsized":"2024-04-26T20:58:43.072821+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:14.194412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.045725+0000","last_change":"2024-04-26T20:58:43.045725+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.045725+0000","last_undegraded":"2024-04-26T20:58:43.045725+0000","last_fullsized":"2024-04-26T20:58:43.045725+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.068193+0000","last_change":"2024-04-26T20:58:43.066409+0000","last_active":"2024-04-26T20:58:43.068193+0000","last_peered":"2024-04-26T20:58:43.068193+0000","last_clean":"2024-04-26T20:58:43.068193+0000","last_became_active":"2024-04-26T20:58:43.065775+0000","last_became_peered":"2024-04-26T20:58:43.065775+0000","last_unstale":"2024-04-26T20:58:43.068193+0000","last_undegraded":"2024-04-26T20:58:43.068193+0000","last_fullsized":"2024-04-26T20:58:43.068193+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:41.335687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.050764+0000","last_change":"2024-04-26T20:58:43.050764+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.050764+0000","last_undegraded":"2024-04-26T20:58:43.050764+0000","last_fullsized":"2024-04-26T20:58:43.050764+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.075472+0000","last_change":"2024-04-26T20:58:43.072643+0000","last_active":"2024-04-26T20:58:43.075472+0000","last_peered":"2024-04-26T20:58:43.075472+0000","last_clean":"2024-04-26T20:58:43.075472+0000","last_became_active":"2024-04-26T20:58:43.072358+0000","last_became_peered":"2024-04-26T20:58:43.072358+0000","last_unstale":"2024-04-26T20:58:43.075472+0000","last_undegraded":"2024-04-26T20:58:43.075472+0000","last_fullsized":"2024-04-26T20:58:43.075472+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:44.341099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.038929+0000","last_change":"2024-04-26T20:58:43.038929+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.038929+0000","last_undegraded":"2024-04-26T20:58:43.038929+0000","last_fullsized":"2024-04-26T20:58:43.038929+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.072986+0000","last_change":"2024-04-26T20:58:43.070983+0000","last_active":"2024-04-26T20:58:43.072986+0000","last_peered":"2024-04-26T20:58:43.072986+0000","last_clean":"2024-04-26T20:58:43.072986+0000","last_became_active":"2024-04-26T20:58:43.070724+0000","last_became_peered":"2024-04-26T20:58:43.070724+0000","last_unstale":"2024-04-26T20:58:43.072986+0000","last_undegraded":"2024-04-26T20:58:43.072986+0000","last_fullsized":"2024-04-26T20:58:43.072986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:36.534833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.057380+0000","last_change":"2024-04-26T20:58:43.057380+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.057380+0000","last_undegraded":"2024-04-26T20:58:43.057380+0000","last_fullsized":"2024-04-26T20:58:43.057380+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074913+0000","last_change":"2024-04-26T20:58:43.069835+0000","last_active":"2024-04-26T20:58:43.074913+0000","last_peered":"2024-04-26T20:58:43.074913+0000","last_clean":"2024-04-26T20:58:43.074913+0000","last_became_active":"2024-04-26T20:58:43.069535+0000","last_became_peered":"2024-04-26T20:58:43.069535+0000","last_unstale":"2024-04-26T20:58:43.074913+0000","last_undegraded":"2024-04-26T20:58:43.074913+0000","last_fullsized":"2024-04-26T20:58:43.074913+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:45.716150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.063901+0000","last_change":"2024-04-26T20:58:43.063901+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.063901+0000","last_undegraded":"2024-04-26T20:58:43.063901+0000","last_fullsized":"2024-04-26T20:58:43.063901+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.064585+0000","last_change":"2024-04-26T20:58:43.060904+0000","last_active":"2024-04-26T20:58:43.064585+0000","last_peered":"2024-04-26T20:58:43.064585+0000","last_clean":"2024-04-26T20:58:43.064585+0000","last_became_active":"2024-04-26T20:58:43.060511+0000","last_became_peered":"2024-04-26T20:58:43.060511+0000","last_unstale":"2024-04-26T20:58:43.064585+0000","last_undegraded":"2024-04-26T20:58:43.064585+0000","last_fullsized":"2024-04-26T20:58:43.064585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:21.497125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055227+0000","last_change":"2024-04-26T20:58:43.055227+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055227+0000","last_undegraded":"2024-04-26T20:58:43.055227+0000","last_fullsized":"2024-04-26T20:58:43.055227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.079030+0000","last_change":"2024-04-26T20:58:43.075337+0000","last_active":"2024-04-26T20:58:43.079030+0000","last_peered":"2024-04-26T20:58:43.079030+0000","last_clean":"2024-04-26T20:58:43.079030+0000","last_became_active":"2024-04-26T20:58:43.075019+0000","last_became_peered":"2024-04-26T20:58:43.075019+0000","last_unstale":"2024-04-26T20:58:43.079030+0000","last_undegraded":"2024-04-26T20:58:43.079030+0000","last_fullsized":"2024-04-26T20:58:43.079030+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:44.310082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.047448+0000","last_change":"2024-04-26T20:58:43.047448+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.047448+0000","last_undegraded":"2024-04-26T20:58:43.047448+0000","last_fullsized":"2024-04-26T20:58:43.047448+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074179+0000","last_change":"2024-04-26T20:58:43.071692+0000","last_active":"2024-04-26T20:58:43.074179+0000","last_peered":"2024-04-26T20:58:43.074179+0000","last_clean":"2024-04-26T20:58:43.074179+0000","last_became_active":"2024-04-26T20:58:43.069775+0000","last_became_peered":"2024-04-26T20:58:43.069775+0000","last_unstale":"2024-04-26T20:58:43.074179+0000","last_undegraded":"2024-04-26T20:58:43.074179+0000","last_fullsized":"2024-04-26T20:58:43.074179+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.491913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.045175+0000","last_change":"2024-04-26T20:58:43.045175+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.045175+0000","last_undegraded":"2024-04-26T20:58:43.045175+0000","last_fullsized":"2024-04-26T20:58:43.045175+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.079032+0000","last_change":"2024-04-26T20:58:43.075348+0000","last_active":"2024-04-26T20:58:43.079032+0000","last_peered":"2024-04-26T20:58:43.079032+0000","last_clean":"2024-04-26T20:58:43.079032+0000","last_became_active":"2024-04-26T20:58:43.075012+0000","last_became_peered":"2024-04-26T20:58:43.075012+0000","last_unstale":"2024-04-26T20:58:43.079032+0000","last_undegraded":"2024-04-26T20:58:43.079032+0000","last_fullsized":"2024-04-26T20:58:43.079032+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:44.341099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.045208+0000","last_change":"2024-04-26T20:58:43.045208+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.045208+0000","last_undegraded":"2024-04-26T20:58:43.045208+0000","last_fullsized":"2024-04-26T20:58:43.045208+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.064494+0000","last_change":"2024-04-26T20:58:43.054765+0000","last_active":"2024-04-26T20:58:43.064494+0000","last_peered":"2024-04-26T20:58:43.064494+0000","last_clean":"2024-04-26T20:58:43.064494+0000","last_became_active":"2024-04-26T20:58:43.054455+0000","last_became_peered":"2024-04-26T20:58:43.054455+0000","last_unstale":"2024-04-26T20:58:43.064494+0000","last_undegraded":"2024-04-26T20:58:43.064494+0000","last_fullsized":"2024-04-26T20:58:43.064494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:36.534833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.062096+0000","last_change":"2024-04-26T20:58:43.062096+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.062096+0000","last_undegraded":"2024-04-26T20:58:43.062096+0000","last_fullsized":"2024-04-26T20:58:43.062096+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.067885+0000","last_change":"2024-04-26T20:58:43.065790+0000","last_active":"2024-04-26T20:58:43.067885+0000","last_peered":"2024-04-26T20:58:43.067885+0000","last_clean":"2024-04-26T20:58:43.067885+0000","last_became_active":"2024-04-26T20:58:43.065469+0000","last_became_peered":"2024-04-26T20:58:43.065469+0000","last_unstale":"2024-04-26T20:58:43.067885+0000","last_undegraded":"2024-04-26T20:58:43.067885+0000","last_fullsized":"2024-04-26T20:58:43.067885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:28.519261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.044116+0000","last_change":"2024-04-26T20:58:43.044116+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.044116+0000","last_undegraded":"2024-04-26T20:58:43.044116+0000","last_fullsized":"2024-04-26T20:58:43.044116+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.059577+0000","last_change":"2024-04-26T20:58:43.056930+0000","last_active":"2024-04-26T20:58:43.059577+0000","last_peered":"2024-04-26T20:58:43.059577+0000","last_clean":"2024-04-26T20:58:43.059577+0000","last_became_active":"2024-04-26T20:58:43.055764+0000","last_became_peered":"2024-04-26T20:58:43.055764+0000","last_unstale":"2024-04-26T20:58:43.059577+0000","last_undegraded":"2024-04-26T20:58:43.059577+0000","last_fullsized":"2024-04-26T20:58:43.059577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:14.346315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055239+0000","last_change":"2024-04-26T20:58:43.055239+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055239+0000","last_undegraded":"2024-04-26T20:58:43.055239+0000","last_fullsized":"2024-04-26T20:58:43.055239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.076095+0000","last_change":"2024-04-26T20:58:43.074219+0000","last_active":"2024-04-26T20:58:43.076095+0000","last_peered":"2024-04-26T20:58:43.076095+0000","last_clean":"2024-04-26T20:58:43.076095+0000","last_became_active":"2024-04-26T20:58:43.072029+0000","last_became_peered":"2024-04-26T20:58:43.072029+0000","last_unstale":"2024-04-26T20:58:43.076095+0000","last_undegraded":"2024-04-26T20:58:43.076095+0000","last_fullsized":"2024-04-26T20:58:43.076095+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:58.207157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":211,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.058660+0000","last_change":"2024-04-26T20:56:46.848603+0000","last_active":"2024-04-26T20:58:43.058660+0000","last_peered":"2024-04-26T20:58:43.058660+0000","last_clean":"2024-04-26T20:58:43.058660+0000","last_became_active":"2024-04-26T20:56:46.846091+0000","last_became_peered":"2024-04-26T20:56:46.846091+0000","last_unstale":"2024-04-26T20:58:43.058660+0000","last_undegraded":"2024-04-26T20:58:43.058660+0000","last_fullsized":"2024-04-26T20:58:43.058660+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_clean_scrub_stamp":"2024-04-26T20:56:45.594041+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:00.232387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":785505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":201404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":201404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":201404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":201404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":791772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:58:49.580 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2024-04-26T20:58:49.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.055951+0000', 'last_change': '2024-04-26T20:58:43.055951+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.055951+0000', 'last_undegraded': '2024-04-26T20:58:43.055951+0000', 'last_fullsized': '2024-04-26T20:58:43.055951+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:58:49.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2024-04-26T20:58:49.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.052235+0000', 'last_change': '2024-04-26T20:58:43.052235+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.052235+0000', 'last_undegraded': '2024-04-26T20:58:43.052235+0000', 'last_fullsized': '2024-04-26T20:58:43.052235+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:58:49.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2024-04-26T20:58:49.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.048628+0000', 'last_change': '2024-04-26T20:58:43.048628+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.048628+0000', 'last_undegraded': '2024-04-26T20:58:43.048628+0000', 'last_fullsized': '2024-04-26T20:58:43.048628+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:58:49.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2024-04-26T20:58:49.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.056479+0000', 'last_change': '2024-04-26T20:58:43.056479+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.056479+0000', 'last_undegraded': '2024-04-26T20:58:43.056479+0000', 'last_fullsized': '2024-04-26T20:58:43.056479+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:58:49.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2024-04-26T20:58:49.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.056437+0000', 'last_change': '2024-04-26T20:58:43.056437+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.056437+0000', 'last_undegraded': '2024-04-26T20:58:43.056437+0000', 'last_fullsized': '2024-04-26T20:58:43.056437+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:58:49.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2024-04-26T20:58:49.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.048264+0000', 'last_change': '2024-04-26T20:58:43.048264+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.048264+0000', 'last_undegraded': '2024-04-26T20:58:43.048264+0000', 'last_fullsized': '2024-04-26T20:58:43.048264+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:58:49.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.39 is not active+clean 2024-04-26T20:58:49.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.049255+0000', 'last_change': '2024-04-26T20:58:43.049255+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.049255+0000', 'last_undegraded': '2024-04-26T20:58:43.049255+0000', 'last_fullsized': '2024-04-26T20:58:43.049255+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:58:49.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2024-04-26T20:58:49.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.051206+0000', 'last_change': '2024-04-26T20:58:43.051206+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.051206+0000', 'last_undegraded': '2024-04-26T20:58:43.051206+0000', 'last_fullsized': '2024-04-26T20:58:43.051206+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:58:49.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.37 is not active+clean 2024-04-26T20:58:49.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.053178+0000', 'last_change': '2024-04-26T20:58:43.053178+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.053178+0000', 'last_undegraded': '2024-04-26T20:58:43.053178+0000', 'last_fullsized': '2024-04-26T20:58:43.053178+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:58:49.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.36 is not active+clean 2024-04-26T20:58:49.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.056034+0000', 'last_change': '2024-04-26T20:58:43.056034+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.056034+0000', 'last_undegraded': '2024-04-26T20:58:43.056034+0000', 'last_fullsized': '2024-04-26T20:58:43.056034+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:58:49.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2024-04-26T20:58:49.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.052135+0000', 'last_change': '2024-04-26T20:58:43.052135+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.052135+0000', 'last_undegraded': '2024-04-26T20:58:43.052135+0000', 'last_fullsized': '2024-04-26T20:58:43.052135+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:58:49.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2024-04-26T20:58:49.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.060146+0000', 'last_change': '2024-04-26T20:58:43.060146+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.060146+0000', 'last_undegraded': '2024-04-26T20:58:43.060146+0000', 'last_fullsized': '2024-04-26T20:58:43.060146+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:58:49.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2024-04-26T20:58:49.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.038802+0000', 'last_change': '2024-04-26T20:58:43.038802+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.038802+0000', 'last_undegraded': '2024-04-26T20:58:43.038802+0000', 'last_fullsized': '2024-04-26T20:58:43.038802+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:58:49.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2024-04-26T20:58:49.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.051120+0000', 'last_change': '2024-04-26T20:58:43.051120+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.051120+0000', 'last_undegraded': '2024-04-26T20:58:43.051120+0000', 'last_fullsized': '2024-04-26T20:58:43.051120+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:58:49.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2024-04-26T20:58:49.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.049339+0000', 'last_change': '2024-04-26T20:58:43.049339+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.049339+0000', 'last_undegraded': '2024-04-26T20:58:43.049339+0000', 'last_fullsized': '2024-04-26T20:58:43.049339+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:58:49.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2024-04-26T20:58:49.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.061785+0000', 'last_change': '2024-04-26T20:58:43.061785+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.061785+0000', 'last_undegraded': '2024-04-26T20:58:43.061785+0000', 'last_fullsized': '2024-04-26T20:58:43.061785+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:58:49.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2024-04-26T20:58:49.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.049592+0000', 'last_change': '2024-04-26T20:58:43.049592+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.049592+0000', 'last_undegraded': '2024-04-26T20:58:43.049592+0000', 'last_fullsized': '2024-04-26T20:58:43.049592+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:58:49.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2024-04-26T20:58:49.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.051921+0000', 'last_change': '2024-04-26T20:58:43.051921+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.051921+0000', 'last_undegraded': '2024-04-26T20:58:43.051921+0000', 'last_fullsized': '2024-04-26T20:58:43.051921+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:58:49.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2024-04-26T20:58:49.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.056057+0000', 'last_change': '2024-04-26T20:58:43.056057+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.056057+0000', 'last_undegraded': '2024-04-26T20:58:43.056057+0000', 'last_fullsized': '2024-04-26T20:58:43.056057+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:58:49.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2c is not active+clean 2024-04-26T20:58:49.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.039248+0000', 'last_change': '2024-04-26T20:58:43.039248+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.039248+0000', 'last_undegraded': '2024-04-26T20:58:43.039248+0000', 'last_fullsized': '2024-04-26T20:58:43.039248+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:58:49.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2024-04-26T20:58:49.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.047349+0000', 'last_change': '2024-04-26T20:58:43.047349+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.047349+0000', 'last_undegraded': '2024-04-26T20:58:43.047349+0000', 'last_fullsized': '2024-04-26T20:58:43.047349+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:58:49.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2a is not active+clean 2024-04-26T20:58:49.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.046758+0000', 'last_change': '2024-04-26T20:58:43.046758+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.046758+0000', 'last_undegraded': '2024-04-26T20:58:43.046758+0000', 'last_fullsized': '2024-04-26T20:58:43.046758+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:58:49.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.29 is not active+clean 2024-04-26T20:58:49.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.059090+0000', 'last_change': '2024-04-26T20:58:43.059090+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.059090+0000', 'last_undegraded': '2024-04-26T20:58:43.059090+0000', 'last_fullsized': '2024-04-26T20:58:43.059090+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:58:49.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2024-04-26T20:58:49.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.060755+0000', 'last_change': '2024-04-26T20:58:43.060755+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.060755+0000', 'last_undegraded': '2024-04-26T20:58:43.060755+0000', 'last_fullsized': '2024-04-26T20:58:43.060755+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:58:49.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2024-04-26T20:58:49.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.064938+0000', 'last_change': '2024-04-26T20:58:43.064938+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.064938+0000', 'last_undegraded': '2024-04-26T20:58:43.064938+0000', 'last_fullsized': '2024-04-26T20:58:43.064938+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:58:49.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2024-04-26T20:58:49.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.040818+0000', 'last_change': '2024-04-26T20:58:43.040818+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.040818+0000', 'last_undegraded': '2024-04-26T20:58:43.040818+0000', 'last_fullsized': '2024-04-26T20:58:43.040818+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:58:49.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2024-04-26T20:58:49.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.046134+0000', 'last_change': '2024-04-26T20:58:43.046134+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.046134+0000', 'last_undegraded': '2024-04-26T20:58:43.046134+0000', 'last_fullsized': '2024-04-26T20:58:43.046134+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:58:49.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2024-04-26T20:58:49.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.058202+0000', 'last_change': '2024-04-26T20:58:43.058202+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.058202+0000', 'last_undegraded': '2024-04-26T20:58:43.058202+0000', 'last_fullsized': '2024-04-26T20:58:43.058202+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:58:49.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2024-04-26T20:58:49.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.056143+0000', 'last_change': '2024-04-26T20:58:43.056143+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.056143+0000', 'last_undegraded': '2024-04-26T20:58:43.056143+0000', 'last_fullsized': '2024-04-26T20:58:43.056143+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:58:49.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2024-04-26T20:58:49.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.047071+0000', 'last_change': '2024-04-26T20:58:43.047071+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.047071+0000', 'last_undegraded': '2024-04-26T20:58:43.047071+0000', 'last_fullsized': '2024-04-26T20:58:43.047071+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:58:49.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2024-04-26T20:58:49.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.047975+0000', 'last_change': '2024-04-26T20:58:43.047975+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.047975+0000', 'last_undegraded': '2024-04-26T20:58:43.047975+0000', 'last_fullsized': '2024-04-26T20:58:43.047975+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:58:49.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2024-04-26T20:58:49.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.040702+0000', 'last_change': '2024-04-26T20:58:43.040702+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.040702+0000', 'last_undegraded': '2024-04-26T20:58:43.040702+0000', 'last_fullsized': '2024-04-26T20:58:43.040702+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:58:49.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2024-04-26T20:58:49.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.053161+0000', 'last_change': '2024-04-26T20:58:43.053161+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.053161+0000', 'last_undegraded': '2024-04-26T20:58:43.053161+0000', 'last_fullsized': '2024-04-26T20:58:43.053161+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:58:49.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2024-04-26T20:58:49.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.064417+0000', 'last_change': '2024-04-26T20:58:43.064417+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.064417+0000', 'last_undegraded': '2024-04-26T20:58:43.064417+0000', 'last_fullsized': '2024-04-26T20:58:43.064417+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:58:49.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2024-04-26T20:58:49.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.057716+0000', 'last_change': '2024-04-26T20:58:43.057716+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.057716+0000', 'last_undegraded': '2024-04-26T20:58:43.057716+0000', 'last_fullsized': '2024-04-26T20:58:43.057716+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:58:49.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2024-04-26T20:58:49.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.055248+0000', 'last_change': '2024-04-26T20:58:43.055248+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.055248+0000', 'last_undegraded': '2024-04-26T20:58:43.055248+0000', 'last_fullsized': '2024-04-26T20:58:43.055248+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:58:49.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2024-04-26T20:58:49.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.045079+0000', 'last_change': '2024-04-26T20:58:43.045079+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.045079+0000', 'last_undegraded': '2024-04-26T20:58:43.045079+0000', 'last_fullsized': '2024-04-26T20:58:43.045079+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:58:49.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2024-04-26T20:58:49.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.055253+0000', 'last_change': '2024-04-26T20:58:43.055253+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.055253+0000', 'last_undegraded': '2024-04-26T20:58:43.055253+0000', 'last_fullsized': '2024-04-26T20:58:43.055253+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:58:49.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2024-04-26T20:58:49.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.050736+0000', 'last_change': '2024-04-26T20:58:43.050736+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.050736+0000', 'last_undegraded': '2024-04-26T20:58:43.050736+0000', 'last_fullsized': '2024-04-26T20:58:43.050736+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:58:49.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2024-04-26T20:58:49.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.059706+0000', 'last_change': '2024-04-26T20:58:43.059706+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.059706+0000', 'last_undegraded': '2024-04-26T20:58:43.059706+0000', 'last_fullsized': '2024-04-26T20:58:43.059706+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:58:49.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2024-04-26T20:58:49.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.050460+0000', 'last_change': '2024-04-26T20:58:43.050460+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.050460+0000', 'last_undegraded': '2024-04-26T20:58:43.050460+0000', 'last_fullsized': '2024-04-26T20:58:43.050460+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:58:49.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2024-04-26T20:58:49.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.047092+0000', 'last_change': '2024-04-26T20:58:43.047092+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.047092+0000', 'last_undegraded': '2024-04-26T20:58:43.047092+0000', 'last_fullsized': '2024-04-26T20:58:43.047092+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:58:49.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-04-26T20:58:49.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.046401+0000', 'last_change': '2024-04-26T20:58:43.046401+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.046401+0000', 'last_undegraded': '2024-04-26T20:58:43.046401+0000', 'last_fullsized': '2024-04-26T20:58:43.046401+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:58:49.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2024-04-26T20:58:49.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.049406+0000', 'last_change': '2024-04-26T20:58:43.049406+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.049406+0000', 'last_undegraded': '2024-04-26T20:58:43.049406+0000', 'last_fullsized': '2024-04-26T20:58:43.049406+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:58:49.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2024-04-26T20:58:49.597 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.055956+0000', 'last_change': '2024-04-26T20:58:43.055956+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.055956+0000', 'last_undegraded': '2024-04-26T20:58:43.055956+0000', 'last_fullsized': '2024-04-26T20:58:43.055956+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:58:49.597 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2024-04-26T20:58:49.597 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.054665+0000', 'last_change': '2024-04-26T20:58:43.054665+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.054665+0000', 'last_undegraded': '2024-04-26T20:58:43.054665+0000', 'last_fullsized': '2024-04-26T20:58:43.054665+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:58:49.597 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2024-04-26T20:58:49.597 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.045954+0000', 'last_change': '2024-04-26T20:58:43.045954+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.045954+0000', 'last_undegraded': '2024-04-26T20:58:43.045954+0000', 'last_fullsized': '2024-04-26T20:58:43.045954+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:58:49.598 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2024-04-26T20:58:49.598 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.048146+0000', 'last_change': '2024-04-26T20:58:43.048146+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.048146+0000', 'last_undegraded': '2024-04-26T20:58:43.048146+0000', 'last_fullsized': '2024-04-26T20:58:43.048146+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:58:49.598 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2024-04-26T20:58:49.598 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.039364+0000', 'last_change': '2024-04-26T20:58:43.039364+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.039364+0000', 'last_undegraded': '2024-04-26T20:58:43.039364+0000', 'last_fullsized': '2024-04-26T20:58:43.039364+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:58:49.598 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2024-04-26T20:58:49.598 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.051926+0000', 'last_change': '2024-04-26T20:58:43.051926+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.051926+0000', 'last_undegraded': '2024-04-26T20:58:43.051926+0000', 'last_fullsized': '2024-04-26T20:58:43.051926+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:58:49.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2024-04-26T20:58:49.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.064001+0000', 'last_change': '2024-04-26T20:58:43.064001+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.064001+0000', 'last_undegraded': '2024-04-26T20:58:43.064001+0000', 'last_fullsized': '2024-04-26T20:58:43.064001+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:58:49.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2024-04-26T20:58:49.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.056614+0000', 'last_change': '2024-04-26T20:58:43.056614+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.056614+0000', 'last_undegraded': '2024-04-26T20:58:43.056614+0000', 'last_fullsized': '2024-04-26T20:58:43.056614+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:58:49.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2024-04-26T20:58:49.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.045725+0000', 'last_change': '2024-04-26T20:58:43.045725+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.045725+0000', 'last_undegraded': '2024-04-26T20:58:43.045725+0000', 'last_fullsized': '2024-04-26T20:58:43.045725+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:58:49.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2024-04-26T20:58:49.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.050764+0000', 'last_change': '2024-04-26T20:58:43.050764+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.050764+0000', 'last_undegraded': '2024-04-26T20:58:43.050764+0000', 'last_fullsized': '2024-04-26T20:58:43.050764+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T20:58:49.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2024-04-26T20:58:49.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.038929+0000', 'last_change': '2024-04-26T20:58:43.038929+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.038929+0000', 'last_undegraded': '2024-04-26T20:58:43.038929+0000', 'last_fullsized': '2024-04-26T20:58:43.038929+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T20:58:49.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2024-04-26T20:58:49.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.057380+0000', 'last_change': '2024-04-26T20:58:43.057380+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.057380+0000', 'last_undegraded': '2024-04-26T20:58:43.057380+0000', 'last_fullsized': '2024-04-26T20:58:43.057380+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:58:49.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2024-04-26T20:58:49.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.063901+0000', 'last_change': '2024-04-26T20:58:43.063901+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.063901+0000', 'last_undegraded': '2024-04-26T20:58:43.063901+0000', 'last_fullsized': '2024-04-26T20:58:43.063901+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T20:58:49.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2024-04-26T20:58:49.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.055227+0000', 'last_change': '2024-04-26T20:58:43.055227+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.055227+0000', 'last_undegraded': '2024-04-26T20:58:43.055227+0000', 'last_fullsized': '2024-04-26T20:58:43.055227+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:58:49.602 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2024-04-26T20:58:49.602 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.047448+0000', 'last_change': '2024-04-26T20:58:43.047448+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.047448+0000', 'last_undegraded': '2024-04-26T20:58:43.047448+0000', 'last_fullsized': '2024-04-26T20:58:43.047448+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T20:58:49.602 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2024-04-26T20:58:49.602 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.045175+0000', 'last_change': '2024-04-26T20:58:43.045175+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.045175+0000', 'last_undegraded': '2024-04-26T20:58:43.045175+0000', 'last_fullsized': '2024-04-26T20:58:43.045175+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:58:49.603 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-04-26T20:58:49.603 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.045208+0000', 'last_change': '2024-04-26T20:58:43.045208+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.045208+0000', 'last_undegraded': '2024-04-26T20:58:43.045208+0000', 'last_fullsized': '2024-04-26T20:58:43.045208+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T20:58:49.603 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2024-04-26T20:58:49.603 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.062096+0000', 'last_change': '2024-04-26T20:58:43.062096+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.062096+0000', 'last_undegraded': '2024-04-26T20:58:43.062096+0000', 'last_fullsized': '2024-04-26T20:58:43.062096+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T20:58:49.603 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2024-04-26T20:58:49.603 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.044116+0000', 'last_change': '2024-04-26T20:58:43.044116+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.044116+0000', 'last_undegraded': '2024-04-26T20:58:43.044116+0000', 'last_fullsized': '2024-04-26T20:58:43.044116+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T20:58:49.604 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2024-04-26T20:58:49.604 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 43, 'state': 'creating+peering', 'last_fresh': '2024-04-26T20:58:43.055239+0000', 'last_change': '2024-04-26T20:58:43.055239+0000', 'last_active': '2024-04-26T20:58:43.016772+0000', 'last_peered': '2024-04-26T20:58:43.016772+0000', 'last_clean': '2024-04-26T20:58:43.016772+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T20:58:43.055239+0000', 'last_undegraded': '2024-04-26T20:58:43.055239+0000', 'last_fullsized': '2024-04-26T20:58:43.055239+0000', 'mapping_epoch': 43, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 43, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'last_clean_scrub_stamp': '2024-04-26T20:58:43.016772+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T20:58:49.605 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T20:58:49.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.832+0000 7fe93f200640 1 Processor -- start 2024-04-26T20:58:49.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.832+0000 7fe93f200640 1 -- start start 2024-04-26T20:58:49.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.834+0000 7fe93f200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe93815d850 0x7fe93815dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:49.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.834+0000 7fe93f200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe93815e170 0x7fe9380d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:49.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.834+0000 7fe93f200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9380b5d20 con 0x7fe9380d13e0 2024-04-26T20:58:49.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.834+0000 7fe93f200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe9380cf060 con 0x7fe93815d850 2024-04-26T20:58:49.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.834+0000 7fe93f200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fe9380d1830 con 0x7fe93815e170 2024-04-26T20:58:49.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.834+0000 7fe93e800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe9380d13e0 0x7fe938160a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:52976/0 (socket says 172.21.15.28:52976) 2024-04-26T20:58:49.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.834+0000 7fe93e800640 1 -- 172.21.15.28:0/390252332 learned_addr learned my addr 172.21.15.28:0/390252332 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:49.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.834+0000 7fe93de00640 1 --2- 172.21.15.28:0/390252332 >> v2:172.21.15.28:3300/0 conn(0x7fe93815d850 0x7fe93815dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:49.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.835+0000 7fe93d400640 1 --2- 172.21.15.28:0/390252332 >> v2:172.21.15.103:3300/0 conn(0x7fe93815e170 0x7fe9380d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:49.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.835+0000 7fe93ca00640 1 -- 172.21.15.28:0/390252332 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3990075841 0 0) 0x7fe9380b5d20 con 0x7fe9380d13e0 2024-04-26T20:58:49.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.835+0000 7fe93ca00640 1 -- 172.21.15.28:0/390252332 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe920003520 con 0x7fe9380d13e0 2024-04-26T20:58:49.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.835+0000 7fe93de00640 1 -- 172.21.15.28:0/390252332 >> v2:172.21.15.103:3300/0 conn(0x7fe93815e170 msgr2=0x7fe9380d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:49.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.835+0000 7fe93de00640 1 --2- 172.21.15.28:0/390252332 >> v2:172.21.15.103:3300/0 conn(0x7fe93815e170 0x7fe9380d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.835+0000 7fe93de00640 1 -- 172.21.15.28:0/390252332 >> v1:172.21.15.28:6789/0 conn(0x7fe9380d13e0 legacy=0x7fe938160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:49.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.835+0000 7fe93de00640 1 -- 172.21.15.28:0/390252332 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9380722f0 con 0x7fe93815d850 2024-04-26T20:58:49.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.836+0000 7fe93de00640 1 --2- 172.21.15.28:0/390252332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe93815d850 0x7fe93815dc30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe938071860 tx=0x7fe92c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67fcec347c2e977 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:49.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.836+0000 7fe93ca00640 1 -- 172.21.15.28:0/390252332 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c037070 con 0x7fe93815d850 2024-04-26T20:58:49.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.837+0000 7fe93ca00640 1 -- 172.21.15.28:0/390252332 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe92c032440 con 0x7fe93815d850 2024-04-26T20:58:49.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.837+0000 7fe93ca00640 1 -- 172.21.15.28:0/390252332 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c032810 con 0x7fe93815d850 2024-04-26T20:58:49.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.837+0000 7fe93f200640 1 -- 172.21.15.28:0/390252332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe93815d850 msgr2=0x7fe93815dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:49.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.837+0000 7fe93f200640 1 --2- 172.21.15.28:0/390252332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe93815d850 0x7fe93815dc30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe938071860 tx=0x7fe92c02a1a0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.837+0000 7fe93f200640 1 -- 172.21.15.28:0/390252332 shutdown_connections 2024-04-26T20:58:49.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.837+0000 7fe93f200640 1 --2- 172.21.15.28:0/390252332 >> v2:172.21.15.103:3300/0 conn(0x7fe93815e170 0x7fe9380d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.837+0000 7fe93f200640 1 --2- 172.21.15.28:0/390252332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe93815d850 0x7fe93815dc30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.837+0000 7fe93f200640 1 -- 172.21.15.28:0/390252332 >> 172.21.15.28:0/390252332 conn(0x7fe93806c770 msgr2=0x7fe93806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:49.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.837+0000 7fe93f200640 1 -- 172.21.15.28:0/390252332 shutdown_connections 2024-04-26T20:58:49.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.837+0000 7fe93f200640 1 -- 172.21.15.28:0/390252332 wait complete. 2024-04-26T20:58:49.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.839+0000 7fe93f200640 1 Processor -- start 2024-04-26T20:58:49.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.839+0000 7fe93f200640 1 -- start start 2024-04-26T20:58:49.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.840+0000 7fe93f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9380d13e0 0x7fe938142be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:49.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.840+0000 7fe93f200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe93815d850 0x7fe938153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:49.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.840+0000 7fe93f200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe93815e170 0x7fe938143480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:49.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.840+0000 7fe93f200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe9380728b0 con 0x7fe93815e170 2024-04-26T20:58:49.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.840+0000 7fe93f200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe938072e70 con 0x7fe9380d13e0 2024-04-26T20:58:49.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.840+0000 7fe93f200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe938072b90 con 0x7fe93815d850 2024-04-26T20:58:49.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.840+0000 7fe93e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe93815e170 0x7fe938143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:49.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.841+0000 7fe93e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe93815e170 0x7fe938143480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47024/0 (socket says 172.21.15.28:47024) 2024-04-26T20:58:49.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.841+0000 7fe93e800640 1 -- 172.21.15.28:0/3177735849 learned_addr learned my addr 172.21.15.28:0/3177735849 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:49.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.841+0000 7fe93d400640 1 --2- 172.21.15.28:0/3177735849 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe93815d850 0x7fe938153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:49.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.841+0000 7fe93de00640 1 --2- 172.21.15.28:0/3177735849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9380d13e0 0x7fe938142be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:49.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.841+0000 7fe93e800640 1 -- 172.21.15.28:0/3177735849 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe93815d850 msgr2=0x7fe938153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:49.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.841+0000 7fe93e800640 1 --2- 172.21.15.28:0/3177735849 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe93815d850 0x7fe938153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.841+0000 7fe93e800640 1 -- 172.21.15.28:0/3177735849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9380d13e0 msgr2=0x7fe938142be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:49.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.841+0000 7fe93e800640 1 --2- 172.21.15.28:0/3177735849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9380d13e0 0x7fe938142be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:49.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.841+0000 7fe93e800640 1 -- 172.21.15.28:0/3177735849 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe938153000 con 0x7fe93815e170 2024-04-26T20:58:49.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.841+0000 7fe93e800640 1 --2- 172.21.15.28:0/3177735849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe93815e170 0x7fe938143480 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe93400cb70 tx=0x7fe93400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:49.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.842+0000 7fe932000640 1 -- 172.21.15.28:0/3177735849 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934017070 con 0x7fe93815e170 2024-04-26T20:58:49.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.842+0000 7fe932000640 1 -- 172.21.15.28:0/3177735849 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe93400ad40 con 0x7fe93815e170 2024-04-26T20:58:49.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.842+0000 7fe932000640 1 -- 172.21.15.28:0/3177735849 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe934012620 con 0x7fe93815e170 2024-04-26T20:58:49.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.842+0000 7fe93f200640 1 -- 172.21.15.28:0/3177735849 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93815bda0 con 0x7fe93815e170 2024-04-26T20:58:49.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.842+0000 7fe93f200640 1 -- 172.21.15.28:0/3177735849 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe938143120 con 0x7fe93815e170 2024-04-26T20:58:49.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.843+0000 7fe93f200640 1 -- 172.21.15.28:0/3177735849 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe93806a360 con 0x7fe93815e170 2024-04-26T20:58:49.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.844+0000 7fe932000640 1 -- 172.21.15.28:0/3177735849 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe93401a070 con 0x7fe93815e170 2024-04-26T20:58:49.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.846+0000 7fe932000640 1 --2- 172.21.15.28:0/3177735849 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe91c06c8f0 0x7fe91c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:49.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.846+0000 7fe932000640 1 -- 172.21.15.28:0/3177735849 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fe93408b9a0 con 0x7fe93815e170 2024-04-26T20:58:49.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.846+0000 7fe93de00640 1 --2- 172.21.15.28:0/3177735849 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe91c06c8f0 0x7fe91c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:49.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.847+0000 7fe93de00640 1 --2- 172.21.15.28:0/3177735849 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe91c06c8f0 0x7fe91c06edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe92c034000 tx=0x7fe92c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:49.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:49.850+0000 7fe932000640 1 -- 172.21.15.28:0/3177735849 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe934059dd0 con 0x7fe93815e170 2024-04-26T20:58:49.998 INFO:tasks.ceph.mds.c.smithi028.stderr:2024-04-26T20:58:49.997+0000 7f2586400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T20:58:50.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.129+0000 7fe93f200640 1 -- 172.21.15.28:0/3177735849 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fe938073cc0 con 0x7fe93815e170 2024-04-26T20:58:50.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.130+0000 7fe932000640 1 -- 172.21.15.28:0/3177735849 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1034 (secure 0 0 0) 0x7fe9340596d0 con 0x7fe93815e170 2024-04-26T20:58:50.131 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:50.131 INFO:teuthology.orchestra.run.smithi028.stdout:{"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":138,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":8,"num_up_osds":8,"osd_up_since":1714165003,"num_in_osds":8,"osd_in_since":1714164994,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":243585024,"bytes_avail":772850528256,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":26,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"c","status":"up:active","gid":5571}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-04-26T20:57:54.726657+0000","services":{}},"progress_events":{}} 2024-04-26T20:58:50.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.132+0000 7fe93f200640 1 -- 172.21.15.28:0/3177735849 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe91c06c8f0 msgr2=0x7fe91c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:50.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.132+0000 7fe93f200640 1 --2- 172.21.15.28:0/3177735849 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe91c06c8f0 0x7fe91c06edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe92c034000 tx=0x7fe92c02e000 comp rx=0 tx=0).stop 2024-04-26T20:58:50.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.132+0000 7fe93f200640 1 -- 172.21.15.28:0/3177735849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe93815e170 msgr2=0x7fe938143480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:50.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.132+0000 7fe93f200640 1 --2- 172.21.15.28:0/3177735849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe93815e170 0x7fe938143480 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe93400cb70 tx=0x7fe93400a430 comp rx=0 tx=0).stop 2024-04-26T20:58:50.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.133+0000 7fe93f200640 1 -- 172.21.15.28:0/3177735849 shutdown_connections 2024-04-26T20:58:50.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.133+0000 7fe93f200640 1 --2- 172.21.15.28:0/3177735849 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe91c06c8f0 0x7fe91c06edb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:50.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.133+0000 7fe93f200640 1 --2- 172.21.15.28:0/3177735849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe93815e170 0x7fe938143480 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:50.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.133+0000 7fe93f200640 1 --2- 172.21.15.28:0/3177735849 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe93815d850 0x7fe938153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:50.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.133+0000 7fe93f200640 1 --2- 172.21.15.28:0/3177735849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9380d13e0 0x7fe938142be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:50.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.133+0000 7fe93f200640 1 -- 172.21.15.28:0/3177735849 >> 172.21.15.28:0/3177735849 conn(0x7fe93806c770 msgr2=0x7fe9380ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:50.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.133+0000 7fe93f200640 1 -- 172.21.15.28:0/3177735849 shutdown_connections 2024-04-26T20:58:50.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.133+0000 7fe93f200640 1 -- 172.21.15.28:0/3177735849 wait complete. 2024-04-26T20:58:50.152 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T20:58:50.152 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:58:50.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.357+0000 7fc4a6600640 1 Processor -- start 2024-04-26T20:58:50.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.357+0000 7fc4a6600640 1 -- start start 2024-04-26T20:58:50.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.359+0000 7fc4a6600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc4a0146670 0x7fc4a0071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:50.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.359+0000 7fc4a6600640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fc4a01610c0 0x7fc4a00cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:50.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.359+0000 7fc4a6600640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4a0146440 con 0x7fc4a0146060 2024-04-26T20:58:50.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.359+0000 7fc4a6600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fc4a006f510 con 0x7fc4a0146670 2024-04-26T20:58:50.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.359+0000 7fc4a6600640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fc4a006f010 con 0x7fc4a01610c0 2024-04-26T20:58:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.359+0000 7fc49fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc4a0146670 0x7fc4a0071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.359+0000 7fc49fe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc4a0146670 0x7fc4a0071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47036/0 (socket says 172.21.15.28:47036) 2024-04-26T20:58:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.359+0000 7fc49fe00640 1 -- 172.21.15.28:0/3028197073 learned_addr learned my addr 172.21.15.28:0/3028197073 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.360+0000 7fc4a5c00640 1 --2- 172.21.15.28:0/3028197073 >> v2:172.21.15.103:3300/0 conn(0x7fc4a01610c0 0x7fc4a00cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.360+0000 7fc49fe00640 1 -- 172.21.15.28:0/3028197073 >> v2:172.21.15.103:3300/0 conn(0x7fc4a01610c0 msgr2=0x7fc4a00cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.360+0000 7fc49fe00640 1 --2- 172.21.15.28:0/3028197073 >> v2:172.21.15.103:3300/0 conn(0x7fc4a01610c0 0x7fc4a00cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:50.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.360+0000 7fc49fe00640 1 -- 172.21.15.28:0/3028197073 >> v1:172.21.15.103:6789/0 conn(0x7fc4a0146060 legacy=0x7fc4a01609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:50.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.360+0000 7fc49fe00640 1 -- 172.21.15.28:0/3028197073 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a01614d0 con 0x7fc4a0146670 2024-04-26T20:58:50.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.361+0000 7fc49fe00640 1 --2- 172.21.15.28:0/3028197073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4a0146670 0x7fc4a0071e20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0072360 tx=0x7fc48c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744bb42adb15d4f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:50.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.361+0000 7fc49f400640 1 -- 172.21.15.28:0/3028197073 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c037070 con 0x7fc4a0146670 2024-04-26T20:58:50.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.361+0000 7fc49f400640 1 -- 172.21.15.28:0/3028197073 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc48c032440 con 0x7fc4a0146670 2024-04-26T20:58:50.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.361+0000 7fc49f400640 1 -- 172.21.15.28:0/3028197073 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48c032810 con 0x7fc4a0146670 2024-04-26T20:58:50.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.362+0000 7fc4a6600640 1 -- 172.21.15.28:0/3028197073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4a0146670 msgr2=0x7fc4a0071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:50.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.362+0000 7fc4a6600640 1 --2- 172.21.15.28:0/3028197073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4a0146670 0x7fc4a0071e20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0072360 tx=0x7fc48c02a1a0 comp rx=0 tx=0).stop 2024-04-26T20:58:50.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.362+0000 7fc4a6600640 1 -- 172.21.15.28:0/3028197073 shutdown_connections 2024-04-26T20:58:50.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.362+0000 7fc4a6600640 1 --2- 172.21.15.28:0/3028197073 >> v2:172.21.15.103:3300/0 conn(0x7fc4a01610c0 0x7fc4a00cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:50.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.362+0000 7fc4a6600640 1 --2- 172.21.15.28:0/3028197073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4a0146670 0x7fc4a0071e20 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:50.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.362+0000 7fc4a6600640 1 -- 172.21.15.28:0/3028197073 >> 172.21.15.28:0/3028197073 conn(0x7fc4a00d0a50 msgr2=0x7fc4a00d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:50.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.362+0000 7fc4a6600640 1 -- 172.21.15.28:0/3028197073 shutdown_connections 2024-04-26T20:58:50.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.362+0000 7fc4a6600640 1 -- 172.21.15.28:0/3028197073 wait complete. 2024-04-26T20:58:50.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.364+0000 7fc4a6600640 1 Processor -- start 2024-04-26T20:58:50.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.364+0000 7fc4a6600640 1 -- start start 2024-04-26T20:58:50.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.365+0000 7fc4a6600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4a0146060 0x7fc4a015d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:50.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.365+0000 7fc4a6600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4a0146670 0x7fc4a00b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:50.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.365+0000 7fc4a6600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4a01610c0 0x7fc4a00b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:50.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.365+0000 7fc4a6600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a006ff60 con 0x7fc4a01610c0 2024-04-26T20:58:50.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.365+0000 7fc4a6600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a006fc80 con 0x7fc4a0146060 2024-04-26T20:58:50.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.365+0000 7fc4a6600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc4a00cb5e0 con 0x7fc4a0146670 2024-04-26T20:58:50.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.365+0000 7fc4a5c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4a01610c0 0x7fc4a00b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:50.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.365+0000 7fc4a5c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4a01610c0 0x7fc4a00b3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47052/0 (socket says 172.21.15.28:47052) 2024-04-26T20:58:50.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.365+0000 7fc4a5c00640 1 -- 172.21.15.28:0/3227691351 learned_addr learned my addr 172.21.15.28:0/3227691351 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:50.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.366+0000 7fc4a5200640 1 --2- 172.21.15.28:0/3227691351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4a0146060 0x7fc4a015d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:50.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.366+0000 7fc49fe00640 1 --2- 172.21.15.28:0/3227691351 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4a0146670 0x7fc4a00b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:50.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.366+0000 7fc4a5c00640 1 -- 172.21.15.28:0/3227691351 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4a0146670 msgr2=0x7fc4a00b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:50.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.366+0000 7fc4a5c00640 1 --2- 172.21.15.28:0/3227691351 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4a0146670 0x7fc4a00b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:50.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.366+0000 7fc4a5c00640 1 -- 172.21.15.28:0/3227691351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4a0146060 msgr2=0x7fc4a015d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:50.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.366+0000 7fc4a5c00640 1 --2- 172.21.15.28:0/3227691351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4a0146060 0x7fc4a015d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:50.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.366+0000 7fc4a5c00640 1 -- 172.21.15.28:0/3227691351 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a015dc60 con 0x7fc4a01610c0 2024-04-26T20:58:50.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.366+0000 7fc4a5200640 1 --2- 172.21.15.28:0/3227691351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4a0146060 0x7fc4a015d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:50.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.366+0000 7fc49fe00640 1 --2- 172.21.15.28:0/3227691351 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4a0146670 0x7fc4a00b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:50.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.366+0000 7fc4a5c00640 1 --2- 172.21.15.28:0/3227691351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4a01610c0 0x7fc4a00b3d50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc498007e90 tx=0x7fc49800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:50.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.367+0000 7fc49cc00640 1 -- 172.21.15.28:0/3227691351 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498014420 con 0x7fc4a01610c0 2024-04-26T20:58:50.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.367+0000 7fc49cc00640 1 -- 172.21.15.28:0/3227691351 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc498019070 con 0x7fc4a01610c0 2024-04-26T20:58:50.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.367+0000 7fc49cc00640 1 -- 172.21.15.28:0/3227691351 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498014800 con 0x7fc4a01610c0 2024-04-26T20:58:50.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.367+0000 7fc4a6600640 1 -- 172.21.15.28:0/3227691351 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a00b4380 con 0x7fc4a01610c0 2024-04-26T20:58:50.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.367+0000 7fc4a6600640 1 -- 172.21.15.28:0/3227691351 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a00b4940 con 0x7fc4a01610c0 2024-04-26T20:58:50.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.368+0000 7fc4a6600640 1 -- 172.21.15.28:0/3227691351 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4a006a360 con 0x7fc4a01610c0 2024-04-26T20:58:50.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.370+0000 7fc49cc00640 1 -- 172.21.15.28:0/3227691351 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4980149a0 con 0x7fc4a01610c0 2024-04-26T20:58:50.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.371+0000 7fc49cc00640 1 --2- 172.21.15.28:0/3227691351 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc48006c8f0 0x7fc48006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:50.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.371+0000 7fc49cc00640 1 -- 172.21.15.28:0/3227691351 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fc49808d300 con 0x7fc4a01610c0 2024-04-26T20:58:50.372 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.371+0000 7fc4a5200640 1 --2- 172.21.15.28:0/3227691351 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc48006c8f0 0x7fc48006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:50.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.372+0000 7fc4a5200640 1 --2- 172.21.15.28:0/3227691351 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc48006c8f0 0x7fc48006edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc494003eb0 tx=0x7fc494009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:50.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.374+0000 7fc49cc00640 1 -- 172.21.15.28:0/3227691351 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc4a015dc60 con 0x7fc4a01610c0 2024-04-26T20:58:50.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.572+0000 7fc4a6600640 1 -- 172.21.15.28:0/3227691351 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc4a0071ec0 con 0x7fc48006c8f0 2024-04-26T20:58:50.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.580+0000 7fc49cc00640 1 -- 172.21.15.28:0/3227691351 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246636 (secure 0 0 0) 0x7fc494054380 con 0x7fc48006c8f0 2024-04-26T20:58:50.584 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:50.586 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T20:58:50.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.585+0000 7fc4a6600640 1 -- 172.21.15.28:0/3227691351 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc48006c8f0 msgr2=0x7fc48006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:50.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.585+0000 7fc4a6600640 1 --2- 172.21.15.28:0/3227691351 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc48006c8f0 0x7fc48006edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc494003eb0 tx=0x7fc494009040 comp rx=0 tx=0).stop 2024-04-26T20:58:50.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.585+0000 7fc4a6600640 1 -- 172.21.15.28:0/3227691351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4a01610c0 msgr2=0x7fc4a00b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:50.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.585+0000 7fc4a6600640 1 --2- 172.21.15.28:0/3227691351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4a01610c0 0x7fc4a00b3d50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc498007e90 tx=0x7fc49800e4b0 comp rx=0 tx=0).stop 2024-04-26T20:58:50.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.585+0000 7fc4a6600640 1 -- 172.21.15.28:0/3227691351 shutdown_connections 2024-04-26T20:58:50.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.585+0000 7fc4a6600640 1 --2- 172.21.15.28:0/3227691351 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc48006c8f0 0x7fc48006edb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:50.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.585+0000 7fc4a6600640 1 --2- 172.21.15.28:0/3227691351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4a01610c0 0x7fc4a00b3d50 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:50.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.585+0000 7fc4a6600640 1 --2- 172.21.15.28:0/3227691351 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4a0146670 0x7fc4a00b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:50.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.585+0000 7fc4a6600640 1 --2- 172.21.15.28:0/3227691351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4a0146060 0x7fc4a015d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:50.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.585+0000 7fc4a6600640 1 -- 172.21.15.28:0/3227691351 >> 172.21.15.28:0/3227691351 conn(0x7fc4a00d0a50 msgr2=0x7fc4a00b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:50.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.585+0000 7fc4a6600640 1 -- 172.21.15.28:0/3227691351 shutdown_connections 2024-04-26T20:58:50.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:50.585+0000 7fc4a6600640 1 -- 172.21.15.28:0/3227691351 wait complete. 2024-04-26T20:58:50.605 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-04-26T20:58:48.741891+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237876,"kb_used_data":5060,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736844,"statfs":{"total":773094113280,"available":772850528256,"internally_reserved":0,"allocated":5181440,"data_stored":2773167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001436"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055951+0000","last_change":"2024-04-26T20:58:43.055951+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055951+0000","last_undegraded":"2024-04-26T20:58:43.055951+0000","last_fullsized":"2024-04-26T20:58:43.055951+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.052235+0000","last_change":"2024-04-26T20:58:43.052235+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.052235+0000","last_undegraded":"2024-04-26T20:58:43.052235+0000","last_fullsized":"2024-04-26T20:58:43.052235+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.048628+0000","last_change":"2024-04-26T20:58:43.048628+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.048628+0000","last_undegraded":"2024-04-26T20:58:43.048628+0000","last_fullsized":"2024-04-26T20:58:43.048628+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056479+0000","last_change":"2024-04-26T20:58:43.056479+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056479+0000","last_undegraded":"2024-04-26T20:58:43.056479+0000","last_fullsized":"2024-04-26T20:58:43.056479+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056437+0000","last_change":"2024-04-26T20:58:43.056437+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056437+0000","last_undegraded":"2024-04-26T20:58:43.056437+0000","last_fullsized":"2024-04-26T20:58:43.056437+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.048264+0000","last_change":"2024-04-26T20:58:43.048264+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.048264+0000","last_undegraded":"2024-04-26T20:58:43.048264+0000","last_fullsized":"2024-04-26T20:58:43.048264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.049255+0000","last_change":"2024-04-26T20:58:43.049255+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.049255+0000","last_undegraded":"2024-04-26T20:58:43.049255+0000","last_fullsized":"2024-04-26T20:58:43.049255+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.051206+0000","last_change":"2024-04-26T20:58:43.051206+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.051206+0000","last_undegraded":"2024-04-26T20:58:43.051206+0000","last_fullsized":"2024-04-26T20:58:43.051206+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.053178+0000","last_change":"2024-04-26T20:58:43.053178+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.053178+0000","last_undegraded":"2024-04-26T20:58:43.053178+0000","last_fullsized":"2024-04-26T20:58:43.053178+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056034+0000","last_change":"2024-04-26T20:58:43.056034+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056034+0000","last_undegraded":"2024-04-26T20:58:43.056034+0000","last_fullsized":"2024-04-26T20:58:43.056034+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.052135+0000","last_change":"2024-04-26T20:58:43.052135+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.052135+0000","last_undegraded":"2024-04-26T20:58:43.052135+0000","last_fullsized":"2024-04-26T20:58:43.052135+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.060146+0000","last_change":"2024-04-26T20:58:43.060146+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.060146+0000","last_undegraded":"2024-04-26T20:58:43.060146+0000","last_fullsized":"2024-04-26T20:58:43.060146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.038802+0000","last_change":"2024-04-26T20:58:43.038802+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.038802+0000","last_undegraded":"2024-04-26T20:58:43.038802+0000","last_fullsized":"2024-04-26T20:58:43.038802+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.051120+0000","last_change":"2024-04-26T20:58:43.051120+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.051120+0000","last_undegraded":"2024-04-26T20:58:43.051120+0000","last_fullsized":"2024-04-26T20:58:43.051120+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.049339+0000","last_change":"2024-04-26T20:58:43.049339+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.049339+0000","last_undegraded":"2024-04-26T20:58:43.049339+0000","last_fullsized":"2024-04-26T20:58:43.049339+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.061785+0000","last_change":"2024-04-26T20:58:43.061785+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.061785+0000","last_undegraded":"2024-04-26T20:58:43.061785+0000","last_fullsized":"2024-04-26T20:58:43.061785+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.049592+0000","last_change":"2024-04-26T20:58:43.049592+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.049592+0000","last_undegraded":"2024-04-26T20:58:43.049592+0000","last_fullsized":"2024-04-26T20:58:43.049592+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.051921+0000","last_change":"2024-04-26T20:58:43.051921+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.051921+0000","last_undegraded":"2024-04-26T20:58:43.051921+0000","last_fullsized":"2024-04-26T20:58:43.051921+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056057+0000","last_change":"2024-04-26T20:58:43.056057+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056057+0000","last_undegraded":"2024-04-26T20:58:43.056057+0000","last_fullsized":"2024-04-26T20:58:43.056057+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.039248+0000","last_change":"2024-04-26T20:58:43.039248+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.039248+0000","last_undegraded":"2024-04-26T20:58:43.039248+0000","last_fullsized":"2024-04-26T20:58:43.039248+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.047349+0000","last_change":"2024-04-26T20:58:43.047349+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.047349+0000","last_undegraded":"2024-04-26T20:58:43.047349+0000","last_fullsized":"2024-04-26T20:58:43.047349+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.046758+0000","last_change":"2024-04-26T20:58:43.046758+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.046758+0000","last_undegraded":"2024-04-26T20:58:43.046758+0000","last_fullsized":"2024-04-26T20:58:43.046758+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.059090+0000","last_change":"2024-04-26T20:58:43.059090+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.059090+0000","last_undegraded":"2024-04-26T20:58:43.059090+0000","last_fullsized":"2024-04-26T20:58:43.059090+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.060755+0000","last_change":"2024-04-26T20:58:43.060755+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.060755+0000","last_undegraded":"2024-04-26T20:58:43.060755+0000","last_fullsized":"2024-04-26T20:58:43.060755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.064938+0000","last_change":"2024-04-26T20:58:43.064938+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.064938+0000","last_undegraded":"2024-04-26T20:58:43.064938+0000","last_fullsized":"2024-04-26T20:58:43.064938+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.040818+0000","last_change":"2024-04-26T20:58:43.040818+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.040818+0000","last_undegraded":"2024-04-26T20:58:43.040818+0000","last_fullsized":"2024-04-26T20:58:43.040818+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.046134+0000","last_change":"2024-04-26T20:58:43.046134+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.046134+0000","last_undegraded":"2024-04-26T20:58:43.046134+0000","last_fullsized":"2024-04-26T20:58:43.046134+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.058202+0000","last_change":"2024-04-26T20:58:43.058202+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.058202+0000","last_undegraded":"2024-04-26T20:58:43.058202+0000","last_fullsized":"2024-04-26T20:58:43.058202+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056143+0000","last_change":"2024-04-26T20:58:43.056143+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056143+0000","last_undegraded":"2024-04-26T20:58:43.056143+0000","last_fullsized":"2024-04-26T20:58:43.056143+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.047071+0000","last_change":"2024-04-26T20:58:43.047071+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.047071+0000","last_undegraded":"2024-04-26T20:58:43.047071+0000","last_fullsized":"2024-04-26T20:58:43.047071+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.047975+0000","last_change":"2024-04-26T20:58:43.047975+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.047975+0000","last_undegraded":"2024-04-26T20:58:43.047975+0000","last_fullsized":"2024-04-26T20:58:43.047975+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.040702+0000","last_change":"2024-04-26T20:58:43.040702+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.040702+0000","last_undegraded":"2024-04-26T20:58:43.040702+0000","last_fullsized":"2024-04-26T20:58:43.040702+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.053161+0000","last_change":"2024-04-26T20:58:43.053161+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.053161+0000","last_undegraded":"2024-04-26T20:58:43.053161+0000","last_fullsized":"2024-04-26T20:58:43.053161+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.072931+0000","last_change":"2024-04-26T20:58:43.070204+0000","last_active":"2024-04-26T20:58:43.072931+0000","last_peered":"2024-04-26T20:58:43.072931+0000","last_clean":"2024-04-26T20:58:43.072931+0000","last_became_active":"2024-04-26T20:58:43.069791+0000","last_became_peered":"2024-04-26T20:58:43.069791+0000","last_unstale":"2024-04-26T20:58:43.072931+0000","last_undegraded":"2024-04-26T20:58:43.072931+0000","last_fullsized":"2024-04-26T20:58:43.072931+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:21.679330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.064417+0000","last_change":"2024-04-26T20:58:43.064417+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.064417+0000","last_undegraded":"2024-04-26T20:58:43.064417+0000","last_fullsized":"2024-04-26T20:58:43.064417+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.076328+0000","last_change":"2024-04-26T20:58:43.073833+0000","last_active":"2024-04-26T20:58:43.076328+0000","last_peered":"2024-04-26T20:58:43.076328+0000","last_clean":"2024-04-26T20:58:43.076328+0000","last_became_active":"2024-04-26T20:58:43.070958+0000","last_became_peered":"2024-04-26T20:58:43.070958+0000","last_unstale":"2024-04-26T20:58:43.076328+0000","last_undegraded":"2024-04-26T20:58:43.076328+0000","last_fullsized":"2024-04-26T20:58:43.076328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:23.656127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.057716+0000","last_change":"2024-04-26T20:58:43.057716+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.057716+0000","last_undegraded":"2024-04-26T20:58:43.057716+0000","last_fullsized":"2024-04-26T20:58:43.057716+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074245+0000","last_change":"2024-04-26T20:58:43.072680+0000","last_active":"2024-04-26T20:58:43.074245+0000","last_peered":"2024-04-26T20:58:43.074245+0000","last_clean":"2024-04-26T20:58:43.074245+0000","last_became_active":"2024-04-26T20:58:43.072369+0000","last_became_peered":"2024-04-26T20:58:43.072369+0000","last_unstale":"2024-04-26T20:58:43.074245+0000","last_undegraded":"2024-04-26T20:58:43.074245+0000","last_fullsized":"2024-04-26T20:58:43.074245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:30.432055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055248+0000","last_change":"2024-04-26T20:58:43.055248+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055248+0000","last_undegraded":"2024-04-26T20:58:43.055248+0000","last_fullsized":"2024-04-26T20:58:43.055248+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074948+0000","last_change":"2024-04-26T20:58:43.070599+0000","last_active":"2024-04-26T20:58:43.074948+0000","last_peered":"2024-04-26T20:58:43.074948+0000","last_clean":"2024-04-26T20:58:43.074948+0000","last_became_active":"2024-04-26T20:58:43.070312+0000","last_became_peered":"2024-04-26T20:58:43.070312+0000","last_unstale":"2024-04-26T20:58:43.074948+0000","last_undegraded":"2024-04-26T20:58:43.074948+0000","last_fullsized":"2024-04-26T20:58:43.074948+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:41.312188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.045079+0000","last_change":"2024-04-26T20:58:43.045079+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.045079+0000","last_undegraded":"2024-04-26T20:58:43.045079+0000","last_fullsized":"2024-04-26T20:58:43.045079+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.068273+0000","last_change":"2024-04-26T20:58:43.066495+0000","last_active":"2024-04-26T20:58:43.068273+0000","last_peered":"2024-04-26T20:58:43.068273+0000","last_clean":"2024-04-26T20:58:43.068273+0000","last_became_active":"2024-04-26T20:58:43.066214+0000","last_became_peered":"2024-04-26T20:58:43.066214+0000","last_unstale":"2024-04-26T20:58:43.068273+0000","last_undegraded":"2024-04-26T20:58:43.068273+0000","last_fullsized":"2024-04-26T20:58:43.068273+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:02.546662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055253+0000","last_change":"2024-04-26T20:58:43.055253+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055253+0000","last_undegraded":"2024-04-26T20:58:43.055253+0000","last_fullsized":"2024-04-26T20:58:43.055253+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.071565+0000","last_change":"2024-04-26T20:58:43.067727+0000","last_active":"2024-04-26T20:58:43.071565+0000","last_peered":"2024-04-26T20:58:43.071565+0000","last_clean":"2024-04-26T20:58:43.071565+0000","last_became_active":"2024-04-26T20:58:43.066679+0000","last_became_peered":"2024-04-26T20:58:43.066679+0000","last_unstale":"2024-04-26T20:58:43.071565+0000","last_undegraded":"2024-04-26T20:58:43.071565+0000","last_fullsized":"2024-04-26T20:58:43.071565+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:28.654953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.050736+0000","last_change":"2024-04-26T20:58:43.050736+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.050736+0000","last_undegraded":"2024-04-26T20:58:43.050736+0000","last_fullsized":"2024-04-26T20:58:43.050736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.066307+0000","last_change":"2024-04-26T20:58:43.057010+0000","last_active":"2024-04-26T20:58:43.066307+0000","last_peered":"2024-04-26T20:58:43.066307+0000","last_clean":"2024-04-26T20:58:43.066307+0000","last_became_active":"2024-04-26T20:58:43.056562+0000","last_became_peered":"2024-04-26T20:58:43.056562+0000","last_unstale":"2024-04-26T20:58:43.066307+0000","last_undegraded":"2024-04-26T20:58:43.066307+0000","last_fullsized":"2024-04-26T20:58:43.066307+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:46.360000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.059706+0000","last_change":"2024-04-26T20:58:43.059706+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.059706+0000","last_undegraded":"2024-04-26T20:58:43.059706+0000","last_fullsized":"2024-04-26T20:58:43.059706+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.068377+0000","last_change":"2024-04-26T20:58:43.066688+0000","last_active":"2024-04-26T20:58:43.068377+0000","last_peered":"2024-04-26T20:58:43.068377+0000","last_clean":"2024-04-26T20:58:43.068377+0000","last_became_active":"2024-04-26T20:58:43.066372+0000","last_became_peered":"2024-04-26T20:58:43.066372+0000","last_unstale":"2024-04-26T20:58:43.068377+0000","last_undegraded":"2024-04-26T20:58:43.068377+0000","last_fullsized":"2024-04-26T20:58:43.068377+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:04:32.800607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.050460+0000","last_change":"2024-04-26T20:58:43.050460+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.050460+0000","last_undegraded":"2024-04-26T20:58:43.050460+0000","last_fullsized":"2024-04-26T20:58:43.050460+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.064945+0000","last_change":"2024-04-26T20:58:43.061818+0000","last_active":"2024-04-26T20:58:43.064945+0000","last_peered":"2024-04-26T20:58:43.064945+0000","last_clean":"2024-04-26T20:58:43.064945+0000","last_became_active":"2024-04-26T20:58:43.059103+0000","last_became_peered":"2024-04-26T20:58:43.059103+0000","last_unstale":"2024-04-26T20:58:43.064945+0000","last_undegraded":"2024-04-26T20:58:43.064945+0000","last_fullsized":"2024-04-26T20:58:43.064945+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:31.981808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.047092+0000","last_change":"2024-04-26T20:58:43.047092+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.047092+0000","last_undegraded":"2024-04-26T20:58:43.047092+0000","last_fullsized":"2024-04-26T20:58:43.047092+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.073123+0000","last_change":"2024-04-26T20:58:43.071259+0000","last_active":"2024-04-26T20:58:43.073123+0000","last_peered":"2024-04-26T20:58:43.073123+0000","last_clean":"2024-04-26T20:58:43.073123+0000","last_became_active":"2024-04-26T20:58:43.070952+0000","last_became_peered":"2024-04-26T20:58:43.070952+0000","last_unstale":"2024-04-26T20:58:43.073123+0000","last_undegraded":"2024-04-26T20:58:43.073123+0000","last_fullsized":"2024-04-26T20:58:43.073123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:39.191097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.046401+0000","last_change":"2024-04-26T20:58:43.046401+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.046401+0000","last_undegraded":"2024-04-26T20:58:43.046401+0000","last_fullsized":"2024-04-26T20:58:43.046401+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.076213+0000","last_change":"2024-04-26T20:58:43.072767+0000","last_active":"2024-04-26T20:58:43.076213+0000","last_peered":"2024-04-26T20:58:43.076213+0000","last_clean":"2024-04-26T20:58:43.076213+0000","last_became_active":"2024-04-26T20:58:43.072408+0000","last_became_peered":"2024-04-26T20:58:43.072408+0000","last_unstale":"2024-04-26T20:58:43.076213+0000","last_undegraded":"2024-04-26T20:58:43.076213+0000","last_fullsized":"2024-04-26T20:58:43.076213+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:46.360000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.049406+0000","last_change":"2024-04-26T20:58:43.049406+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.049406+0000","last_undegraded":"2024-04-26T20:58:43.049406+0000","last_fullsized":"2024-04-26T20:58:43.049406+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.060882+0000","last_change":"2024-04-26T20:58:43.059043+0000","last_active":"2024-04-26T20:58:43.060882+0000","last_peered":"2024-04-26T20:58:43.060882+0000","last_clean":"2024-04-26T20:58:43.060882+0000","last_became_active":"2024-04-26T20:58:43.058718+0000","last_became_peered":"2024-04-26T20:58:43.058718+0000","last_unstale":"2024-04-26T20:58:43.060882+0000","last_undegraded":"2024-04-26T20:58:43.060882+0000","last_fullsized":"2024-04-26T20:58:43.060882+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:18:57.286045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055956+0000","last_change":"2024-04-26T20:58:43.055956+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055956+0000","last_undegraded":"2024-04-26T20:58:43.055956+0000","last_fullsized":"2024-04-26T20:58:43.055956+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.062506+0000","last_change":"2024-04-26T20:58:43.057592+0000","last_active":"2024-04-26T20:58:43.062506+0000","last_peered":"2024-04-26T20:58:43.062506+0000","last_clean":"2024-04-26T20:58:43.062506+0000","last_became_active":"2024-04-26T20:58:43.057285+0000","last_became_peered":"2024-04-26T20:58:43.057285+0000","last_unstale":"2024-04-26T20:58:43.062506+0000","last_undegraded":"2024-04-26T20:58:43.062506+0000","last_fullsized":"2024-04-26T20:58:43.062506+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:47.781660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.054665+0000","last_change":"2024-04-26T20:58:43.054665+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.054665+0000","last_undegraded":"2024-04-26T20:58:43.054665+0000","last_fullsized":"2024-04-26T20:58:43.054665+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074564+0000","last_change":"2024-04-26T20:58:43.068260+0000","last_active":"2024-04-26T20:58:43.074564+0000","last_peered":"2024-04-26T20:58:43.074564+0000","last_clean":"2024-04-26T20:58:43.074564+0000","last_became_active":"2024-04-26T20:58:43.067852+0000","last_became_peered":"2024-04-26T20:58:43.067852+0000","last_unstale":"2024-04-26T20:58:43.074564+0000","last_undegraded":"2024-04-26T20:58:43.074564+0000","last_fullsized":"2024-04-26T20:58:43.074564+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:15.110999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.045954+0000","last_change":"2024-04-26T20:58:43.045954+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.045954+0000","last_undegraded":"2024-04-26T20:58:43.045954+0000","last_fullsized":"2024-04-26T20:58:43.045954+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.061318+0000","last_change":"2024-04-26T20:58:43.059009+0000","last_active":"2024-04-26T20:58:43.061318+0000","last_peered":"2024-04-26T20:58:43.061318+0000","last_clean":"2024-04-26T20:58:43.061318+0000","last_became_active":"2024-04-26T20:58:43.058725+0000","last_became_peered":"2024-04-26T20:58:43.058725+0000","last_unstale":"2024-04-26T20:58:43.061318+0000","last_undegraded":"2024-04-26T20:58:43.061318+0000","last_fullsized":"2024-04-26T20:58:43.061318+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:51.552587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.048146+0000","last_change":"2024-04-26T20:58:43.048146+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.048146+0000","last_undegraded":"2024-04-26T20:58:43.048146+0000","last_fullsized":"2024-04-26T20:58:43.048146+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.072981+0000","last_change":"2024-04-26T20:58:43.071075+0000","last_active":"2024-04-26T20:58:43.072981+0000","last_peered":"2024-04-26T20:58:43.072981+0000","last_clean":"2024-04-26T20:58:43.072981+0000","last_became_active":"2024-04-26T20:58:43.070762+0000","last_became_peered":"2024-04-26T20:58:43.070762+0000","last_unstale":"2024-04-26T20:58:43.072981+0000","last_undegraded":"2024-04-26T20:58:43.072981+0000","last_fullsized":"2024-04-26T20:58:43.072981+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:03:31.981808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.039364+0000","last_change":"2024-04-26T20:58:43.039364+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.039364+0000","last_undegraded":"2024-04-26T20:58:43.039364+0000","last_fullsized":"2024-04-26T20:58:43.039364+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.053458+0000","last_change":"2024-04-26T20:58:43.048980+0000","last_active":"2024-04-26T20:58:43.053458+0000","last_peered":"2024-04-26T20:58:43.053458+0000","last_clean":"2024-04-26T20:58:43.053458+0000","last_became_active":"2024-04-26T20:58:43.048625+0000","last_became_peered":"2024-04-26T20:58:43.048625+0000","last_unstale":"2024-04-26T20:58:43.053458+0000","last_undegraded":"2024-04-26T20:58:43.053458+0000","last_fullsized":"2024-04-26T20:58:43.053458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:49:57.621037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.051926+0000","last_change":"2024-04-26T20:58:43.051926+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.051926+0000","last_undegraded":"2024-04-26T20:58:43.051926+0000","last_fullsized":"2024-04-26T20:58:43.051926+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.066737+0000","last_change":"2024-04-26T20:58:43.063230+0000","last_active":"2024-04-26T20:58:43.066737+0000","last_peered":"2024-04-26T20:58:43.066737+0000","last_clean":"2024-04-26T20:58:43.066737+0000","last_became_active":"2024-04-26T20:58:43.062868+0000","last_became_peered":"2024-04-26T20:58:43.062868+0000","last_unstale":"2024-04-26T20:58:43.066737+0000","last_undegraded":"2024-04-26T20:58:43.066737+0000","last_fullsized":"2024-04-26T20:58:43.066737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:00:39.191097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.064001+0000","last_change":"2024-04-26T20:58:43.064001+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.064001+0000","last_undegraded":"2024-04-26T20:58:43.064001+0000","last_fullsized":"2024-04-26T20:58:43.064001+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074548+0000","last_change":"2024-04-26T20:58:43.070226+0000","last_active":"2024-04-26T20:58:43.074548+0000","last_peered":"2024-04-26T20:58:43.074548+0000","last_clean":"2024-04-26T20:58:43.074548+0000","last_became_active":"2024-04-26T20:58:43.067146+0000","last_became_peered":"2024-04-26T20:58:43.067146+0000","last_unstale":"2024-04-26T20:58:43.074548+0000","last_undegraded":"2024-04-26T20:58:43.074548+0000","last_fullsized":"2024-04-26T20:58:43.074548+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:44.310082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.056614+0000","last_change":"2024-04-26T20:58:43.056614+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.056614+0000","last_undegraded":"2024-04-26T20:58:43.056614+0000","last_fullsized":"2024-04-26T20:58:43.056614+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.072821+0000","last_change":"2024-04-26T20:58:43.070073+0000","last_active":"2024-04-26T20:58:43.072821+0000","last_peered":"2024-04-26T20:58:43.072821+0000","last_clean":"2024-04-26T20:58:43.072821+0000","last_became_active":"2024-04-26T20:58:43.069554+0000","last_became_peered":"2024-04-26T20:58:43.069554+0000","last_unstale":"2024-04-26T20:58:43.072821+0000","last_undegraded":"2024-04-26T20:58:43.072821+0000","last_fullsized":"2024-04-26T20:58:43.072821+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:14.194412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.045725+0000","last_change":"2024-04-26T20:58:43.045725+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.045725+0000","last_undegraded":"2024-04-26T20:58:43.045725+0000","last_fullsized":"2024-04-26T20:58:43.045725+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.068193+0000","last_change":"2024-04-26T20:58:43.066409+0000","last_active":"2024-04-26T20:58:43.068193+0000","last_peered":"2024-04-26T20:58:43.068193+0000","last_clean":"2024-04-26T20:58:43.068193+0000","last_became_active":"2024-04-26T20:58:43.065775+0000","last_became_peered":"2024-04-26T20:58:43.065775+0000","last_unstale":"2024-04-26T20:58:43.068193+0000","last_undegraded":"2024-04-26T20:58:43.068193+0000","last_fullsized":"2024-04-26T20:58:43.068193+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:41.335687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.050764+0000","last_change":"2024-04-26T20:58:43.050764+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.050764+0000","last_undegraded":"2024-04-26T20:58:43.050764+0000","last_fullsized":"2024-04-26T20:58:43.050764+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.075472+0000","last_change":"2024-04-26T20:58:43.072643+0000","last_active":"2024-04-26T20:58:43.075472+0000","last_peered":"2024-04-26T20:58:43.075472+0000","last_clean":"2024-04-26T20:58:43.075472+0000","last_became_active":"2024-04-26T20:58:43.072358+0000","last_became_peered":"2024-04-26T20:58:43.072358+0000","last_unstale":"2024-04-26T20:58:43.075472+0000","last_undegraded":"2024-04-26T20:58:43.075472+0000","last_fullsized":"2024-04-26T20:58:43.075472+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:44.341099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.038929+0000","last_change":"2024-04-26T20:58:43.038929+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.038929+0000","last_undegraded":"2024-04-26T20:58:43.038929+0000","last_fullsized":"2024-04-26T20:58:43.038929+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.072986+0000","last_change":"2024-04-26T20:58:43.070983+0000","last_active":"2024-04-26T20:58:43.072986+0000","last_peered":"2024-04-26T20:58:43.072986+0000","last_clean":"2024-04-26T20:58:43.072986+0000","last_became_active":"2024-04-26T20:58:43.070724+0000","last_became_peered":"2024-04-26T20:58:43.070724+0000","last_unstale":"2024-04-26T20:58:43.072986+0000","last_undegraded":"2024-04-26T20:58:43.072986+0000","last_fullsized":"2024-04-26T20:58:43.072986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:36.534833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.057380+0000","last_change":"2024-04-26T20:58:43.057380+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.057380+0000","last_undegraded":"2024-04-26T20:58:43.057380+0000","last_fullsized":"2024-04-26T20:58:43.057380+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074913+0000","last_change":"2024-04-26T20:58:43.069835+0000","last_active":"2024-04-26T20:58:43.074913+0000","last_peered":"2024-04-26T20:58:43.074913+0000","last_clean":"2024-04-26T20:58:43.074913+0000","last_became_active":"2024-04-26T20:58:43.069535+0000","last_became_peered":"2024-04-26T20:58:43.069535+0000","last_unstale":"2024-04-26T20:58:43.074913+0000","last_undegraded":"2024-04-26T20:58:43.074913+0000","last_fullsized":"2024-04-26T20:58:43.074913+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:45.716150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.063901+0000","last_change":"2024-04-26T20:58:43.063901+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.063901+0000","last_undegraded":"2024-04-26T20:58:43.063901+0000","last_fullsized":"2024-04-26T20:58:43.063901+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.064585+0000","last_change":"2024-04-26T20:58:43.060904+0000","last_active":"2024-04-26T20:58:43.064585+0000","last_peered":"2024-04-26T20:58:43.064585+0000","last_clean":"2024-04-26T20:58:43.064585+0000","last_became_active":"2024-04-26T20:58:43.060511+0000","last_became_peered":"2024-04-26T20:58:43.060511+0000","last_unstale":"2024-04-26T20:58:43.064585+0000","last_undegraded":"2024-04-26T20:58:43.064585+0000","last_fullsized":"2024-04-26T20:58:43.064585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:21.497125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055227+0000","last_change":"2024-04-26T20:58:43.055227+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055227+0000","last_undegraded":"2024-04-26T20:58:43.055227+0000","last_fullsized":"2024-04-26T20:58:43.055227+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.079030+0000","last_change":"2024-04-26T20:58:43.075337+0000","last_active":"2024-04-26T20:58:43.079030+0000","last_peered":"2024-04-26T20:58:43.079030+0000","last_clean":"2024-04-26T20:58:43.079030+0000","last_became_active":"2024-04-26T20:58:43.075019+0000","last_became_peered":"2024-04-26T20:58:43.075019+0000","last_unstale":"2024-04-26T20:58:43.079030+0000","last_undegraded":"2024-04-26T20:58:43.079030+0000","last_fullsized":"2024-04-26T20:58:43.079030+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:44.310082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.047448+0000","last_change":"2024-04-26T20:58:43.047448+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.047448+0000","last_undegraded":"2024-04-26T20:58:43.047448+0000","last_fullsized":"2024-04-26T20:58:43.047448+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.074179+0000","last_change":"2024-04-26T20:58:43.071692+0000","last_active":"2024-04-26T20:58:43.074179+0000","last_peered":"2024-04-26T20:58:43.074179+0000","last_clean":"2024-04-26T20:58:43.074179+0000","last_became_active":"2024-04-26T20:58:43.069775+0000","last_became_peered":"2024-04-26T20:58:43.069775+0000","last_unstale":"2024-04-26T20:58:43.074179+0000","last_undegraded":"2024-04-26T20:58:43.074179+0000","last_fullsized":"2024-04-26T20:58:43.074179+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.491913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.045175+0000","last_change":"2024-04-26T20:58:43.045175+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.045175+0000","last_undegraded":"2024-04-26T20:58:43.045175+0000","last_fullsized":"2024-04-26T20:58:43.045175+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.079032+0000","last_change":"2024-04-26T20:58:43.075348+0000","last_active":"2024-04-26T20:58:43.079032+0000","last_peered":"2024-04-26T20:58:43.079032+0000","last_clean":"2024-04-26T20:58:43.079032+0000","last_became_active":"2024-04-26T20:58:43.075012+0000","last_became_peered":"2024-04-26T20:58:43.075012+0000","last_unstale":"2024-04-26T20:58:43.079032+0000","last_undegraded":"2024-04-26T20:58:43.079032+0000","last_fullsized":"2024-04-26T20:58:43.079032+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:44.341099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.045208+0000","last_change":"2024-04-26T20:58:43.045208+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.045208+0000","last_undegraded":"2024-04-26T20:58:43.045208+0000","last_fullsized":"2024-04-26T20:58:43.045208+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.064494+0000","last_change":"2024-04-26T20:58:43.054765+0000","last_active":"2024-04-26T20:58:43.064494+0000","last_peered":"2024-04-26T20:58:43.064494+0000","last_clean":"2024-04-26T20:58:43.064494+0000","last_became_active":"2024-04-26T20:58:43.054455+0000","last_became_peered":"2024-04-26T20:58:43.054455+0000","last_unstale":"2024-04-26T20:58:43.064494+0000","last_undegraded":"2024-04-26T20:58:43.064494+0000","last_fullsized":"2024-04-26T20:58:43.064494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:36.534833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.062096+0000","last_change":"2024-04-26T20:58:43.062096+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.062096+0000","last_undegraded":"2024-04-26T20:58:43.062096+0000","last_fullsized":"2024-04-26T20:58:43.062096+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.067885+0000","last_change":"2024-04-26T20:58:43.065790+0000","last_active":"2024-04-26T20:58:43.067885+0000","last_peered":"2024-04-26T20:58:43.067885+0000","last_clean":"2024-04-26T20:58:43.067885+0000","last_became_active":"2024-04-26T20:58:43.065469+0000","last_became_peered":"2024-04-26T20:58:43.065469+0000","last_unstale":"2024-04-26T20:58:43.067885+0000","last_undegraded":"2024-04-26T20:58:43.067885+0000","last_fullsized":"2024-04-26T20:58:43.067885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:28.519261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.044116+0000","last_change":"2024-04-26T20:58:43.044116+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.044116+0000","last_undegraded":"2024-04-26T20:58:43.044116+0000","last_fullsized":"2024-04-26T20:58:43.044116+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.059577+0000","last_change":"2024-04-26T20:58:43.056930+0000","last_active":"2024-04-26T20:58:43.059577+0000","last_peered":"2024-04-26T20:58:43.059577+0000","last_clean":"2024-04-26T20:58:43.059577+0000","last_became_active":"2024-04-26T20:58:43.055764+0000","last_became_peered":"2024-04-26T20:58:43.055764+0000","last_unstale":"2024-04-26T20:58:43.059577+0000","last_undegraded":"2024-04-26T20:58:43.059577+0000","last_fullsized":"2024-04-26T20:58:43.059577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:14.346315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":43,"state":"creating+peering","last_fresh":"2024-04-26T20:58:43.055239+0000","last_change":"2024-04-26T20:58:43.055239+0000","last_active":"2024-04-26T20:58:43.016772+0000","last_peered":"2024-04-26T20:58:43.016772+0000","last_clean":"2024-04-26T20:58:43.016772+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T20:58:43.055239+0000","last_undegraded":"2024-04-26T20:58:43.055239+0000","last_fullsized":"2024-04-26T20:58:43.055239+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.076095+0000","last_change":"2024-04-26T20:58:43.074219+0000","last_active":"2024-04-26T20:58:43.076095+0000","last_peered":"2024-04-26T20:58:43.076095+0000","last_clean":"2024-04-26T20:58:43.076095+0000","last_became_active":"2024-04-26T20:58:43.072029+0000","last_became_peered":"2024-04-26T20:58:43.072029+0000","last_unstale":"2024-04-26T20:58:43.076095+0000","last_undegraded":"2024-04-26T20:58:43.076095+0000","last_fullsized":"2024-04-26T20:58:43.076095+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:58.207157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":211,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-04-26T20:58:43.058660+0000","last_change":"2024-04-26T20:56:46.848603+0000","last_active":"2024-04-26T20:58:43.058660+0000","last_peered":"2024-04-26T20:58:43.058660+0000","last_clean":"2024-04-26T20:58:43.058660+0000","last_became_active":"2024-04-26T20:56:46.846091+0000","last_became_peered":"2024-04-26T20:56:46.846091+0000","last_unstale":"2024-04-26T20:58:43.058660+0000","last_undegraded":"2024-04-26T20:58:43.058660+0000","last_fullsized":"2024-04-26T20:58:43.058660+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_clean_scrub_stamp":"2024-04-26T20:56:45.594041+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:00.232387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":785505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":201404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":491520,"data_stored":195137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":201404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":201404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":503808,"data_stored":201404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":791772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:58:53.610 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T20:58:53.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.827+0000 7f78aba00640 1 Processor -- start 2024-04-26T20:58:53.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.827+0000 7f78aba00640 1 -- start start 2024-04-26T20:58:53.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.829+0000 7f78aba00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f78a40b6810 0x7f78a415d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:53.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.829+0000 7f78aba00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f78a40740e0 0x7f78a415daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:53.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.829+0000 7f78aba00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78a40d0a30 con 0x7f78a4146980 2024-04-26T20:58:53.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.829+0000 7f78aba00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f78a415e510 con 0x7f78a40740e0 2024-04-26T20:58:53.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.829+0000 7f78aba00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f78a415e230 con 0x7f78a40b6810 2024-04-26T20:58:53.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.830+0000 7f78a3e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f78a40b6810 0x7f78a415d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:53.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.830+0000 7f78a9000640 1 --1- >> v1:172.21.15.103:6789/0 conn(0x7f78a4146980 0x7f78a40b6100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.103:6789/0 says I am v1:172.21.15.28:33276/0 (socket says 172.21.15.28:33276) 2024-04-26T20:58:53.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.830+0000 7f78a3e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f78a40b6810 0x7f78a415d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.28:58276/0 (socket says 172.21.15.28:58276) 2024-04-26T20:58:53.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.830+0000 7f78a9000640 1 -- 172.21.15.28:0/3055545249 learned_addr learned my addr 172.21.15.28:0/3055545249 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:53.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.830+0000 7f78a9a00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f78a40740e0 0x7f78a415daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:53.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.831+0000 7f78a9a00640 1 -- 172.21.15.28:0/3055545249 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f78a40b6810 msgr2=0x7f78a415d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:53.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.831+0000 7f78a9a00640 1 --2- 172.21.15.28:0/3055545249 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f78a40b6810 0x7f78a415d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:53.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.831+0000 7f78a9a00640 1 -- 172.21.15.28:0/3055545249 >> v1:172.21.15.103:6789/0 conn(0x7f78a4146980 legacy=0x7f78a40b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:53.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.831+0000 7f78a9a00640 1 -- 172.21.15.28:0/3055545249 --> v2:172.21.15.103:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a4073600 con 0x7f78a40740e0 2024-04-26T20:58:53.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.832+0000 7f78a9a00640 1 --2- 172.21.15.28:0/3055545249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78a40740e0 0x7f78a415daf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f789c009d90 tx=0x7f789c02a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2238a5b0e164967 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:53.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.833+0000 7f78a3400640 1 -- 172.21.15.28:0/3055545249 <== mon.1 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c031600 con 0x7f78a40740e0 2024-04-26T20:58:53.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.833+0000 7f78a3400640 1 -- 172.21.15.28:0/3055545249 <== mon.1 v2:172.21.15.103:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f789c0317a0 con 0x7f78a40740e0 2024-04-26T20:58:53.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.833+0000 7f78a3400640 1 -- 172.21.15.28:0/3055545249 <== mon.1 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f789c031b70 con 0x7f78a40740e0 2024-04-26T20:58:53.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.833+0000 7f78aba00640 1 -- 172.21.15.28:0/3055545249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78a40740e0 msgr2=0x7f78a415daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:53.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.833+0000 7f78aba00640 1 --2- 172.21.15.28:0/3055545249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78a40740e0 0x7f78a415daf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f789c009d90 tx=0x7f789c02a5d0 comp rx=0 tx=0).stop 2024-04-26T20:58:53.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.834+0000 7f78aba00640 1 -- 172.21.15.28:0/3055545249 shutdown_connections 2024-04-26T20:58:53.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.834+0000 7f78aba00640 1 --2- 172.21.15.28:0/3055545249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78a40740e0 0x7f78a415daf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:53.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.834+0000 7f78aba00640 1 --2- 172.21.15.28:0/3055545249 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f78a40b6810 0x7f78a415d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:53.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.834+0000 7f78aba00640 1 -- 172.21.15.28:0/3055545249 >> 172.21.15.28:0/3055545249 conn(0x7f78a406e220 msgr2=0x7f78a406e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:53.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.834+0000 7f78aba00640 1 -- 172.21.15.28:0/3055545249 shutdown_connections 2024-04-26T20:58:53.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.834+0000 7f78aba00640 1 -- 172.21.15.28:0/3055545249 wait complete. 2024-04-26T20:58:53.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.836+0000 7f78aba00640 1 Processor -- start 2024-04-26T20:58:53.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.836+0000 7f78aba00640 1 -- start start 2024-04-26T20:58:53.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.837+0000 7f78aba00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78a40740e0 0x7f78a41396b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:53.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.837+0000 7f78aba00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f78a40b6810 0x7f78a4139ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:53.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.837+0000 7f78aba00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78a4146980 0x7f78a413b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:53.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.837+0000 7f78aba00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f78a4071ff0 con 0x7f78a40740e0 2024-04-26T20:58:53.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.837+0000 7f78aba00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78a4073bc0 con 0x7f78a4146980 2024-04-26T20:58:53.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.837+0000 7f78aba00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f78a4072170 con 0x7f78a40b6810 2024-04-26T20:58:53.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.837+0000 7f78a9000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78a40740e0 0x7f78a41396b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:53.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.837+0000 7f78a9000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78a40740e0 0x7f78a41396b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47064/0 (socket says 172.21.15.28:47064) 2024-04-26T20:58:53.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.837+0000 7f78a9000640 1 -- 172.21.15.28:0/183249222 learned_addr learned my addr 172.21.15.28:0/183249222 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:53.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.838+0000 7f78a9000640 1 -- 172.21.15.28:0/183249222 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f78a40b6810 msgr2=0x7f78a4139ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:53.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.838+0000 7f78a9000640 1 --2- 172.21.15.28:0/183249222 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f78a40b6810 0x7f78a4139ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:53.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.838+0000 7f78a9000640 1 -- 172.21.15.28:0/183249222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78a4146980 msgr2=0x7f78a413b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:53.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.838+0000 7f78a9000640 1 --2- 172.21.15.28:0/183249222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78a4146980 0x7f78a413b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:53.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.838+0000 7f78a9000640 1 -- 172.21.15.28:0/183249222 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a413a740 con 0x7f78a40740e0 2024-04-26T20:58:53.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.838+0000 7f78a9000640 1 --2- 172.21.15.28:0/183249222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78a40740e0 0x7f78a41396b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f789800c3c0 tx=0x7f789800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:53.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.838+0000 7f78a0c00640 1 -- 172.21.15.28:0/183249222 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7898017070 con 0x7f78a40740e0 2024-04-26T20:58:53.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.839+0000 7f78a0c00640 1 -- 172.21.15.28:0/183249222 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7898005ce0 con 0x7f78a40740e0 2024-04-26T20:58:53.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.839+0000 7f78aba00640 1 -- 172.21.15.28:0/183249222 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a413aa20 con 0x7f78a40740e0 2024-04-26T20:58:53.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.839+0000 7f78a0c00640 1 -- 172.21.15.28:0/183249222 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78980072f0 con 0x7f78a40740e0 2024-04-26T20:58:53.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.839+0000 7f78aba00640 1 -- 172.21.15.28:0/183249222 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a413afe0 con 0x7f78a40740e0 2024-04-26T20:58:53.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.840+0000 7f78aba00640 1 -- 172.21.15.28:0/183249222 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a406a400 con 0x7f78a40740e0 2024-04-26T20:58:53.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.841+0000 7f78a0c00640 1 -- 172.21.15.28:0/183249222 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f789801a070 con 0x7f78a40740e0 2024-04-26T20:58:53.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.842+0000 7f78a0c00640 1 --2- 172.21.15.28:0/183249222 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f788406c8f0 0x7f788406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:53.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.842+0000 7f78a0c00640 1 -- 172.21.15.28:0/183249222 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f789808af30 con 0x7f78a40740e0 2024-04-26T20:58:53.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.842+0000 7f78a3e00640 1 --2- 172.21.15.28:0/183249222 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f788406c8f0 0x7f788406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:53.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.843+0000 7f78a3e00640 1 --2- 172.21.15.28:0/183249222 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f788406c8f0 0x7f788406edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7890006a90 tx=0x7f7890006a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:53.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:53.847+0000 7f78a0c00640 1 -- 172.21.15.28:0/183249222 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f78a40d0a30 con 0x7f78a40740e0 2024-04-26T20:58:54.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.046+0000 7f78aba00640 1 -- 172.21.15.28:0/183249222 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f78a406ff00 con 0x7f788406c8f0 2024-04-26T20:58:54.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.063+0000 7f78a0c00640 1 -- 172.21.15.28:0/183249222 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251960 (secure 0 0 0) 0x7f78a406ff00 con 0x7f788406c8f0 2024-04-26T20:58:54.068 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:54.069 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T20:58:54.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.069+0000 7f78aba00640 1 -- 172.21.15.28:0/183249222 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f788406c8f0 msgr2=0x7f788406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:54.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.069+0000 7f78aba00640 1 --2- 172.21.15.28:0/183249222 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f788406c8f0 0x7f788406edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7890006a90 tx=0x7f7890006a20 comp rx=0 tx=0).stop 2024-04-26T20:58:54.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.069+0000 7f78aba00640 1 -- 172.21.15.28:0/183249222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78a40740e0 msgr2=0x7f78a41396b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:54.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.069+0000 7f78aba00640 1 --2- 172.21.15.28:0/183249222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78a40740e0 0x7f78a41396b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f789800c3c0 tx=0x7f789800c880 comp rx=0 tx=0).stop 2024-04-26T20:58:54.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.069+0000 7f78aba00640 1 -- 172.21.15.28:0/183249222 shutdown_connections 2024-04-26T20:58:54.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.069+0000 7f78aba00640 1 --2- 172.21.15.28:0/183249222 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f788406c8f0 0x7f788406edb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.069+0000 7f78aba00640 1 --2- 172.21.15.28:0/183249222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78a4146980 0x7f78a413b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.069+0000 7f78aba00640 1 --2- 172.21.15.28:0/183249222 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f78a40b6810 0x7f78a4139ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.069+0000 7f78aba00640 1 --2- 172.21.15.28:0/183249222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78a40740e0 0x7f78a41396b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.069+0000 7f78aba00640 1 -- 172.21.15.28:0/183249222 >> 172.21.15.28:0/183249222 conn(0x7f78a406e220 msgr2=0x7f78a4142a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:54.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.069+0000 7f78aba00640 1 -- 172.21.15.28:0/183249222 shutdown_connections 2024-04-26T20:58:54.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.070+0000 7f78aba00640 1 -- 172.21.15.28:0/183249222 wait complete. 2024-04-26T20:58:54.090 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-04-26T20:58:52.743757+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242348,"kb_used_data":5352,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732372,"statfs":{"total":773094113280,"available":772845948928,"internally_reserved":0,"allocated":5480448,"data_stored":2901388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.706082"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.363241+0000","last_change":"2024-04-26T20:58:44.068694+0000","last_active":"2024-04-26T20:58:45.363241+0000","last_peered":"2024-04-26T20:58:45.363241+0000","last_clean":"2024-04-26T20:58:45.363241+0000","last_became_active":"2024-04-26T20:58:44.068361+0000","last_became_peered":"2024-04-26T20:58:44.068361+0000","last_unstale":"2024-04-26T20:58:45.363241+0000","last_undegraded":"2024-04-26T20:58:45.363241+0000","last_fullsized":"2024-04-26T20:58:45.363241+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:44.521862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.361645+0000","last_change":"2024-04-26T20:58:44.093788+0000","last_active":"2024-04-26T20:58:45.361645+0000","last_peered":"2024-04-26T20:58:45.361645+0000","last_clean":"2024-04-26T20:58:45.361645+0000","last_became_active":"2024-04-26T20:58:44.093449+0000","last_became_peered":"2024-04-26T20:58:44.093449+0000","last_unstale":"2024-04-26T20:58:45.361645+0000","last_undegraded":"2024-04-26T20:58:45.361645+0000","last_fullsized":"2024-04-26T20:58:45.361645+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:38.450510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.361759+0000","last_change":"2024-04-26T20:58:44.081883+0000","last_active":"2024-04-26T20:58:45.361759+0000","last_peered":"2024-04-26T20:58:45.361759+0000","last_clean":"2024-04-26T20:58:45.361759+0000","last_became_active":"2024-04-26T20:58:44.081035+0000","last_became_peered":"2024-04-26T20:58:44.081035+0000","last_unstale":"2024-04-26T20:58:45.361759+0000","last_undegraded":"2024-04-26T20:58:45.361759+0000","last_fullsized":"2024-04-26T20:58:45.361759+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:52.567410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.360989+0000","last_change":"2024-04-26T20:58:44.086391+0000","last_active":"2024-04-26T20:58:45.360989+0000","last_peered":"2024-04-26T20:58:45.360989+0000","last_clean":"2024-04-26T20:58:45.360989+0000","last_became_active":"2024-04-26T20:58:44.081795+0000","last_became_peered":"2024-04-26T20:58:44.081795+0000","last_unstale":"2024-04-26T20:58:45.360989+0000","last_undegraded":"2024-04-26T20:58:45.360989+0000","last_fullsized":"2024-04-26T20:58:45.360989+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:41.261363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.361581+0000","last_change":"2024-04-26T20:58:44.087142+0000","last_active":"2024-04-26T20:58:45.361581+0000","last_peered":"2024-04-26T20:58:45.361581+0000","last_clean":"2024-04-26T20:58:45.361581+0000","last_became_active":"2024-04-26T20:58:44.086828+0000","last_became_peered":"2024-04-26T20:58:44.086828+0000","last_unstale":"2024-04-26T20:58:45.361581+0000","last_undegraded":"2024-04-26T20:58:45.361581+0000","last_fullsized":"2024-04-26T20:58:45.361581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:45.018355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.063687+0000","last_change":"2024-04-26T20:58:44.060279+0000","last_active":"2024-04-26T20:58:45.063687+0000","last_peered":"2024-04-26T20:58:45.063687+0000","last_clean":"2024-04-26T20:58:45.063687+0000","last_became_active":"2024-04-26T20:58:44.058008+0000","last_became_peered":"2024-04-26T20:58:44.058008+0000","last_unstale":"2024-04-26T20:58:45.063687+0000","last_undegraded":"2024-04-26T20:58:45.063687+0000","last_fullsized":"2024-04-26T20:58:45.063687+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:24.773022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.359988+0000","last_change":"2024-04-26T20:58:44.064928+0000","last_active":"2024-04-26T20:58:45.359988+0000","last_peered":"2024-04-26T20:58:45.359988+0000","last_clean":"2024-04-26T20:58:45.359988+0000","last_became_active":"2024-04-26T20:58:44.064069+0000","last_became_peered":"2024-04-26T20:58:44.064069+0000","last_unstale":"2024-04-26T20:58:45.359988+0000","last_undegraded":"2024-04-26T20:58:45.359988+0000","last_fullsized":"2024-04-26T20:58:45.359988+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:57.868880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.360056+0000","last_change":"2024-04-26T20:58:44.074686+0000","last_active":"2024-04-26T20:58:45.360056+0000","last_peered":"2024-04-26T20:58:45.360056+0000","last_clean":"2024-04-26T20:58:45.360056+0000","last_became_active":"2024-04-26T20:58:44.074287+0000","last_became_peered":"2024-04-26T20:58:44.074287+0000","last_unstale":"2024-04-26T20:58:45.360056+0000","last_undegraded":"2024-04-26T20:58:45.360056+0000","last_fullsized":"2024-04-26T20:58:45.360056+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:03.561484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.361654+0000","last_change":"2024-04-26T20:58:44.086920+0000","last_active":"2024-04-26T20:58:45.361654+0000","last_peered":"2024-04-26T20:58:45.361654+0000","last_clean":"2024-04-26T20:58:45.361654+0000","last_became_active":"2024-04-26T20:58:44.085458+0000","last_became_peered":"2024-04-26T20:58:44.085458+0000","last_unstale":"2024-04-26T20:58:45.361654+0000","last_undegraded":"2024-04-26T20:58:45.361654+0000","last_fullsized":"2024-04-26T20:58:45.361654+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:26.672532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.361466+0000","last_change":"2024-04-26T20:58:44.085502+0000","last_active":"2024-04-26T20:58:45.361466+0000","last_peered":"2024-04-26T20:58:45.361466+0000","last_clean":"2024-04-26T20:58:45.361466+0000","last_became_active":"2024-04-26T20:58:44.073082+0000","last_became_peered":"2024-04-26T20:58:44.073082+0000","last_unstale":"2024-04-26T20:58:45.361466+0000","last_undegraded":"2024-04-26T20:58:45.361466+0000","last_fullsized":"2024-04-26T20:58:45.361466+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:38.450510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.360772+0000","last_change":"2024-04-26T20:58:44.082299+0000","last_active":"2024-04-26T20:58:45.360772+0000","last_peered":"2024-04-26T20:58:45.360772+0000","last_clean":"2024-04-26T20:58:45.360772+0000","last_became_active":"2024-04-26T20:58:44.082024+0000","last_became_peered":"2024-04-26T20:58:44.082024+0000","last_unstale":"2024-04-26T20:58:45.360772+0000","last_undegraded":"2024-04-26T20:58:45.360772+0000","last_fullsized":"2024-04-26T20:58:45.360772+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:30:03.335997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.360898+0000","last_change":"2024-04-26T20:58:44.084528+0000","last_active":"2024-04-26T20:58:45.360898+0000","last_peered":"2024-04-26T20:58:45.360898+0000","last_clean":"2024-04-26T20:58:45.360898+0000","last_became_active":"2024-04-26T20:58:44.084194+0000","last_became_peered":"2024-04-26T20:58:44.084194+0000","last_unstale":"2024-04-26T20:58:45.360898+0000","last_undegraded":"2024-04-26T20:58:45.360898+0000","last_fullsized":"2024-04-26T20:58:45.360898+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:41.261363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.065787+0000","last_change":"2024-04-26T20:58:44.082685+0000","last_active":"2024-04-26T20:58:45.065787+0000","last_peered":"2024-04-26T20:58:45.065787+0000","last_clean":"2024-04-26T20:58:45.065787+0000","last_became_active":"2024-04-26T20:58:44.082401+0000","last_became_peered":"2024-04-26T20:58:44.082401+0000","last_unstale":"2024-04-26T20:58:45.065787+0000","last_undegraded":"2024-04-26T20:58:45.065787+0000","last_fullsized":"2024-04-26T20:58:45.065787+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:31:35.714040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.360290+0000","last_change":"2024-04-26T20:58:44.081341+0000","last_active":"2024-04-26T20:58:45.360290+0000","last_peered":"2024-04-26T20:58:45.360290+0000","last_clean":"2024-04-26T20:58:45.360290+0000","last_became_active":"2024-04-26T20:58:44.081029+0000","last_became_peered":"2024-04-26T20:58:44.081029+0000","last_unstale":"2024-04-26T20:58:45.360290+0000","last_undegraded":"2024-04-26T20:58:45.360290+0000","last_fullsized":"2024-04-26T20:58:45.360290+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:15:22.694153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.063798+0000","last_change":"2024-04-26T20:58:44.061695+0000","last_active":"2024-04-26T20:58:45.063798+0000","last_peered":"2024-04-26T20:58:45.063798+0000","last_clean":"2024-04-26T20:58:45.063798+0000","last_became_active":"2024-04-26T20:58:44.061374+0000","last_became_peered":"2024-04-26T20:58:44.061374+0000","last_unstale":"2024-04-26T20:58:45.063798+0000","last_undegraded":"2024-04-26T20:58:45.063798+0000","last_fullsized":"2024-04-26T20:58:45.063798+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:31:27.081174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.067397+0000","last_change":"2024-04-26T20:58:44.087039+0000","last_active":"2024-04-26T20:58:45.067397+0000","last_peered":"2024-04-26T20:58:45.067397+0000","last_clean":"2024-04-26T20:58:45.067397+0000","last_became_active":"2024-04-26T20:58:44.086703+0000","last_became_peered":"2024-04-26T20:58:44.086703+0000","last_unstale":"2024-04-26T20:58:45.067397+0000","last_undegraded":"2024-04-26T20:58:45.067397+0000","last_fullsized":"2024-04-26T20:58:45.067397+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:35:31.446877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.363596+0000","last_change":"2024-04-26T20:58:44.079313+0000","last_active":"2024-04-26T20:58:45.363596+0000","last_peered":"2024-04-26T20:58:45.363596+0000","last_clean":"2024-04-26T20:58:45.363596+0000","last_became_active":"2024-04-26T20:58:44.079040+0000","last_became_peered":"2024-04-26T20:58:44.079040+0000","last_unstale":"2024-04-26T20:58:45.363596+0000","last_undegraded":"2024-04-26T20:58:45.363596+0000","last_fullsized":"2024-04-26T20:58:45.363596+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:15.209235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.361477+0000","last_change":"2024-04-26T20:58:44.087625+0000","last_active":"2024-04-26T20:58:45.361477+0000","last_peered":"2024-04-26T20:58:45.361477+0000","last_clean":"2024-04-26T20:58:45.361477+0000","last_became_active":"2024-04-26T20:58:44.086805+0000","last_became_peered":"2024-04-26T20:58:44.086805+0000","last_unstale":"2024-04-26T20:58:45.361477+0000","last_undegraded":"2024-04-26T20:58:45.361477+0000","last_fullsized":"2024-04-26T20:58:45.361477+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:48.796482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.069423+0000","last_change":"2024-04-26T20:58:44.093422+0000","last_active":"2024-04-26T20:58:45.069423+0000","last_peered":"2024-04-26T20:58:45.069423+0000","last_clean":"2024-04-26T20:58:45.069423+0000","last_became_active":"2024-04-26T20:58:44.093085+0000","last_became_peered":"2024-04-26T20:58:44.093085+0000","last_unstale":"2024-04-26T20:58:45.069423+0000","last_undegraded":"2024-04-26T20:58:45.069423+0000","last_fullsized":"2024-04-26T20:58:45.069423+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:48.796482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.065929+0000","last_change":"2024-04-26T20:58:44.081649+0000","last_active":"2024-04-26T20:58:45.065929+0000","last_peered":"2024-04-26T20:58:45.065929+0000","last_clean":"2024-04-26T20:58:45.065929+0000","last_became_active":"2024-04-26T20:58:44.081336+0000","last_became_peered":"2024-04-26T20:58:44.081336+0000","last_unstale":"2024-04-26T20:58:45.065929+0000","last_undegraded":"2024-04-26T20:58:45.065929+0000","last_fullsized":"2024-04-26T20:58:45.065929+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:23.360416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.064647+0000","last_change":"2024-04-26T20:58:44.075626+0000","last_active":"2024-04-26T20:58:45.064647+0000","last_peered":"2024-04-26T20:58:45.064647+0000","last_clean":"2024-04-26T20:58:45.064647+0000","last_became_active":"2024-04-26T20:58:44.069752+0000","last_became_peered":"2024-04-26T20:58:44.069752+0000","last_unstale":"2024-04-26T20:58:45.064647+0000","last_undegraded":"2024-04-26T20:58:45.064647+0000","last_fullsized":"2024-04-26T20:58:45.064647+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:17:37.433985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.360088+0000","last_change":"2024-04-26T20:58:44.081251+0000","last_active":"2024-04-26T20:58:45.360088+0000","last_peered":"2024-04-26T20:58:45.360088+0000","last_clean":"2024-04-26T20:58:45.360088+0000","last_became_active":"2024-04-26T20:58:44.080888+0000","last_became_peered":"2024-04-26T20:58:44.080888+0000","last_unstale":"2024-04-26T20:58:45.360088+0000","last_undegraded":"2024-04-26T20:58:45.360088+0000","last_fullsized":"2024-04-26T20:58:45.360088+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:42.350509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.068052+0000","last_change":"2024-04-26T20:58:44.077531+0000","last_active":"2024-04-26T20:58:45.068052+0000","last_peered":"2024-04-26T20:58:45.068052+0000","last_clean":"2024-04-26T20:58:45.068052+0000","last_became_active":"2024-04-26T20:58:44.075960+0000","last_became_peered":"2024-04-26T20:58:44.075960+0000","last_unstale":"2024-04-26T20:58:45.068052+0000","last_undegraded":"2024-04-26T20:58:45.068052+0000","last_fullsized":"2024-04-26T20:58:45.068052+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:42.327010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.067856+0000","last_change":"2024-04-26T20:58:44.071558+0000","last_active":"2024-04-26T20:58:45.067856+0000","last_peered":"2024-04-26T20:58:45.067856+0000","last_clean":"2024-04-26T20:58:45.067856+0000","last_became_active":"2024-04-26T20:58:44.071263+0000","last_became_peered":"2024-04-26T20:58:44.071263+0000","last_unstale":"2024-04-26T20:58:45.067856+0000","last_undegraded":"2024-04-26T20:58:45.067856+0000","last_fullsized":"2024-04-26T20:58:45.067856+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:46.730972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.069072+0000","last_change":"2024-04-26T20:58:44.073208+0000","last_active":"2024-04-26T20:58:45.069072+0000","last_peered":"2024-04-26T20:58:45.069072+0000","last_clean":"2024-04-26T20:58:45.069072+0000","last_became_active":"2024-04-26T20:58:44.072894+0000","last_became_peered":"2024-04-26T20:58:44.072894+0000","last_unstale":"2024-04-26T20:58:45.069072+0000","last_undegraded":"2024-04-26T20:58:45.069072+0000","last_fullsized":"2024-04-26T20:58:45.069072+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:59.221980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.065956+0000","last_change":"2024-04-26T20:58:44.066117+0000","last_active":"2024-04-26T20:58:45.065956+0000","last_peered":"2024-04-26T20:58:45.065956+0000","last_clean":"2024-04-26T20:58:45.065956+0000","last_became_active":"2024-04-26T20:58:44.065765+0000","last_became_peered":"2024-04-26T20:58:44.065765+0000","last_unstale":"2024-04-26T20:58:45.065956+0000","last_undegraded":"2024-04-26T20:58:45.065956+0000","last_fullsized":"2024-04-26T20:58:45.065956+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:29.917994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.362209+0000","last_change":"2024-04-26T20:58:44.075054+0000","last_active":"2024-04-26T20:58:45.362209+0000","last_peered":"2024-04-26T20:58:45.362209+0000","last_clean":"2024-04-26T20:58:45.362209+0000","last_became_active":"2024-04-26T20:58:44.074719+0000","last_became_peered":"2024-04-26T20:58:44.074719+0000","last_unstale":"2024-04-26T20:58:45.362209+0000","last_undegraded":"2024-04-26T20:58:45.362209+0000","last_fullsized":"2024-04-26T20:58:45.362209+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:42.350509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.361016+0000","last_change":"2024-04-26T20:58:44.083318+0000","last_active":"2024-04-26T20:58:45.361016+0000","last_peered":"2024-04-26T20:58:45.361016+0000","last_clean":"2024-04-26T20:58:45.361016+0000","last_became_active":"2024-04-26T20:58:44.082997+0000","last_became_peered":"2024-04-26T20:58:44.082997+0000","last_unstale":"2024-04-26T20:58:45.361016+0000","last_undegraded":"2024-04-26T20:58:45.361016+0000","last_fullsized":"2024-04-26T20:58:45.361016+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:57.868880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.067625+0000","last_change":"2024-04-26T20:58:44.080649+0000","last_active":"2024-04-26T20:58:45.067625+0000","last_peered":"2024-04-26T20:58:45.067625+0000","last_clean":"2024-04-26T20:58:45.067625+0000","last_became_active":"2024-04-26T20:58:44.080172+0000","last_became_peered":"2024-04-26T20:58:44.080172+0000","last_unstale":"2024-04-26T20:58:45.067625+0000","last_undegraded":"2024-04-26T20:58:45.067625+0000","last_fullsized":"2024-04-26T20:58:45.067625+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:16.125822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.361271+0000","last_change":"2024-04-26T20:58:44.056875+0000","last_active":"2024-04-26T20:58:45.361271+0000","last_peered":"2024-04-26T20:58:45.361271+0000","last_clean":"2024-04-26T20:58:45.361271+0000","last_became_active":"2024-04-26T20:58:44.056468+0000","last_became_peered":"2024-04-26T20:58:44.056468+0000","last_unstale":"2024-04-26T20:58:45.361271+0000","last_undegraded":"2024-04-26T20:58:45.361271+0000","last_fullsized":"2024-04-26T20:58:45.361271+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:20:59.906583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.063799+0000","last_change":"2024-04-26T20:58:44.078026+0000","last_active":"2024-04-26T20:58:45.063799+0000","last_peered":"2024-04-26T20:58:45.063799+0000","last_clean":"2024-04-26T20:58:45.063799+0000","last_became_active":"2024-04-26T20:58:44.077676+0000","last_became_peered":"2024-04-26T20:58:44.077676+0000","last_unstale":"2024-04-26T20:58:45.063799+0000","last_undegraded":"2024-04-26T20:58:45.063799+0000","last_fullsized":"2024-04-26T20:58:45.063799+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:27:31.654296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.065179+0000","last_change":"2024-04-26T20:58:44.067857+0000","last_active":"2024-04-26T20:58:45.065179+0000","last_peered":"2024-04-26T20:58:45.065179+0000","last_clean":"2024-04-26T20:58:45.065179+0000","last_became_active":"2024-04-26T20:58:44.067582+0000","last_became_peered":"2024-04-26T20:58:44.067582+0000","last_unstale":"2024-04-26T20:58:45.065179+0000","last_undegraded":"2024-04-26T20:58:45.065179+0000","last_fullsized":"2024-04-26T20:58:45.065179+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:04:36.570799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.359182+0000","last_change":"2024-04-26T20:58:44.075484+0000","last_active":"2024-04-26T20:58:45.359182+0000","last_peered":"2024-04-26T20:58:45.359182+0000","last_clean":"2024-04-26T20:58:45.359182+0000","last_became_active":"2024-04-26T20:58:44.073173+0000","last_became_peered":"2024-04-26T20:58:44.073173+0000","last_unstale":"2024-04-26T20:58:45.359182+0000","last_undegraded":"2024-04-26T20:58:45.359182+0000","last_fullsized":"2024-04-26T20:58:45.359182+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:59.221980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.064996+0000","last_change":"2024-04-26T20:58:43.070204+0000","last_active":"2024-04-26T20:58:45.064996+0000","last_peered":"2024-04-26T20:58:45.064996+0000","last_clean":"2024-04-26T20:58:45.064996+0000","last_became_active":"2024-04-26T20:58:43.069791+0000","last_became_peered":"2024-04-26T20:58:43.069791+0000","last_unstale":"2024-04-26T20:58:45.064996+0000","last_undegraded":"2024-04-26T20:58:45.064996+0000","last_fullsized":"2024-04-26T20:58:45.064996+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:56.854057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.069343+0000","last_change":"2024-04-26T20:58:44.072567+0000","last_active":"2024-04-26T20:58:45.069343+0000","last_peered":"2024-04-26T20:58:45.069343+0000","last_clean":"2024-04-26T20:58:45.069343+0000","last_became_active":"2024-04-26T20:58:44.072269+0000","last_became_peered":"2024-04-26T20:58:44.072269+0000","last_unstale":"2024-04-26T20:58:45.069343+0000","last_undegraded":"2024-04-26T20:58:45.069343+0000","last_fullsized":"2024-04-26T20:58:45.069343+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:20:59.906583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.362437+0000","last_change":"2024-04-26T20:58:43.073833+0000","last_active":"2024-04-26T20:58:45.362437+0000","last_peered":"2024-04-26T20:58:45.362437+0000","last_clean":"2024-04-26T20:58:45.362437+0000","last_became_active":"2024-04-26T20:58:43.070958+0000","last_became_peered":"2024-04-26T20:58:43.070958+0000","last_unstale":"2024-04-26T20:58:45.362437+0000","last_undegraded":"2024-04-26T20:58:45.362437+0000","last_fullsized":"2024-04-26T20:58:45.362437+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:17:28.519261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.361992+0000","last_change":"2024-04-26T20:58:44.075716+0000","last_active":"2024-04-26T20:58:45.361992+0000","last_peered":"2024-04-26T20:58:45.361992+0000","last_clean":"2024-04-26T20:58:45.361992+0000","last_became_active":"2024-04-26T20:58:44.075400+0000","last_became_peered":"2024-04-26T20:58:44.075400+0000","last_unstale":"2024-04-26T20:58:45.361992+0000","last_undegraded":"2024-04-26T20:58:45.361992+0000","last_fullsized":"2024-04-26T20:58:45.361992+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:44.521862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.359541+0000","last_change":"2024-04-26T20:58:43.072680+0000","last_active":"2024-04-26T20:58:45.359541+0000","last_peered":"2024-04-26T20:58:45.359541+0000","last_clean":"2024-04-26T20:58:45.359541+0000","last_became_active":"2024-04-26T20:58:43.072369+0000","last_became_peered":"2024-04-26T20:58:43.072369+0000","last_unstale":"2024-04-26T20:58:45.359541+0000","last_undegraded":"2024-04-26T20:58:45.359541+0000","last_fullsized":"2024-04-26T20:58:45.359541+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:45.716150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.359462+0000","last_change":"2024-04-26T20:58:44.072006+0000","last_active":"2024-04-26T20:58:45.359462+0000","last_peered":"2024-04-26T20:58:45.359462+0000","last_clean":"2024-04-26T20:58:45.359462+0000","last_became_active":"2024-04-26T20:58:44.071722+0000","last_became_peered":"2024-04-26T20:58:44.071722+0000","last_unstale":"2024-04-26T20:58:45.359462+0000","last_undegraded":"2024-04-26T20:58:45.359462+0000","last_fullsized":"2024-04-26T20:58:45.359462+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:20:59.906583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.363161+0000","last_change":"2024-04-26T20:58:43.070599+0000","last_active":"2024-04-26T20:58:45.363161+0000","last_peered":"2024-04-26T20:58:45.363161+0000","last_clean":"2024-04-26T20:58:45.363161+0000","last_became_active":"2024-04-26T20:58:43.070312+0000","last_became_peered":"2024-04-26T20:58:43.070312+0000","last_unstale":"2024-04-26T20:58:45.363161+0000","last_undegraded":"2024-04-26T20:58:45.363161+0000","last_fullsized":"2024-04-26T20:58:45.363161+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:51.552587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.361478+0000","last_change":"2024-04-26T20:58:44.072906+0000","last_active":"2024-04-26T20:58:45.361478+0000","last_peered":"2024-04-26T20:58:45.361478+0000","last_clean":"2024-04-26T20:58:45.361478+0000","last_became_active":"2024-04-26T20:58:44.071610+0000","last_became_peered":"2024-04-26T20:58:44.071610+0000","last_unstale":"2024-04-26T20:58:45.361478+0000","last_undegraded":"2024-04-26T20:58:45.361478+0000","last_fullsized":"2024-04-26T20:58:45.361478+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:34:03.561484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.064326+0000","last_change":"2024-04-26T20:58:43.066495+0000","last_active":"2024-04-26T20:58:45.064326+0000","last_peered":"2024-04-26T20:58:45.064326+0000","last_clean":"2024-04-26T20:58:45.064326+0000","last_became_active":"2024-04-26T20:58:43.066214+0000","last_became_peered":"2024-04-26T20:58:43.066214+0000","last_unstale":"2024-04-26T20:58:45.064326+0000","last_undegraded":"2024-04-26T20:58:45.064326+0000","last_fullsized":"2024-04-26T20:58:45.064326+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:54.023695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.360842+0000","last_change":"2024-04-26T20:58:44.090537+0000","last_active":"2024-04-26T20:58:45.360842+0000","last_peered":"2024-04-26T20:58:45.360842+0000","last_clean":"2024-04-26T20:58:45.360842+0000","last_became_active":"2024-04-26T20:58:44.090243+0000","last_became_peered":"2024-04-26T20:58:44.090243+0000","last_unstale":"2024-04-26T20:58:45.360842+0000","last_undegraded":"2024-04-26T20:58:45.360842+0000","last_fullsized":"2024-04-26T20:58:45.360842+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:55.038517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.361536+0000","last_change":"2024-04-26T20:58:43.067727+0000","last_active":"2024-04-26T20:58:45.361536+0000","last_peered":"2024-04-26T20:58:45.361536+0000","last_clean":"2024-04-26T20:58:45.361536+0000","last_became_active":"2024-04-26T20:58:43.066679+0000","last_became_peered":"2024-04-26T20:58:43.066679+0000","last_unstale":"2024-04-26T20:58:45.361536+0000","last_undegraded":"2024-04-26T20:58:45.361536+0000","last_fullsized":"2024-04-26T20:58:45.361536+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:32.719125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.360314+0000","last_change":"2024-04-26T20:58:44.083325+0000","last_active":"2024-04-26T20:58:45.360314+0000","last_peered":"2024-04-26T20:58:45.360314+0000","last_clean":"2024-04-26T20:58:45.360314+0000","last_became_active":"2024-04-26T20:58:44.080805+0000","last_became_peered":"2024-04-26T20:58:44.080805+0000","last_unstale":"2024-04-26T20:58:45.360314+0000","last_undegraded":"2024-04-26T20:58:45.360314+0000","last_fullsized":"2024-04-26T20:58:45.360314+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:22:50.462726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.359788+0000","last_change":"2024-04-26T20:58:43.057010+0000","last_active":"2024-04-26T20:58:45.359788+0000","last_peered":"2024-04-26T20:58:45.359788+0000","last_clean":"2024-04-26T20:58:45.359788+0000","last_became_active":"2024-04-26T20:58:43.056562+0000","last_became_peered":"2024-04-26T20:58:43.056562+0000","last_unstale":"2024-04-26T20:58:45.359788+0000","last_undegraded":"2024-04-26T20:58:45.359788+0000","last_fullsized":"2024-04-26T20:58:45.359788+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:20:58.891761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.067576+0000","last_change":"2024-04-26T20:58:44.076837+0000","last_active":"2024-04-26T20:58:45.067576+0000","last_peered":"2024-04-26T20:58:45.067576+0000","last_clean":"2024-04-26T20:58:45.067576+0000","last_became_active":"2024-04-26T20:58:44.076304+0000","last_became_peered":"2024-04-26T20:58:44.076304+0000","last_unstale":"2024-04-26T20:58:45.067576+0000","last_undegraded":"2024-04-26T20:58:45.067576+0000","last_fullsized":"2024-04-26T20:58:45.067576+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:57.336685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.360366+0000","last_change":"2024-04-26T20:58:43.066688+0000","last_active":"2024-04-26T20:58:45.360366+0000","last_peered":"2024-04-26T20:58:45.360366+0000","last_clean":"2024-04-26T20:58:45.360366+0000","last_became_active":"2024-04-26T20:58:43.066372+0000","last_became_peered":"2024-04-26T20:58:43.066372+0000","last_unstale":"2024-04-26T20:58:45.360366+0000","last_undegraded":"2024-04-26T20:58:45.360366+0000","last_fullsized":"2024-04-26T20:58:45.360366+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:28.654953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.361999+0000","last_change":"2024-04-26T20:58:44.078162+0000","last_active":"2024-04-26T20:58:45.361999+0000","last_peered":"2024-04-26T20:58:45.361999+0000","last_clean":"2024-04-26T20:58:45.361999+0000","last_became_active":"2024-04-26T20:58:44.077146+0000","last_became_peered":"2024-04-26T20:58:44.077146+0000","last_unstale":"2024-04-26T20:58:45.361999+0000","last_undegraded":"2024-04-26T20:58:45.361999+0000","last_fullsized":"2024-04-26T20:58:45.361999+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:22.511947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.363762+0000","last_change":"2024-04-26T20:58:43.061818+0000","last_active":"2024-04-26T20:58:45.363762+0000","last_peered":"2024-04-26T20:58:45.363762+0000","last_clean":"2024-04-26T20:58:45.363762+0000","last_became_active":"2024-04-26T20:58:43.059103+0000","last_became_peered":"2024-04-26T20:58:43.059103+0000","last_unstale":"2024-04-26T20:58:45.363762+0000","last_undegraded":"2024-04-26T20:58:45.363762+0000","last_fullsized":"2024-04-26T20:58:45.363762+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:47.958156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.363629+0000","last_change":"2024-04-26T20:58:44.081007+0000","last_active":"2024-04-26T20:58:45.363629+0000","last_peered":"2024-04-26T20:58:45.363629+0000","last_clean":"2024-04-26T20:58:45.363629+0000","last_became_active":"2024-04-26T20:58:44.080663+0000","last_became_peered":"2024-04-26T20:58:44.080663+0000","last_unstale":"2024-04-26T20:58:45.363629+0000","last_undegraded":"2024-04-26T20:58:45.363629+0000","last_fullsized":"2024-04-26T20:58:45.363629+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:29.167665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.361565+0000","last_change":"2024-04-26T20:58:43.071259+0000","last_active":"2024-04-26T20:58:45.361565+0000","last_peered":"2024-04-26T20:58:45.361565+0000","last_clean":"2024-04-26T20:58:45.361565+0000","last_became_active":"2024-04-26T20:58:43.070952+0000","last_became_peered":"2024-04-26T20:58:43.070952+0000","last_unstale":"2024-04-26T20:58:45.361565+0000","last_undegraded":"2024-04-26T20:58:45.361565+0000","last_fullsized":"2024-04-26T20:58:45.361565+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:57:47.958156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.064916+0000","last_change":"2024-04-26T20:58:44.070970+0000","last_active":"2024-04-26T20:58:45.064916+0000","last_peered":"2024-04-26T20:58:45.064916+0000","last_clean":"2024-04-26T20:58:45.064916+0000","last_became_active":"2024-04-26T20:58:44.070699+0000","last_became_peered":"2024-04-26T20:58:44.070699+0000","last_unstale":"2024-04-26T20:58:45.064916+0000","last_undegraded":"2024-04-26T20:58:45.064916+0000","last_fullsized":"2024-04-26T20:58:45.064916+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:37:08.795516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.363083+0000","last_change":"2024-04-26T20:58:43.072767+0000","last_active":"2024-04-26T20:58:45.363083+0000","last_peered":"2024-04-26T20:58:45.363083+0000","last_clean":"2024-04-26T20:58:45.363083+0000","last_became_active":"2024-04-26T20:58:43.072408+0000","last_became_peered":"2024-04-26T20:58:43.072408+0000","last_unstale":"2024-04-26T20:58:45.363083+0000","last_undegraded":"2024-04-26T20:58:45.363083+0000","last_fullsized":"2024-04-26T20:58:45.363083+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:21.497125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.362693+0000","last_change":"2024-04-26T20:58:44.071168+0000","last_active":"2024-04-26T20:58:45.362693+0000","last_peered":"2024-04-26T20:58:45.362693+0000","last_clean":"2024-04-26T20:58:45.362693+0000","last_became_active":"2024-04-26T20:58:44.070815+0000","last_became_peered":"2024-04-26T20:58:44.070815+0000","last_unstale":"2024-04-26T20:58:45.362693+0000","last_undegraded":"2024-04-26T20:58:45.362693+0000","last_fullsized":"2024-04-26T20:58:45.362693+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:38.450510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.065839+0000","last_change":"2024-04-26T20:58:43.059043+0000","last_active":"2024-04-26T20:58:45.065839+0000","last_peered":"2024-04-26T20:58:45.065839+0000","last_clean":"2024-04-26T20:58:45.065839+0000","last_became_active":"2024-04-26T20:58:43.058718+0000","last_became_peered":"2024-04-26T20:58:43.058718+0000","last_unstale":"2024-04-26T20:58:45.065839+0000","last_undegraded":"2024-04-26T20:58:45.065839+0000","last_fullsized":"2024-04-26T20:58:45.065839+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:26.992295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.068067+0000","last_change":"2024-04-26T20:58:44.086863+0000","last_active":"2024-04-26T20:58:45.068067+0000","last_peered":"2024-04-26T20:58:45.068067+0000","last_clean":"2024-04-26T20:58:45.068067+0000","last_became_active":"2024-04-26T20:58:44.086556+0000","last_became_peered":"2024-04-26T20:58:44.086556+0000","last_unstale":"2024-04-26T20:58:45.068067+0000","last_undegraded":"2024-04-26T20:58:45.068067+0000","last_fullsized":"2024-04-26T20:58:45.068067+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:05:47.374823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.064770+0000","last_change":"2024-04-26T20:58:43.057592+0000","last_active":"2024-04-26T20:58:45.064770+0000","last_peered":"2024-04-26T20:58:45.064770+0000","last_clean":"2024-04-26T20:58:45.064770+0000","last_became_active":"2024-04-26T20:58:43.057285+0000","last_became_peered":"2024-04-26T20:58:43.057285+0000","last_unstale":"2024-04-26T20:58:45.064770+0000","last_undegraded":"2024-04-26T20:58:45.064770+0000","last_fullsized":"2024-04-26T20:58:45.064770+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:30:02.321174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.361131+0000","last_change":"2024-04-26T20:58:44.090309+0000","last_active":"2024-04-26T20:58:45.361131+0000","last_peered":"2024-04-26T20:58:45.361131+0000","last_clean":"2024-04-26T20:58:45.361131+0000","last_became_active":"2024-04-26T20:58:44.085769+0000","last_became_peered":"2024-04-26T20:58:44.085769+0000","last_unstale":"2024-04-26T20:58:45.361131+0000","last_undegraded":"2024-04-26T20:58:45.361131+0000","last_fullsized":"2024-04-26T20:58:45.361131+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:57.868880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.360788+0000","last_change":"2024-04-26T20:58:43.068260+0000","last_active":"2024-04-26T20:58:45.360788+0000","last_peered":"2024-04-26T20:58:45.360788+0000","last_clean":"2024-04-26T20:58:45.360788+0000","last_became_active":"2024-04-26T20:58:43.067852+0000","last_became_peered":"2024-04-26T20:58:43.067852+0000","last_unstale":"2024-04-26T20:58:45.360788+0000","last_undegraded":"2024-04-26T20:58:45.360788+0000","last_fullsized":"2024-04-26T20:58:45.360788+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:58.207157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.063685+0000","last_change":"2024-04-26T20:58:44.070771+0000","last_active":"2024-04-26T20:58:45.063685+0000","last_peered":"2024-04-26T20:58:45.063685+0000","last_clean":"2024-04-26T20:58:45.063685+0000","last_became_active":"2024-04-26T20:58:44.067743+0000","last_became_peered":"2024-04-26T20:58:44.067743+0000","last_unstale":"2024-04-26T20:58:45.063685+0000","last_undegraded":"2024-04-26T20:58:45.063685+0000","last_fullsized":"2024-04-26T20:58:45.063685+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:29.315126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.063951+0000","last_change":"2024-04-26T20:58:43.059009+0000","last_active":"2024-04-26T20:58:45.063951+0000","last_peered":"2024-04-26T20:58:45.063951+0000","last_clean":"2024-04-26T20:58:45.063951+0000","last_became_active":"2024-04-26T20:58:43.058725+0000","last_became_peered":"2024-04-26T20:58:43.058725+0000","last_unstale":"2024-04-26T20:58:45.063951+0000","last_undegraded":"2024-04-26T20:58:45.063951+0000","last_fullsized":"2024-04-26T20:58:45.063951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:22:49.447904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.360077+0000","last_change":"2024-04-26T20:58:44.070281+0000","last_active":"2024-04-26T20:58:45.360077+0000","last_peered":"2024-04-26T20:58:45.360077+0000","last_clean":"2024-04-26T20:58:45.360077+0000","last_became_active":"2024-04-26T20:58:44.069908+0000","last_became_peered":"2024-04-26T20:58:44.069908+0000","last_unstale":"2024-04-26T20:58:45.360077+0000","last_undegraded":"2024-04-26T20:58:45.360077+0000","last_fullsized":"2024-04-26T20:58:45.360077+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:45.018355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":22,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.360623+0000","last_change":"2024-04-26T20:58:43.071075+0000","last_active":"2024-04-26T20:58:45.360623+0000","last_peered":"2024-04-26T20:58:45.360623+0000","last_clean":"2024-04-26T20:58:45.360623+0000","last_became_active":"2024-04-26T20:58:43.070762+0000","last_became_peered":"2024-04-26T20:58:43.070762+0000","last_unstale":"2024-04-26T20:58:45.360623+0000","last_undegraded":"2024-04-26T20:58:45.360623+0000","last_fullsized":"2024-04-26T20:58:45.360623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:23.656127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.066062+0000","last_change":"2024-04-26T20:58:44.083207+0000","last_active":"2024-04-26T20:58:45.066062+0000","last_peered":"2024-04-26T20:58:45.066062+0000","last_clean":"2024-04-26T20:58:45.066062+0000","last_became_active":"2024-04-26T20:58:44.082814+0000","last_became_peered":"2024-04-26T20:58:44.082814+0000","last_unstale":"2024-04-26T20:58:45.066062+0000","last_undegraded":"2024-04-26T20:58:45.066062+0000","last_fullsized":"2024-04-26T20:58:45.066062+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:35:31.143535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.065777+0000","last_change":"2024-04-26T20:58:43.048980+0000","last_active":"2024-04-26T20:58:45.065777+0000","last_peered":"2024-04-26T20:58:45.065777+0000","last_clean":"2024-04-26T20:58:45.065777+0000","last_became_active":"2024-04-26T20:58:43.048625+0000","last_became_peered":"2024-04-26T20:58:43.048625+0000","last_unstale":"2024-04-26T20:58:45.065777+0000","last_undegraded":"2024-04-26T20:58:45.065777+0000","last_fullsized":"2024-04-26T20:58:45.065777+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:40:13.250951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.361876+0000","last_change":"2024-04-26T20:58:44.091037+0000","last_active":"2024-04-26T20:58:45.361876+0000","last_peered":"2024-04-26T20:58:45.361876+0000","last_clean":"2024-04-26T20:58:45.361876+0000","last_became_active":"2024-04-26T20:58:44.087260+0000","last_became_peered":"2024-04-26T20:58:44.087260+0000","last_unstale":"2024-04-26T20:58:45.361876+0000","last_undegraded":"2024-04-26T20:58:45.361876+0000","last_fullsized":"2024-04-26T20:58:45.361876+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:29.167665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.362722+0000","last_change":"2024-04-26T20:58:43.063230+0000","last_active":"2024-04-26T20:58:45.362722+0000","last_peered":"2024-04-26T20:58:45.362722+0000","last_clean":"2024-04-26T20:58:45.362722+0000","last_became_active":"2024-04-26T20:58:43.062868+0000","last_became_peered":"2024-04-26T20:58:43.062868+0000","last_unstale":"2024-04-26T20:58:45.362722+0000","last_undegraded":"2024-04-26T20:58:45.362722+0000","last_fullsized":"2024-04-26T20:58:45.362722+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:32:59.491913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.068440+0000","last_change":"2024-04-26T20:58:44.072990+0000","last_active":"2024-04-26T20:58:45.068440+0000","last_peered":"2024-04-26T20:58:45.068440+0000","last_clean":"2024-04-26T20:58:45.068440+0000","last_became_active":"2024-04-26T20:58:44.070887+0000","last_became_peered":"2024-04-26T20:58:44.070887+0000","last_unstale":"2024-04-26T20:58:45.068440+0000","last_undegraded":"2024-04-26T20:58:45.068440+0000","last_fullsized":"2024-04-26T20:58:45.068440+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:45.324904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.362331+0000","last_change":"2024-04-26T20:58:43.070226+0000","last_active":"2024-04-26T20:58:45.362331+0000","last_peered":"2024-04-26T20:58:45.362331+0000","last_clean":"2024-04-26T20:58:45.362331+0000","last_became_active":"2024-04-26T20:58:43.067146+0000","last_became_peered":"2024-04-26T20:58:43.067146+0000","last_unstale":"2024-04-26T20:58:45.362331+0000","last_undegraded":"2024-04-26T20:58:45.362331+0000","last_fullsized":"2024-04-26T20:58:45.362331+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:20:21.497125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.069040+0000","last_change":"2024-04-26T20:58:44.080869+0000","last_active":"2024-04-26T20:58:45.069040+0000","last_peered":"2024-04-26T20:58:45.069040+0000","last_clean":"2024-04-26T20:58:45.069040+0000","last_became_active":"2024-04-26T20:58:44.080545+0000","last_became_peered":"2024-04-26T20:58:44.080545+0000","last_unstale":"2024-04-26T20:58:45.069040+0000","last_undegraded":"2024-04-26T20:58:45.069040+0000","last_fullsized":"2024-04-26T20:58:45.069040+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:45.355922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.064934+0000","last_change":"2024-04-26T20:58:43.070073+0000","last_active":"2024-04-26T20:58:45.064934+0000","last_peered":"2024-04-26T20:58:45.064934+0000","last_clean":"2024-04-26T20:58:45.064934+0000","last_became_active":"2024-04-26T20:58:43.069554+0000","last_became_peered":"2024-04-26T20:58:43.069554+0000","last_unstale":"2024-04-26T20:58:45.064934+0000","last_undegraded":"2024-04-26T20:58:45.064934+0000","last_fullsized":"2024-04-26T20:58:45.064934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:44.003533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.063811+0000","last_change":"2024-04-26T20:58:44.073119+0000","last_active":"2024-04-26T20:58:45.063811+0000","last_peered":"2024-04-26T20:58:45.063811+0000","last_clean":"2024-04-26T20:58:45.063811+0000","last_became_active":"2024-04-26T20:58:44.072807+0000","last_became_peered":"2024-04-26T20:58:44.072807+0000","last_unstale":"2024-04-26T20:58:45.063811+0000","last_undegraded":"2024-04-26T20:58:45.063811+0000","last_fullsized":"2024-04-26T20:58:45.063811+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:50:43.039692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.064886+0000","last_change":"2024-04-26T20:58:43.066409+0000","last_active":"2024-04-26T20:58:45.064886+0000","last_peered":"2024-04-26T20:58:45.064886+0000","last_clean":"2024-04-26T20:58:45.064886+0000","last_became_active":"2024-04-26T20:58:43.065775+0000","last_became_peered":"2024-04-26T20:58:43.065775+0000","last_unstale":"2024-04-26T20:58:45.064886+0000","last_undegraded":"2024-04-26T20:58:45.064886+0000","last_fullsized":"2024-04-26T20:58:45.064886+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:28:40.246541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.359062+0000","last_change":"2024-04-26T20:58:44.079988+0000","last_active":"2024-04-26T20:58:45.359062+0000","last_peered":"2024-04-26T20:58:45.359062+0000","last_clean":"2024-04-26T20:58:45.359062+0000","last_became_active":"2024-04-26T20:58:44.079643+0000","last_became_peered":"2024-04-26T20:58:44.079643+0000","last_unstale":"2024-04-26T20:58:45.359062+0000","last_undegraded":"2024-04-26T20:58:45.359062+0000","last_fullsized":"2024-04-26T20:58:45.359062+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:25:45.355922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.359676+0000","last_change":"2024-04-26T20:58:43.072643+0000","last_active":"2024-04-26T20:58:45.359676+0000","last_peered":"2024-04-26T20:58:45.359676+0000","last_clean":"2024-04-26T20:58:45.359676+0000","last_became_active":"2024-04-26T20:58:43.072358+0000","last_became_peered":"2024-04-26T20:58:43.072358+0000","last_unstale":"2024-04-26T20:58:45.359676+0000","last_undegraded":"2024-04-26T20:58:45.359676+0000","last_fullsized":"2024-04-26T20:58:45.359676+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:47.781660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.065332+0000","last_change":"2024-04-26T20:58:44.070546+0000","last_active":"2024-04-26T20:58:45.065332+0000","last_peered":"2024-04-26T20:58:45.065332+0000","last_clean":"2024-04-26T20:58:45.065332+0000","last_became_active":"2024-04-26T20:58:44.070200+0000","last_became_peered":"2024-04-26T20:58:44.070200+0000","last_unstale":"2024-04-26T20:58:45.065332+0000","last_undegraded":"2024-04-26T20:58:45.065332+0000","last_fullsized":"2024-04-26T20:58:45.065332+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:41:54.010847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.360741+0000","last_change":"2024-04-26T20:58:43.070983+0000","last_active":"2024-04-26T20:58:45.360741+0000","last_peered":"2024-04-26T20:58:45.360741+0000","last_clean":"2024-04-26T20:58:45.360741+0000","last_became_active":"2024-04-26T20:58:43.070724+0000","last_became_peered":"2024-04-26T20:58:43.070724+0000","last_unstale":"2024-04-26T20:58:45.360741+0000","last_undegraded":"2024-04-26T20:58:45.360741+0000","last_fullsized":"2024-04-26T20:58:45.360741+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:45.562738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.360271+0000","last_change":"2024-04-26T20:58:44.072029+0000","last_active":"2024-04-26T20:58:45.360271+0000","last_peered":"2024-04-26T20:58:45.360271+0000","last_clean":"2024-04-26T20:58:45.360271+0000","last_became_active":"2024-04-26T20:58:44.068443+0000","last_became_peered":"2024-04-26T20:58:44.068443+0000","last_unstale":"2024-04-26T20:58:45.360271+0000","last_undegraded":"2024-04-26T20:58:45.360271+0000","last_fullsized":"2024-04-26T20:58:45.360271+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:53:15.209235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.359876+0000","last_change":"2024-04-26T20:58:43.069835+0000","last_active":"2024-04-26T20:58:45.359876+0000","last_peered":"2024-04-26T20:58:45.359876+0000","last_clean":"2024-04-26T20:58:45.359876+0000","last_became_active":"2024-04-26T20:58:43.069535+0000","last_became_peered":"2024-04-26T20:58:43.069535+0000","last_unstale":"2024-04-26T20:58:45.359876+0000","last_undegraded":"2024-04-26T20:58:45.359876+0000","last_fullsized":"2024-04-26T20:58:45.359876+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:20:58.891761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.069505+0000","last_change":"2024-04-26T20:58:44.077625+0000","last_active":"2024-04-26T20:58:45.069505+0000","last_peered":"2024-04-26T20:58:45.069505+0000","last_clean":"2024-04-26T20:58:45.069505+0000","last_became_active":"2024-04-26T20:58:44.077344+0000","last_became_peered":"2024-04-26T20:58:44.077344+0000","last_unstale":"2024-04-26T20:58:45.069505+0000","last_undegraded":"2024-04-26T20:58:45.069505+0000","last_fullsized":"2024-04-26T20:58:45.069505+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:52.567410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"45'1","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.065496+0000","last_change":"2024-04-26T20:58:43.060904+0000","last_active":"2024-04-26T20:58:45.065496+0000","last_peered":"2024-04-26T20:58:45.065496+0000","last_clean":"2024-04-26T20:58:45.065496+0000","last_became_active":"2024-04-26T20:58:43.060511+0000","last_became_peered":"2024-04-26T20:58:43.060511+0000","last_unstale":"2024-04-26T20:58:45.065496+0000","last_undegraded":"2024-04-26T20:58:45.065496+0000","last_fullsized":"2024-04-26T20:58:45.065496+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:37.435688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.361134+0000","last_change":"2024-04-26T20:58:44.083743+0000","last_active":"2024-04-26T20:58:45.361134+0000","last_peered":"2024-04-26T20:58:45.361134+0000","last_clean":"2024-04-26T20:58:45.361134+0000","last_became_active":"2024-04-26T20:58:44.081424+0000","last_became_peered":"2024-04-26T20:58:44.081424+0000","last_unstale":"2024-04-26T20:58:45.361134+0000","last_undegraded":"2024-04-26T20:58:45.361134+0000","last_fullsized":"2024-04-26T20:58:45.361134+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:37:45.018355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.363379+0000","last_change":"2024-04-26T20:58:43.075337+0000","last_active":"2024-04-26T20:58:45.363379+0000","last_peered":"2024-04-26T20:58:45.363379+0000","last_clean":"2024-04-26T20:58:45.363379+0000","last_became_active":"2024-04-26T20:58:43.075019+0000","last_became_peered":"2024-04-26T20:58:43.075019+0000","last_unstale":"2024-04-26T20:58:45.363379+0000","last_undegraded":"2024-04-26T20:58:45.363379+0000","last_fullsized":"2024-04-26T20:58:45.363379+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:54:41.335687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.363267+0000","last_change":"2024-04-26T20:58:44.069662+0000","last_active":"2024-04-26T20:58:45.363267+0000","last_peered":"2024-04-26T20:58:45.363267+0000","last_clean":"2024-04-26T20:58:45.363267+0000","last_became_active":"2024-04-26T20:58:44.069126+0000","last_became_peered":"2024-04-26T20:58:44.069126+0000","last_unstale":"2024-04-26T20:58:45.363267+0000","last_undegraded":"2024-04-26T20:58:45.363267+0000","last_fullsized":"2024-04-26T20:58:45.363267+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:26.672532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.358709+0000","last_change":"2024-04-26T20:58:43.071692+0000","last_active":"2024-04-26T20:58:45.358709+0000","last_peered":"2024-04-26T20:58:45.358709+0000","last_clean":"2024-04-26T20:58:45.358709+0000","last_became_active":"2024-04-26T20:58:43.069775+0000","last_became_peered":"2024-04-26T20:58:43.069775+0000","last_unstale":"2024-04-26T20:58:45.358709+0000","last_undegraded":"2024-04-26T20:58:45.358709+0000","last_fullsized":"2024-04-26T20:58:45.358709+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:15.110999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.063953+0000","last_change":"2024-04-26T20:58:44.061879+0000","last_active":"2024-04-26T20:58:45.063953+0000","last_peered":"2024-04-26T20:58:45.063953+0000","last_clean":"2024-04-26T20:58:45.063953+0000","last_became_active":"2024-04-26T20:58:44.060183+0000","last_became_peered":"2024-04-26T20:58:44.060183+0000","last_unstale":"2024-04-26T20:58:45.063953+0000","last_undegraded":"2024-04-26T20:58:45.063953+0000","last_fullsized":"2024-04-26T20:58:45.063953+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:44:59.244879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.362813+0000","last_change":"2024-04-26T20:58:43.075348+0000","last_active":"2024-04-26T20:58:45.362813+0000","last_peered":"2024-04-26T20:58:45.362813+0000","last_clean":"2024-04-26T20:58:45.362813+0000","last_became_active":"2024-04-26T20:58:43.075012+0000","last_became_peered":"2024-04-26T20:58:43.075012+0000","last_unstale":"2024-04-26T20:58:45.362813+0000","last_undegraded":"2024-04-26T20:58:45.362813+0000","last_fullsized":"2024-04-26T20:58:45.362813+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:48:47.781660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.064192+0000","last_change":"2024-04-26T20:58:44.076402+0000","last_active":"2024-04-26T20:58:45.064192+0000","last_peered":"2024-04-26T20:58:45.064192+0000","last_clean":"2024-04-26T20:58:45.064192+0000","last_became_active":"2024-04-26T20:58:44.075961+0000","last_became_peered":"2024-04-26T20:58:44.075961+0000","last_unstale":"2024-04-26T20:58:45.064192+0000","last_undegraded":"2024-04-26T20:58:45.064192+0000","last_fullsized":"2024-04-26T20:58:45.064192+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:53:43.970964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.361853+0000","last_change":"2024-04-26T20:58:43.054765+0000","last_active":"2024-04-26T20:58:45.361853+0000","last_peered":"2024-04-26T20:58:45.361853+0000","last_clean":"2024-04-26T20:58:45.361853+0000","last_became_active":"2024-04-26T20:58:43.054455+0000","last_became_peered":"2024-04-26T20:58:43.054455+0000","last_unstale":"2024-04-26T20:58:45.361853+0000","last_undegraded":"2024-04-26T20:58:45.361853+0000","last_fullsized":"2024-04-26T20:58:45.361853+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:17:45.562738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.360345+0000","last_change":"2024-04-26T20:58:44.079343+0000","last_active":"2024-04-26T20:58:45.360345+0000","last_peered":"2024-04-26T20:58:45.360345+0000","last_clean":"2024-04-26T20:58:45.360345+0000","last_became_active":"2024-04-26T20:58:44.078616+0000","last_became_peered":"2024-04-26T20:58:44.078616+0000","last_unstale":"2024-04-26T20:58:45.360345+0000","last_undegraded":"2024-04-26T20:58:45.360345+0000","last_fullsized":"2024-04-26T20:58:45.360345+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:55.038517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.358691+0000","last_change":"2024-04-26T20:58:43.065790+0000","last_active":"2024-04-26T20:58:45.358691+0000","last_peered":"2024-04-26T20:58:45.358691+0000","last_clean":"2024-04-26T20:58:45.358691+0000","last_became_active":"2024-04-26T20:58:43.065469+0000","last_became_peered":"2024-04-26T20:58:43.065469+0000","last_unstale":"2024-04-26T20:58:45.358691+0000","last_undegraded":"2024-04-26T20:58:45.358691+0000","last_fullsized":"2024-04-26T20:58:45.358691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:43:56.321863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.360635+0000","last_change":"2024-04-26T20:58:44.055168+0000","last_active":"2024-04-26T20:58:45.360635+0000","last_peered":"2024-04-26T20:58:45.360635+0000","last_clean":"2024-04-26T20:58:45.360635+0000","last_became_active":"2024-04-26T20:58:44.054290+0000","last_became_peered":"2024-04-26T20:58:44.054290+0000","last_unstale":"2024-04-26T20:58:45.360635+0000","last_undegraded":"2024-04-26T20:58:45.360635+0000","last_fullsized":"2024-04-26T20:58:45.360635+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:52:59.221980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.360631+0000","last_change":"2024-04-26T20:58:43.056930+0000","last_active":"2024-04-26T20:58:45.360631+0000","last_peered":"2024-04-26T20:58:45.360631+0000","last_clean":"2024-04-26T20:58:45.360631+0000","last_became_active":"2024-04-26T20:58:43.055764+0000","last_became_peered":"2024-04-26T20:58:43.055764+0000","last_unstale":"2024-04-26T20:58:45.360631+0000","last_undegraded":"2024-04-26T20:58:45.360631+0000","last_fullsized":"2024-04-26T20:58:45.360631+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:32.719125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.361636+0000","last_change":"2024-04-26T20:58:44.086588+0000","last_active":"2024-04-26T20:58:45.361636+0000","last_peered":"2024-04-26T20:58:45.361636+0000","last_clean":"2024-04-26T20:58:45.361636+0000","last_became_active":"2024-04-26T20:58:44.085777+0000","last_became_peered":"2024-04-26T20:58:44.085777+0000","last_unstale":"2024-04-26T20:58:45.361636+0000","last_undegraded":"2024-04-26T20:58:45.361636+0000","last_fullsized":"2024-04-26T20:58:45.361636+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:43.016772+0000","last_clean_scrub_stamp":"2024-04-26T20:58:43.016772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:24:44.521862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.361868+0000","last_change":"2024-04-26T20:58:43.074219+0000","last_active":"2024-04-26T20:58:45.361868+0000","last_peered":"2024-04-26T20:58:45.361868+0000","last_clean":"2024-04-26T20:58:45.361868+0000","last_became_active":"2024-04-26T20:58:43.072029+0000","last_became_peered":"2024-04-26T20:58:43.072029+0000","last_unstale":"2024-04-26T20:58:45.361868+0000","last_undegraded":"2024-04-26T20:58:45.361868+0000","last_fullsized":"2024-04-26T20:58:45.361868+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:58:42.001950+0000","last_clean_scrub_stamp":"2024-04-26T20:58:42.001950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:50:51.552587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":216,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T20:58:45.359715+0000","last_change":"2024-04-26T20:56:46.848603+0000","last_active":"2024-04-26T20:58:45.359715+0000","last_peered":"2024-04-26T20:58:45.359715+0000","last_clean":"2024-04-26T20:58:45.359715+0000","last_became_active":"2024-04-26T20:56:46.846091+0000","last_became_peered":"2024-04-26T20:56:46.846091+0000","last_unstale":"2024-04-26T20:58:45.359715+0000","last_undegraded":"2024-04-26T20:58:45.359715+0000","last_fullsized":"2024-04-26T20:58:45.359715+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_clean_scrub_stamp":"2024-04-26T20:56:45.594041+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:00.232387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1126400,"data_stored":804937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574875,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":215151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":532480,"data_stored":214549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":540672,"data_stored":216091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":540672,"data_stored":215117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":544768,"data_stored":216113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31700,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":532480,"data_stored":214549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":804881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T20:58:54.094 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T20:58:54.094 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T20:58:54.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.355+0000 7f413a200640 1 Processor -- start 2024-04-26T20:58:54.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.355+0000 7f413a200640 1 -- start start 2024-04-26T20:58:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.356+0000 7f413a200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41340d3510 0x7f41340cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.356+0000 7f413a200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41340b3f20 con 0x7f41340d1750 2024-04-26T20:58:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.356+0000 7f413a200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41340b4200 con 0x7f41340d1370 2024-04-26T20:58:54.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.356+0000 7f413a200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f41340b3d00 con 0x7f41340d3510 2024-04-26T20:58:54.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.356+0000 7f4139800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f41340d1750 0x7f41340b0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34144/0 (socket says 172.21.15.28:34144) 2024-04-26T20:58:54.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.356+0000 7f4139800640 1 -- 172.21.15.28:0/3794323740 learned_addr learned my addr 172.21.15.28:0/3794323740 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:54.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.357+0000 7f4133e00640 1 --2- 172.21.15.28:0/3794323740 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41340d3510 0x7f41340cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:54.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.357+0000 7f4133400640 1 -- 172.21.15.28:0/3794323740 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 585054180 0 0) 0x7f41340b3f20 con 0x7f41340d1750 2024-04-26T20:58:54.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.357+0000 7f4133400640 1 -- 172.21.15.28:0/3794323740 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4118003520 con 0x7f41340d1750 2024-04-26T20:58:54.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.357+0000 7f4133400640 1 -- 172.21.15.28:0/3794323740 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3652800951 0 0) 0x7f4118003520 con 0x7f41340d1750 2024-04-26T20:58:54.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.357+0000 7f4133400640 1 -- 172.21.15.28:0/3794323740 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41340d3510 msgr2=0x7f41340cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:54.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.357+0000 7f4133400640 1 --2- 172.21.15.28:0/3794323740 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41340d3510 0x7f41340cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.357+0000 7f4133400640 1 -- 172.21.15.28:0/3794323740 >> v1:172.21.15.103:6789/0 conn(0x7f41340d1370 legacy=0x7f41340b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:54.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.357+0000 7f4133400640 1 -- 172.21.15.28:0/3794323740 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41340b44e0 con 0x7f41340d1750 2024-04-26T20:58:54.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.357+0000 7f4133400640 1 -- 172.21.15.28:0/3794323740 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f412c0019f0 con 0x7f41340d1750 2024-04-26T20:58:54.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.357+0000 7f4133400640 1 -- 172.21.15.28:0/3794323740 >> v1:172.21.15.28:6789/0 conn(0x7f41340d1750 legacy=0x7f41340b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:54.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.357+0000 7f4133400640 1 --2- 172.21.15.28:0/3794323740 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41340d3510 0x7f41180039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:54.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.358+0000 7f4133400640 1 --2- 172.21.15.28:0/3794323740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4118003f60 0x7f4118004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:54.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.358+0000 7f4133400640 1 --2- 172.21.15.28:0/3794323740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4118004d40 0x7f41180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:54.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.358+0000 7f4133400640 1 -- 172.21.15.28:0/3794323740 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f41340b3d00 con 0x7f4118003f60 2024-04-26T20:58:54.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.358+0000 7f4133400640 1 -- 172.21.15.28:0/3794323740 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f411800bb30 con 0x7f4118004d40 2024-04-26T20:58:54.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.358+0000 7f4133400640 1 -- 172.21.15.28:0/3794323740 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f411800bd30 con 0x7f41340d3510 2024-04-26T20:58:54.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.358+0000 7f4133e00640 1 --2- 172.21.15.28:0/3794323740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4118003f60 0x7f4118004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:54.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.358+0000 7f4138e00640 1 --2- 172.21.15.28:0/3794323740 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41340d3510 0x7f41180039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:54.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.358+0000 7f4139800640 1 --2- 172.21.15.28:0/3794323740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4118004d40 0x7f41180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:54.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.358+0000 7f4133e00640 1 -- 172.21.15.28:0/3794323740 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41340d3510 msgr2=0x7f41180039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:54.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.358+0000 7f4133e00640 1 --2- 172.21.15.28:0/3794323740 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41340d3510 0x7f41180039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.358+0000 7f4133e00640 1 -- 172.21.15.28:0/3794323740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4118004d40 msgr2=0x7f41180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:54.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.358+0000 7f4133e00640 1 --2- 172.21.15.28:0/3794323740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4118004d40 0x7f41180051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.358+0000 7f4133e00640 1 -- 172.21.15.28:0/3794323740 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4118003520 con 0x7f4118003f60 2024-04-26T20:58:54.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.359+0000 7f4133e00640 1 --2- 172.21.15.28:0/3794323740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4118003f60 0x7f4118004800 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f411c009ac0 tx=0x7f4118005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0aeba767c038e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:54.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.360+0000 7f4133400640 1 -- 172.21.15.28:0/3794323740 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c036070 con 0x7f4118003f60 2024-04-26T20:58:54.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.360+0000 7f4133400640 1 -- 172.21.15.28:0/3794323740 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f411c031400 con 0x7f4118003f60 2024-04-26T20:58:54.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.360+0000 7f4133400640 1 -- 172.21.15.28:0/3794323740 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f411c009690 con 0x7f4118003f60 2024-04-26T20:58:54.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.360+0000 7f413a200640 1 -- 172.21.15.28:0/3794323740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4118003f60 msgr2=0x7f4118004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:54.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.360+0000 7f413a200640 1 --2- 172.21.15.28:0/3794323740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4118003f60 0x7f4118004800 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f411c009ac0 tx=0x7f4118005830 comp rx=0 tx=0).stop 2024-04-26T20:58:54.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.360+0000 7f4139800640 1 -- 172.21.15.28:0/3794323740 reap_dead start 2024-04-26T20:58:54.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.360+0000 7f413a200640 1 -- 172.21.15.28:0/3794323740 shutdown_connections 2024-04-26T20:58:54.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.360+0000 7f413a200640 1 -- 172.21.15.28:0/3794323740 >> 172.21.15.28:0/3794323740 conn(0x7f413409ba50 msgr2=0x7f413409be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:54.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.360+0000 7f413a200640 1 -- 172.21.15.28:0/3794323740 shutdown_connections 2024-04-26T20:58:54.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.360+0000 7f413a200640 1 -- 172.21.15.28:0/3794323740 wait complete. 2024-04-26T20:58:54.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.362+0000 7f413a200640 1 Processor -- start 2024-04-26T20:58:54.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.362+0000 7f413a200640 1 -- start start 2024-04-26T20:58:54.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.363+0000 7f413a200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4118003f60 0x7f413406df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:54.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.363+0000 7f413a200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41340c6810 0x7f4134152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:54.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.363+0000 7f413a200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41340cb9b0 0x7f41340cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:54.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.363+0000 7f413a200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f41340cfe20 con 0x7f41340cb9b0 2024-04-26T20:58:54.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.363+0000 7f413a200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f41340cfca0 con 0x7f4118003f60 2024-04-26T20:58:54.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.363+0000 7f413a200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f41340cffa0 con 0x7f41340c6810 2024-04-26T20:58:54.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.363+0000 7f4139800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41340cb9b0 0x7f41340cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:54.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.363+0000 7f4139800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41340cb9b0 0x7f41340cbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47086/0 (socket says 172.21.15.28:47086) 2024-04-26T20:58:54.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.363+0000 7f4139800640 1 -- 172.21.15.28:0/1591035535 learned_addr learned my addr 172.21.15.28:0/1591035535 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:54.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.364+0000 7f4133e00640 1 --2- 172.21.15.28:0/1591035535 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41340c6810 0x7f4134152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:54.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.364+0000 7f4139800640 1 -- 172.21.15.28:0/1591035535 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41340c6810 msgr2=0x7f4134152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:54.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.364+0000 7f4139800640 1 --2- 172.21.15.28:0/1591035535 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41340c6810 0x7f4134152080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.364+0000 7f4139800640 1 -- 172.21.15.28:0/1591035535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4118003f60 msgr2=0x7f413406df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:54.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.364+0000 7f4139800640 1 --2- 172.21.15.28:0/1591035535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4118003f60 0x7f413406df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.364+0000 7f4139800640 1 -- 172.21.15.28:0/1591035535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413415aa90 con 0x7f41340cb9b0 2024-04-26T20:58:54.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.364+0000 7f4139800640 1 --2- 172.21.15.28:0/1591035535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41340cb9b0 0x7f41340cbe30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f412c001580 tx=0x7f412c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:54.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.364+0000 7f4130c00640 1 -- 172.21.15.28:0/1591035535 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c004d60 con 0x7f41340cb9b0 2024-04-26T20:58:54.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.364+0000 7f4130c00640 1 -- 172.21.15.28:0/1591035535 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f412c00bb80 con 0x7f41340cb9b0 2024-04-26T20:58:54.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.364+0000 7f4130c00640 1 -- 172.21.15.28:0/1591035535 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412c007070 con 0x7f41340cb9b0 2024-04-26T20:58:54.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.364+0000 7f413a200640 1 -- 172.21.15.28:0/1591035535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41340c81f0 con 0x7f41340cb9b0 2024-04-26T20:58:54.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.365+0000 7f413a200640 1 -- 172.21.15.28:0/1591035535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41340c7040 con 0x7f41340cb9b0 2024-04-26T20:58:54.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.366+0000 7f413a200640 1 -- 172.21.15.28:0/1591035535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f413406a2c0 con 0x7f41340cb9b0 2024-04-26T20:58:54.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.367+0000 7f4130c00640 1 -- 172.21.15.28:0/1591035535 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f412c007210 con 0x7f41340cb9b0 2024-04-26T20:58:54.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.368+0000 7f4130c00640 1 --2- 172.21.15.28:0/1591035535 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f411006c8f0 0x7f411006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:54.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.368+0000 7f4130c00640 1 -- 172.21.15.28:0/1591035535 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f412c08bac0 con 0x7f41340cb9b0 2024-04-26T20:58:54.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.368+0000 7f4138e00640 1 --2- 172.21.15.28:0/1591035535 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f411006c8f0 0x7f411006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:54.370 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.369+0000 7f4138e00640 1 --2- 172.21.15.28:0/1591035535 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f411006c8f0 0x7f411006edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4124007fd0 tx=0x7f4124009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.372+0000 7f4130c00640 1 -- 172.21.15.28:0/1591035535 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f412c059e50 con 0x7f41340cb9b0 2024-04-26T20:58:54.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.662+0000 7f413a200640 1 -- 172.21.15.28:0/1591035535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f41340b8cc0 con 0x7f41340cb9b0 2024-04-26T20:58:54.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.667+0000 7f4130c00640 1 -- 172.21.15.28:0/1591035535 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) ==== 190+0+147 (secure 0 0 0) 0x7f412c059750 con 0x7f41340cb9b0 2024-04-26T20:58:54.668 INFO:teuthology.orchestra.run.smithi028.stdout:[client.0] 2024-04-26T20:58:54.668 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQDVFCxm/rQRLhAAGsnaSU0YseRGCeDvxvvp/A== 2024-04-26T20:58:54.668 INFO:teuthology.orchestra.run.smithi028.stdout: caps mds = "allow" 2024-04-26T20:58:54.668 INFO:teuthology.orchestra.run.smithi028.stdout: caps mon = "allow r" 2024-04-26T20:58:54.668 INFO:teuthology.orchestra.run.smithi028.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T20:58:54.669 INFO:teuthology.orchestra.run.smithi028.stderr:updated caps for client.0 2024-04-26T20:58:54.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.669+0000 7f413a200640 1 -- 172.21.15.28:0/1591035535 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f411006c8f0 msgr2=0x7f411006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:54.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.669+0000 7f413a200640 1 --2- 172.21.15.28:0/1591035535 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f411006c8f0 0x7f411006edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4124007fd0 tx=0x7f4124009040 comp rx=0 tx=0).stop 2024-04-26T20:58:54.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.669+0000 7f413a200640 1 -- 172.21.15.28:0/1591035535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41340cb9b0 msgr2=0x7f41340cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:54.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.669+0000 7f413a200640 1 --2- 172.21.15.28:0/1591035535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41340cb9b0 0x7f41340cbe30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f412c001580 tx=0x7f412c00b4a0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.670+0000 7f413a200640 1 -- 172.21.15.28:0/1591035535 shutdown_connections 2024-04-26T20:58:54.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.670+0000 7f413a200640 1 --2- 172.21.15.28:0/1591035535 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f411006c8f0 0x7f411006edb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.670+0000 7f413a200640 1 --2- 172.21.15.28:0/1591035535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41340cb9b0 0x7f41340cbe30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.670+0000 7f413a200640 1 --2- 172.21.15.28:0/1591035535 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f41340c6810 0x7f4134152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.670+0000 7f413a200640 1 --2- 172.21.15.28:0/1591035535 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4118003f60 0x7f413406df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.670+0000 7f413a200640 1 -- 172.21.15.28:0/1591035535 >> 172.21.15.28:0/1591035535 conn(0x7f413409ba50 msgr2=0x7f4134070250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:54.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.670+0000 7f413a200640 1 -- 172.21.15.28:0/1591035535 shutdown_connections 2024-04-26T20:58:54.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.670+0000 7f413a200640 1 -- 172.21.15.28:0/1591035535 wait complete. 2024-04-26T20:58:54.691 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T20:58:54.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.904+0000 7faac5600640 1 Processor -- start 2024-04-26T20:58:54.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.904+0000 7faac5600640 1 -- start start 2024-04-26T20:58:54.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.906+0000 7faac5600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac0146980 0x7faac00b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:54.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.906+0000 7faac5600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7faac00740e0 0x7faac00b6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:54.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.906+0000 7faac5600640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7faac015d5b0 0x7faac015da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:54.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.906+0000 7faac5600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7faac015e510 con 0x7faac00740e0 2024-04-26T20:58:54.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.906+0000 7faac5600640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7faac00d0a30 con 0x7faac015d5b0 2024-04-26T20:58:54.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.906+0000 7faac5600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7faac015df80 con 0x7faac0146980 2024-04-26T20:58:54.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.906+0000 7faabe000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7faac00740e0 0x7faac00b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:54.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.907+0000 7faabe000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7faac00740e0 0x7faac00b6640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47100/0 (socket says 172.21.15.28:47100) 2024-04-26T20:58:54.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.907+0000 7faabe000640 1 -- 172.21.15.28:0/1923792836 learned_addr learned my addr 172.21.15.28:0/1923792836 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:54.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.907+0000 7faabea00640 1 --2- 172.21.15.28:0/1923792836 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac0146980 0x7faac00b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:54.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.907+0000 7faabf400640 1 --2- 172.21.15.28:0/1923792836 >> v2:172.21.15.103:3300/0 conn(0x7faac015d5b0 0x7faac015da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:54.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.907+0000 7faabe000640 1 -- 172.21.15.28:0/1923792836 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac0146980 msgr2=0x7faac00b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:54.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.907+0000 7faabe000640 1 --2- 172.21.15.28:0/1923792836 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac0146980 0x7faac00b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.907+0000 7faabe000640 1 -- 172.21.15.28:0/1923792836 >> v2:172.21.15.103:3300/0 conn(0x7faac015d5b0 msgr2=0x7faac015da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:54.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.907+0000 7faabe000640 1 --2- 172.21.15.28:0/1923792836 >> v2:172.21.15.103:3300/0 conn(0x7faac015d5b0 0x7faac015da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.907+0000 7faabe000640 1 -- 172.21.15.28:0/1923792836 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac015e230 con 0x7faac00740e0 2024-04-26T20:58:54.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.907+0000 7faabea00640 1 --2- 172.21.15.28:0/1923792836 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac0146980 0x7faac00b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T20:58:54.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.908+0000 7faabe000640 1 --2- 172.21.15.28:0/1923792836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faac00740e0 0x7faac00b6640 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7faac0075f80 tx=0x7faaa802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2883202ecf91440 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:54.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.909+0000 7faabd600640 1 -- 172.21.15.28:0/1923792836 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8037070 con 0x7faac00740e0 2024-04-26T20:58:54.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.909+0000 7faabd600640 1 -- 172.21.15.28:0/1923792836 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa8032440 con 0x7faac00740e0 2024-04-26T20:58:54.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.909+0000 7faabd600640 1 -- 172.21.15.28:0/1923792836 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8032810 con 0x7faac00740e0 2024-04-26T20:58:54.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.909+0000 7faac5600640 1 -- 172.21.15.28:0/1923792836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faac00740e0 msgr2=0x7faac00b6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:54.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.909+0000 7faac5600640 1 --2- 172.21.15.28:0/1923792836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faac00740e0 0x7faac00b6640 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7faac0075f80 tx=0x7faaa802a140 comp rx=0 tx=0).stop 2024-04-26T20:58:54.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.910+0000 7faac5600640 1 -- 172.21.15.28:0/1923792836 shutdown_connections 2024-04-26T20:58:54.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.910+0000 7faac5600640 1 --2- 172.21.15.28:0/1923792836 >> v2:172.21.15.103:3300/0 conn(0x7faac015d5b0 0x7faac015da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.910+0000 7faac5600640 1 --2- 172.21.15.28:0/1923792836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faac00740e0 0x7faac00b6640 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.910+0000 7faac5600640 1 --2- 172.21.15.28:0/1923792836 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac0146980 0x7faac00b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.910+0000 7faac5600640 1 -- 172.21.15.28:0/1923792836 >> 172.21.15.28:0/1923792836 conn(0x7faac006e220 msgr2=0x7faac006e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:54.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.910+0000 7faac5600640 1 -- 172.21.15.28:0/1923792836 shutdown_connections 2024-04-26T20:58:54.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.910+0000 7faac5600640 1 -- 172.21.15.28:0/1923792836 wait complete. 2024-04-26T20:58:54.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.912+0000 7faac5600640 1 Processor -- start 2024-04-26T20:58:54.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.912+0000 7faac5600640 1 -- start start 2024-04-26T20:58:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.913+0000 7faac5600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac00740e0 0x7faac0157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.913+0000 7faac5600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faac0146980 0x7faac0139480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.913+0000 7faac5600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faac015d5b0 0x7faac0141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.913+0000 7faac5600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7faac0073bc0 con 0x7faac0146980 2024-04-26T20:58:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.913+0000 7faac5600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faac00738e0 con 0x7faac015d5b0 2024-04-26T20:58:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.913+0000 7faac5600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7faac0160bd0 con 0x7faac00740e0 2024-04-26T20:58:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.913+0000 7faabe000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faac0146980 0x7faac0139480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.913+0000 7faabe000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faac0146980 0x7faac0139480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47114/0 (socket says 172.21.15.28:47114) 2024-04-26T20:58:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.913+0000 7faabe000640 1 -- 172.21.15.28:0/2895230765 learned_addr learned my addr 172.21.15.28:0/2895230765 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:54.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.913+0000 7faabea00640 1 --2- 172.21.15.28:0/2895230765 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac00740e0 0x7faac0157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.914+0000 7faabe000640 1 -- 172.21.15.28:0/2895230765 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac00740e0 msgr2=0x7faac0157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.914+0000 7faabe000640 1 --2- 172.21.15.28:0/2895230765 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac00740e0 0x7faac0157e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.914+0000 7faabf400640 1 --2- 172.21.15.28:0/2895230765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faac015d5b0 0x7faac0141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.914+0000 7faabe000640 1 -- 172.21.15.28:0/2895230765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faac015d5b0 msgr2=0x7faac0141f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.914+0000 7faabe000640 1 --2- 172.21.15.28:0/2895230765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faac015d5b0 0x7faac0141f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.914+0000 7faabe000640 1 -- 172.21.15.28:0/2895230765 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac0158450 con 0x7faac0146980 2024-04-26T20:58:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.914+0000 7faabf400640 1 --2- 172.21.15.28:0/2895230765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faac015d5b0 0x7faac0141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:54.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.914+0000 7faabe000640 1 --2- 172.21.15.28:0/2895230765 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faac0146980 0x7faac0139480 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7faaa802a180 tx=0x7faaa80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.914+0000 7faab6a00640 1 -- 172.21.15.28:0/2895230765 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8041020 con 0x7faac0146980 2024-04-26T20:58:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.914+0000 7faab6a00640 1 -- 172.21.15.28:0/2895230765 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa80306c0 con 0x7faac0146980 2024-04-26T20:58:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.914+0000 7faac5600640 1 -- 172.21.15.28:0/2895230765 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac0158730 con 0x7faac0146980 2024-04-26T20:58:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.914+0000 7faab6a00640 1 -- 172.21.15.28:0/2895230765 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8037040 con 0x7faac0146980 2024-04-26T20:58:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.915+0000 7faac5600640 1 -- 172.21.15.28:0/2895230765 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac01442f0 con 0x7faac0146980 2024-04-26T20:58:54.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.916+0000 7faac5600640 1 -- 172.21.15.28:0/2895230765 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faac006a400 con 0x7faac0146980 2024-04-26T20:58:54.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.917+0000 7faab6a00640 1 -- 172.21.15.28:0/2895230765 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faaa8044020 con 0x7faac0146980 2024-04-26T20:58:54.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.918+0000 7faab6a00640 1 --2- 172.21.15.28:0/2895230765 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7faa9c06c8f0 0x7faa9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:54.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.918+0000 7faab6a00640 1 -- 172.21.15.28:0/2895230765 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7faaa80ac440 con 0x7faac0146980 2024-04-26T20:58:54.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.918+0000 7faabea00640 1 --2- 172.21.15.28:0/2895230765 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7faa9c06c8f0 0x7faa9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:54.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.919+0000 7faabea00640 1 --2- 172.21.15.28:0/2895230765 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7faa9c06c8f0 0x7faa9c06edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faab0006fd0 tx=0x7faab0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:54.923 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:54.923+0000 7faab6a00640 1 -- 172.21.15.28:0/2895230765 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faaa804b030 con 0x7faac0146980 2024-04-26T20:58:55.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.187+0000 7faac5600640 1 -- 172.21.15.28:0/2895230765 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faac0072830 con 0x7faac0146980 2024-04-26T20:58:55.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.188+0000 7faab6a00640 1 -- 172.21.15.28:0/2895230765 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5184 (secure 0 0 0) 0x7faaa807a7d0 con 0x7faac0146980 2024-04-26T20:58:55.189 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T20:58:55.189 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/356870645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":356870645},{"type":"v1","addr":"172.21.15.28:6835","nonce":356870645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4829,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2957511478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":2957511478},{"type":"v1","addr":"172.21.15.103:6835","nonce":2957511478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/2060632176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2060632176},{"type":"v1","addr":"172.21.15.103:6834","nonce":2060632176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:58:43.534341+0000","modified":"2024-04-26T20:58:46.581816+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5571},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6837/2997047727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":2997047727},{"type":"v1","addr":"172.21.15.28:6837","nonce":2997047727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5571,"qdb_cluster":[5571]},"id":3}]} 2024-04-26T20:58:55.189 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 26 2024-04-26T20:58:55.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.191+0000 7faac5600640 1 -- 172.21.15.28:0/2895230765 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7faa9c06c8f0 msgr2=0x7faa9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:55.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.191+0000 7faac5600640 1 --2- 172.21.15.28:0/2895230765 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7faa9c06c8f0 0x7faa9c06edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faab0006fd0 tx=0x7faab0008040 comp rx=0 tx=0).stop 2024-04-26T20:58:55.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.191+0000 7faac5600640 1 -- 172.21.15.28:0/2895230765 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faac0146980 msgr2=0x7faac0139480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:55.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.191+0000 7faac5600640 1 --2- 172.21.15.28:0/2895230765 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faac0146980 0x7faac0139480 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7faaa802a180 tx=0x7faaa80046c0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.192 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.191+0000 7faac5600640 1 -- 172.21.15.28:0/2895230765 shutdown_connections 2024-04-26T20:58:55.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.191+0000 7faac5600640 1 --2- 172.21.15.28:0/2895230765 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7faa9c06c8f0 0x7faa9c06edb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.191+0000 7faac5600640 1 --2- 172.21.15.28:0/2895230765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faac015d5b0 0x7faac0141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.191+0000 7faac5600640 1 --2- 172.21.15.28:0/2895230765 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faac0146980 0x7faac0139480 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.191+0000 7faac5600640 1 --2- 172.21.15.28:0/2895230765 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7faac00740e0 0x7faac0157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.191+0000 7faac5600640 1 -- 172.21.15.28:0/2895230765 >> 172.21.15.28:0/2895230765 conn(0x7faac006e220 msgr2=0x7faac0160f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:55.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.191+0000 7faac5600640 1 -- 172.21.15.28:0/2895230765 shutdown_connections 2024-04-26T20:58:55.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.191+0000 7faac5600640 1 -- 172.21.15.28:0/2895230765 wait complete. 2024-04-26T20:58:55.616 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T20:58:43.534341+0000', 'modified': '2024-04-26T20:58:46.581816+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5571}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5571': {'gid': 5571, 'name': 'c', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.28:6837/2997047727', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.28:6836', 'nonce': 2997047727}, {'type': 'v1', 'addr': '172.21.15.28:6837', 'nonce': 2997047727}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5571, 'qdb_cluster': [5571]} 2024-04-26T20:58:55.616 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T20:58:55.616 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-26T20:58:55.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.836+0000 7f2fee200640 1 Processor -- start 2024-04-26T20:58:55.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.836+0000 7f2fee200640 1 -- start start 2024-04-26T20:58:55.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.838+0000 7f2fee200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2fe80d5540 0x7f2fe80b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.838+0000 7f2fee200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fe80d0710 con 0x7f2fe815f1b0 2024-04-26T20:58:55.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.838+0000 7f2fee200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fe80d4b40 con 0x7f2fe80d4700 2024-04-26T20:58:55.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.838+0000 7f2fee200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f2fe80cff00 con 0x7f2fe80d5540 2024-04-26T20:58:55.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.838+0000 7f2fece00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f2fe815f1b0 0x7f2fe80cf700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34160/0 (socket says 172.21.15.28:34160) 2024-04-26T20:58:55.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.838+0000 7f2fece00640 1 -- 172.21.15.28:0/3516565983 learned_addr learned my addr 172.21.15.28:0/3516565983 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:55.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.838+0000 7f2fed800640 1 --2- 172.21.15.28:0/3516565983 >> v2:172.21.15.28:3300/0 conn(0x7f2fe80d5540 0x7f2fe80b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:55.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.838+0000 7f2fed800640 1 -- 172.21.15.28:0/3516565983 >> v1:172.21.15.103:6789/0 conn(0x7f2fe80d4700 legacy=0x7f2fe80d4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:55.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.838+0000 7f2fed800640 1 -- 172.21.15.28:0/3516565983 >> v1:172.21.15.28:6789/0 conn(0x7f2fe815f1b0 legacy=0x7f2fe80cf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:55.839 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.838+0000 7f2fed800640 1 -- 172.21.15.28:0/3516565983 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe80d0150 con 0x7f2fe80d5540 2024-04-26T20:58:55.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.839+0000 7f2fed800640 1 --2- 172.21.15.28:0/3516565983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fe80d5540 0x7f2fe80b80b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009d90 tx=0x7f2fe002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9d05eddf6dc5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:55.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.840+0000 7f2fe7400640 1 -- 172.21.15.28:0/3516565983 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0007740 con 0x7f2fe80d5540 2024-04-26T20:58:55.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.840+0000 7f2fe7400640 1 -- 172.21.15.28:0/3516565983 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe00078e0 con 0x7f2fe80d5540 2024-04-26T20:58:55.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.840+0000 7f2fe7400640 1 -- 172.21.15.28:0/3516565983 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0007cb0 con 0x7f2fe80d5540 2024-04-26T20:58:55.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.840+0000 7f2fee200640 1 -- 172.21.15.28:0/3516565983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fe80d5540 msgr2=0x7f2fe80b80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:55.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.840+0000 7f2fee200640 1 --2- 172.21.15.28:0/3516565983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fe80d5540 0x7f2fe80b80b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009d90 tx=0x7f2fe002a730 comp rx=0 tx=0).stop 2024-04-26T20:58:55.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.841+0000 7f2fee200640 1 -- 172.21.15.28:0/3516565983 shutdown_connections 2024-04-26T20:58:55.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.841+0000 7f2fee200640 1 --2- 172.21.15.28:0/3516565983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fe80d5540 0x7f2fe80b80b0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.841+0000 7f2fee200640 1 -- 172.21.15.28:0/3516565983 >> 172.21.15.28:0/3516565983 conn(0x7f2fe806c770 msgr2=0x7f2fe806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:55.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.841+0000 7f2fee200640 1 -- 172.21.15.28:0/3516565983 shutdown_connections 2024-04-26T20:58:55.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.841+0000 7f2fee200640 1 -- 172.21.15.28:0/3516565983 wait complete. 2024-04-26T20:58:55.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.844+0000 7f2fee200640 1 Processor -- start 2024-04-26T20:58:55.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.844+0000 7f2fee200640 1 -- start start 2024-04-26T20:58:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.845+0000 7f2fee200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2fe80d4700 0x7f2fe80ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.845+0000 7f2fee200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2fe80d5540 0x7f2fe80ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.845+0000 7f2fee200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fe815f1b0 0x7f2fe815c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.845+0000 7f2fee200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe80d3f00 con 0x7f2fe815f1b0 2024-04-26T20:58:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.845+0000 7f2fee200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe80d4270 con 0x7f2fe80d4700 2024-04-26T20:58:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.845+0000 7f2fee200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe815f590 con 0x7f2fe80d5540 2024-04-26T20:58:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.845+0000 7f2fed800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fe815f1b0 0x7f2fe815c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.845+0000 7f2fed800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fe815f1b0 0x7f2fe815c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47126/0 (socket says 172.21.15.28:47126) 2024-04-26T20:58:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.845+0000 7f2fed800640 1 -- 172.21.15.28:0/3041359623 learned_addr learned my addr 172.21.15.28:0/3041359623 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.845+0000 7f2fece00640 1 --2- 172.21.15.28:0/3041359623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2fe80d4700 0x7f2fe80ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:55.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.846+0000 7f2fe7e00640 1 --2- 172.21.15.28:0/3041359623 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2fe80d5540 0x7f2fe80ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.846+0000 7f2fed800640 1 -- 172.21.15.28:0/3041359623 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2fe80d5540 msgr2=0x7f2fe80ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.846+0000 7f2fed800640 1 --2- 172.21.15.28:0/3041359623 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2fe80d5540 0x7f2fe80ab8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.846+0000 7f2fed800640 1 -- 172.21.15.28:0/3041359623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2fe80d4700 msgr2=0x7f2fe80ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.846+0000 7f2fed800640 1 --2- 172.21.15.28:0/3041359623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2fe80d4700 0x7f2fe80ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.846+0000 7f2fed800640 1 -- 172.21.15.28:0/3041359623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe8160830 con 0x7f2fe815f1b0 2024-04-26T20:58:55.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.846+0000 7f2fed800640 1 --2- 172.21.15.28:0/3041359623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fe815f1b0 0x7f2fe815c600 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009d90 tx=0x7f2fe0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.846+0000 7f2fe4c00640 1 -- 172.21.15.28:0/3041359623 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0040070 con 0x7f2fe815f1b0 2024-04-26T20:58:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.846+0000 7f2fe4c00640 1 -- 172.21.15.28:0/3041359623 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe0031c90 con 0x7f2fe815f1b0 2024-04-26T20:58:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.846+0000 7f2fe4c00640 1 -- 172.21.15.28:0/3041359623 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe003c640 con 0x7f2fe815f1b0 2024-04-26T20:58:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.846+0000 7f2fee200640 1 -- 172.21.15.28:0/3041359623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe8160ad0 con 0x7f2fe815f1b0 2024-04-26T20:58:55.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.846+0000 7f2fee200640 1 -- 172.21.15.28:0/3041359623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe8161090 con 0x7f2fe815f1b0 2024-04-26T20:58:55.849 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.849+0000 7f2fe4c00640 1 -- 172.21.15.28:0/3041359623 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2fe0039040 con 0x7f2fe815f1b0 2024-04-26T20:58:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.849+0000 7f2fe4c00640 1 --2- 172.21.15.28:0/3041359623 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2fc806c8f0 0x7f2fc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.849+0000 7f2fe4c00640 1 -- 172.21.15.28:0/3041359623 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f2fe00ace00 con 0x7f2fe815f1b0 2024-04-26T20:58:55.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.849+0000 7f2fece00640 1 --2- 172.21.15.28:0/3041359623 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2fc806c8f0 0x7f2fc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.850+0000 7f2ff27e0740 1 Processor -- start 2024-04-26T20:58:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.850+0000 7f2ff27e0740 1 -- start start 2024-04-26T20:58:55.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.850+0000 7f2fece00640 1 --2- 172.21.15.28:0/3041359623 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2fc806c8f0 0x7f2fc806edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc000f60 tx=0x7f2fdc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.851+0000 7f2ff27e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e2dfcf7b0 0x564e2dfd1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.851+0000 7f2ff27e0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2e00be80 0x564e2dfcfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.851+0000 7f2ff27e0740 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x564e2dffa270 0x564e2dffa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.851+0000 7f2ff27e0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x564e2df33600 con 0x564e2e00be80 2024-04-26T20:58:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.851+0000 7f2ff27e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564e2ded0100 con 0x564e2dfcf7b0 2024-04-26T20:58:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.851+0000 7f2ff27e0740 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x564e2df2f5c0 con 0x564e2dffa270 2024-04-26T20:58:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.851+0000 7f2fe7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2e00be80 0x564e2dfcfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.851+0000 7f2fe7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2e00be80 0x564e2dfcfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47132/0 (socket says 172.21.15.28:47132) 2024-04-26T20:58:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.851+0000 7f2fe7e00640 1 -- 172.21.15.28:0/10824011 learned_addr learned my addr 172.21.15.28:0/10824011 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.851+0000 7f2fece00640 1 --2- 172.21.15.28:0/10824011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e2dfcf7b0 0x564e2dfd1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.852+0000 7f2fed800640 1 --2- 172.21.15.28:0/10824011 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x564e2dffa270 0x564e2dffa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.852+0000 7f2fe7e00640 1 -- 172.21.15.28:0/10824011 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x564e2dffa270 msgr2=0x564e2dffa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.852+0000 7f2fe7e00640 1 --2- 172.21.15.28:0/10824011 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x564e2dffa270 0x564e2dffa6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.852+0000 7f2fe7e00640 1 -- 172.21.15.28:0/10824011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e2dfcf7b0 msgr2=0x564e2dfd1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:55.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.852+0000 7f2fe7e00640 1 --2- 172.21.15.28:0/10824011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e2dfcf7b0 0x564e2dfd1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.852+0000 7f2fe7e00640 1 -- 172.21.15.28:0/10824011 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e2dc452d0 con 0x564e2e00be80 2024-04-26T20:58:55.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.852+0000 7f2fe7e00640 1 --2- 172.21.15.28:0/10824011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2e00be80 0x564e2dfcfb90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400b760 tx=0x7f2fd400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a508a4b7a0cc3a04 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:55.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.852+0000 7f2fd9600640 1 -- 172.21.15.28:0/10824011 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400d8f0 con 0x564e2e00be80 2024-04-26T20:58:55.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.852+0000 7f2fd9600640 1 -- 172.21.15.28:0/10824011 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd4019070 con 0x564e2e00be80 2024-04-26T20:58:55.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.852+0000 7f2fd9600640 1 -- 172.21.15.28:0/10824011 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400dcd0 con 0x564e2e00be80 2024-04-26T20:58:55.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.853+0000 7f2ff27e0740 1 -- 172.21.15.28:0/10824011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2e00be80 msgr2=0x564e2dfcfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:55.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.853+0000 7f2ff27e0740 1 --2- 172.21.15.28:0/10824011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2e00be80 0x564e2dfcfb90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400b760 tx=0x7f2fd400bc20 comp rx=0 tx=0).stop 2024-04-26T20:58:55.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.853+0000 7f2ff27e0740 1 -- 172.21.15.28:0/10824011 shutdown_connections 2024-04-26T20:58:55.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.853+0000 7f2ff27e0740 1 --2- 172.21.15.28:0/10824011 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x564e2dffa270 0x564e2dffa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.853+0000 7f2ff27e0740 1 --2- 172.21.15.28:0/10824011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2e00be80 0x564e2dfcfb90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.853+0000 7f2ff27e0740 1 --2- 172.21.15.28:0/10824011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e2dfcf7b0 0x564e2dfd1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.853+0000 7f2ff27e0740 1 -- 172.21.15.28:0/10824011 >> 172.21.15.28:0/10824011 conn(0x564e2df63d10 msgr2=0x564e2e015270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:55.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.853+0000 7f2ff27e0740 1 -- 172.21.15.28:0/10824011 shutdown_connections 2024-04-26T20:58:55.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.853+0000 7f2ff27e0740 1 -- 172.21.15.28:0/10824011 wait complete. 2024-04-26T20:58:55.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.854+0000 7f2ff27e0740 1 -- start start 2024-04-26T20:58:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.854+0000 7f2ff27e0740 1 Processor -- start 2024-04-26T20:58:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.855+0000 7f2ff27e0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dfcf7b0 0x564e2dffbea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.855+0000 7f2ff27e0740 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x564e2dffa270 0x564e2e0177f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.855+0000 7f2ff27e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e2e00be80 0x564e2dfe97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.855+0000 7f2ff27e0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x564e2debaf70 con 0x564e2dfcf7b0 2024-04-26T20:58:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.856+0000 7f2ff27e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564e2dff7dc0 con 0x564e2e00be80 2024-04-26T20:58:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.856+0000 7f2ff27e0740 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x564e2dffcf90 con 0x564e2dffa270 2024-04-26T20:58:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.856+0000 7f2ff27e0740 10 client.0 _ll_register_callbacks cb 0x564e2df29480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:58:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.856+0000 7f2fece00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dfcf7b0 0x564e2dffbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.856+0000 7f2fece00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dfcf7b0 0x564e2dffbea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47140/0 (socket says 172.21.15.28:47140) 2024-04-26T20:58:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.856+0000 7f2fece00640 1 -- 172.21.15.28:0/3786108944 learned_addr learned my addr 172.21.15.28:0/3786108944 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.856+0000 7f2fece00640 1 -- 172.21.15.28:0/3786108944 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x564e2dffa270 msgr2=0x564e2e0177f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:55.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.856+0000 7f2fece00640 1 --2- 172.21.15.28:0/3786108944 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x564e2dffa270 0x564e2e0177f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.856+0000 7f2fece00640 1 -- 172.21.15.28:0/3786108944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e2e00be80 msgr2=0x564e2dfe97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:55.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.856+0000 7f2fece00640 1 --2- 172.21.15.28:0/3786108944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e2e00be80 0x564e2dfe97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.856+0000 7f2fece00640 1 -- 172.21.15.28:0/3786108944 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e2dd0f6b0 con 0x564e2dfcf7b0 2024-04-26T20:58:55.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.856+0000 7f2fece00640 1 --2- 172.21.15.28:0/3786108944 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dfcf7b0 0x564e2dffbea0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc007150 tx=0x7f2fdc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7604f9feabb9e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:55.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.856+0000 7f2fce000640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-04-26T20:58:55.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.857+0000 7f2fce000640 1 -- 172.21.15.28:0/3786108944 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc027060 con 0x564e2dfcf7b0 2024-04-26T20:58:55.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.857+0000 7f2fce000640 1 -- 172.21.15.28:0/3786108944 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f2fdc010040 con 0x564e2dfcf7b0 2024-04-26T20:58:55.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.857+0000 7f2ff27e0740 1 -- 172.21.15.28:0/3786108944 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fd4019070 con 0x564e2dfcf7b0 2024-04-26T20:58:55.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.857+0000 7f2fce000640 1 -- 172.21.15.28:0/3786108944 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2fdc00eab0 con 0x564e2dfcf7b0 2024-04-26T20:58:55.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.857+0000 7f2ff27e0740 10 client.5733 fetch_fsmap learned FSMap version 26 2024-04-26T20:58:55.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.857+0000 7f2ff27e0740 1 -- 172.21.15.28:0/3786108944 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x564e2e00d980 con 0x564e2dfcf7b0 2024-04-26T20:58:55.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.857+0000 7f2fce000640 1 -- 172.21.15.28:0/3786108944 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fd4019070 con 0x564e2dfcf7b0 2024-04-26T20:58:55.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.857+0000 7f2fce000640 1 -- 172.21.15.28:0/3786108944 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f2fdc013420 con 0x564e2dfcf7b0 2024-04-26T20:58:55.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.857+0000 7f2ff27e0740 10 client.5733 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T20:58:55.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.858+0000 7f2ff27e0740 10 client.5733 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T20:58:55.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.858+0000 7f2ff27e0740 20 client.5733 populate_metadata read hostname 'smithi028' 2024-04-26T20:58:55.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.858+0000 7f2ff27e0740 1 --2- 172.21.15.28:0/3786108944 >> [v2:172.21.15.28:6836/2997047727,v1:172.21.15.28:6837/2997047727] conn(0x564e2e0288c0 0x564e2e00cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.858+0000 7f2ff27e0740 4 client.5733 mds_command: new command op to 5571 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T20:58:55.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.858+0000 7f2ff27e0740 1 -- 172.21.15.28:0/3786108944 --> [v2:172.21.15.28:6836/2997047727,v1:172.21.15.28:6837/2997047727] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e2df41210 con 0x564e2e0288c0 2024-04-26T20:58:55.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.858+0000 7f2fe7e00640 1 --2- 172.21.15.28:0/3786108944 >> [v2:172.21.15.28:6836/2997047727,v1:172.21.15.28:6837/2997047727] conn(0x564e2e0288c0 0x564e2e00cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:55.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.859+0000 7f2fe7e00640 1 --2- 172.21.15.28:0/3786108944 >> [v2:172.21.15.28:6836/2997047727,v1:172.21.15.28:6837/2997047727] conn(0x564e2e0288c0 0x564e2e00cc20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41bd69b1edd7abd6 server_cookie=5a29442c2dd1dd72 in_seq=0 out_seq=0 2024-04-26T20:58:55.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.859+0000 7f2fce000640 10 client.5733 ms_handle_connect on v2:172.21.15.28:6836/2997047727 2024-04-26T20:58:55.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.859+0000 7f2fce000640 1 -- 172.21.15.28:0/3786108944 <== mds.0 v2:172.21.15.28:6836/2997047727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564e2df41210 con 0x564e2e0288c0 2024-04-26T20:58:55.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.859+0000 7f2fce000640 10 client.5733 handle_command_reply: tid=0 2024-04-26T20:58:55.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.860+0000 7f2ff27e0740 1 client.5733 shutdown 2024-04-26T20:58:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.860+0000 7f2ff27e0740 1 -- 172.21.15.28:0/3786108944 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dfcf7b0 msgr2=0x564e2dffbea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.860+0000 7f2ff27e0740 1 --2- 172.21.15.28:0/3786108944 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dfcf7b0 0x564e2dffbea0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc007150 tx=0x7f2fdc000bc0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.860+0000 7f2ff27e0740 1 -- 172.21.15.28:0/3786108944 shutdown_connections 2024-04-26T20:58:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.860+0000 7f2ff27e0740 1 --2- 172.21.15.28:0/3786108944 >> [v2:172.21.15.28:6836/2997047727,v1:172.21.15.28:6837/2997047727] conn(0x564e2e0288c0 0x564e2e00cc20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.860+0000 7f2ff27e0740 1 --2- 172.21.15.28:0/3786108944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e2e00be80 0x564e2dfe97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.860+0000 7f2ff27e0740 1 --2- 172.21.15.28:0/3786108944 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x564e2dffa270 0x564e2e0177f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.860+0000 7f2ff27e0740 1 --2- 172.21.15.28:0/3786108944 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dfcf7b0 0x564e2dffbea0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.860+0000 7f2fce000640 0 client.5733 ms_handle_reset on v2:172.21.15.28:6836/2997047727 2024-04-26T20:58:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.860+0000 7f2ff27e0740 1 -- 172.21.15.28:0/3786108944 >> 172.21.15.28:0/3786108944 conn(0x564e2df63d10 msgr2=0x564e2dff52e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.860+0000 7f2ff27e0740 1 -- 172.21.15.28:0/3786108944 shutdown_connections 2024-04-26T20:58:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.860+0000 7f2ff27e0740 1 -- 172.21.15.28:0/3786108944 wait complete. 2024-04-26T20:58:55.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.860+0000 7f2ff27e0740 20 client.5733 trim_cache size 0 max 16384 2024-04-26T20:58:55.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.881+0000 7f2ff27e0740 1 Processor -- start 2024-04-26T20:58:55.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.881+0000 7f2ff27e0740 1 -- start start 2024-04-26T20:58:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.882+0000 7f2ff27e0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dfcf7b0 0x564e2dffd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.882+0000 7f2ff27e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e2dffa270 0x564e2e00cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.882+0000 7f2ff27e0740 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x564e2e00be80 0x564e2e00d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.882+0000 7f2ff27e0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe8200810 con 0x564e2dfcf7b0 2024-04-26T20:58:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.882+0000 7f2ff27e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe81f5d20 con 0x564e2dffa270 2024-04-26T20:58:55.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.882+0000 7f2ff27e0740 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe8160120 con 0x564e2e00be80 2024-04-26T20:58:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.882+0000 7f2fece00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dfcf7b0 0x564e2dffd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.882+0000 7f2fece00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dfcf7b0 0x564e2dffd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47144/0 (socket says 172.21.15.28:47144) 2024-04-26T20:58:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.882+0000 7f2fece00640 1 -- 172.21.15.28:0/970178228 learned_addr learned my addr 172.21.15.28:0/970178228 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.883+0000 7f2fece00640 1 -- 172.21.15.28:0/970178228 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x564e2e00be80 msgr2=0x564e2e00d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.883+0000 7f2fece00640 1 --2- 172.21.15.28:0/970178228 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x564e2e00be80 0x564e2e00d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.883+0000 7f2fece00640 1 -- 172.21.15.28:0/970178228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e2dffa270 msgr2=0x564e2e00cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:55.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.883+0000 7f2fece00640 1 --2- 172.21.15.28:0/970178228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e2dffa270 0x564e2e00cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.883+0000 7f2fece00640 1 -- 172.21.15.28:0/970178228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e2dfcdb10 con 0x564e2dfcf7b0 2024-04-26T20:58:55.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.883+0000 7f2fece00640 1 --2- 172.21.15.28:0/970178228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dfcf7b0 0x564e2dffd620 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc001370 tx=0x7f2fdc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33261bbec93779fa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:55.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.883+0000 7f2fd8c00640 1 -- 172.21.15.28:0/970178228 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc013420 con 0x564e2dfcf7b0 2024-04-26T20:58:55.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.883+0000 7f2fd8c00640 1 -- 172.21.15.28:0/970178228 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fdc013970 con 0x564e2dfcf7b0 2024-04-26T20:58:55.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.883+0000 7f2fd8c00640 1 -- 172.21.15.28:0/970178228 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc013b60 con 0x564e2dfcf7b0 2024-04-26T20:58:55.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.884+0000 7f2ff27e0740 1 -- 172.21.15.28:0/970178228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dfcf7b0 msgr2=0x564e2dffd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:55.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.884+0000 7f2ff27e0740 1 --2- 172.21.15.28:0/970178228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dfcf7b0 0x564e2dffd620 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc001370 tx=0x7f2fdc01f000 comp rx=0 tx=0).stop 2024-04-26T20:58:55.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.884+0000 7f2ff27e0740 1 -- 172.21.15.28:0/970178228 shutdown_connections 2024-04-26T20:58:55.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.884+0000 7f2ff27e0740 1 --2- 172.21.15.28:0/970178228 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x564e2e00be80 0x564e2e00d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.884+0000 7f2ff27e0740 1 --2- 172.21.15.28:0/970178228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e2dffa270 0x564e2e00cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.884+0000 7f2ff27e0740 1 --2- 172.21.15.28:0/970178228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dfcf7b0 0x564e2dffd620 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.884+0000 7f2ff27e0740 1 -- 172.21.15.28:0/970178228 >> 172.21.15.28:0/970178228 conn(0x564e2df63d10 msgr2=0x564e2dffa8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:55.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.884+0000 7f2ff27e0740 1 -- 172.21.15.28:0/970178228 shutdown_connections 2024-04-26T20:58:55.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.884+0000 7f2ff27e0740 1 -- 172.21.15.28:0/970178228 wait complete. 2024-04-26T20:58:55.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.885+0000 7f2ff27e0740 1 -- start start 2024-04-26T20:58:55.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.885+0000 7f2ff27e0740 1 Processor -- start 2024-04-26T20:58:55.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.886+0000 7f2ff27e0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e2dfcf7b0 0x564e2dff5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.886+0000 7f2ff27e0740 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dffa270 0x564e2e081d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.886+0000 7f2ff27e0740 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x564e2e00be80 0x564e2dff6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.886+0000 7f2ff27e0740 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x564e2e028740 con 0x564e2dffa270 2024-04-26T20:58:55.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.886+0000 7f2ff27e0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564e2dffb300 con 0x564e2dfcf7b0 2024-04-26T20:58:55.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.886+0000 7f2ff27e0740 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x564e2e028da0 con 0x564e2e00be80 2024-04-26T20:58:55.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.886+0000 7f2ff27e0740 10 client.0 _ll_register_callbacks cb 0x7f2fdc005ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T20:58:55.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.886+0000 7f2fe7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dffa270 0x564e2e081d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:55.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.886+0000 7f2fe7e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dffa270 0x564e2e081d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47148/0 (socket says 172.21.15.28:47148) 2024-04-26T20:58:55.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.886+0000 7f2fe7e00640 1 -- 172.21.15.28:0/3730792407 learned_addr learned my addr 172.21.15.28:0/3730792407 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:55.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.887+0000 7f2fed800640 1 --2- 172.21.15.28:0/3730792407 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x564e2e00be80 0x564e2dff6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:55.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.887+0000 7f2fece00640 1 --2- 172.21.15.28:0/3730792407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e2dfcf7b0 0x564e2dff5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:55.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.887+0000 7f2fe7e00640 1 -- 172.21.15.28:0/3730792407 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x564e2e00be80 msgr2=0x564e2dff6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:55.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.887+0000 7f2fe7e00640 1 --2- 172.21.15.28:0/3730792407 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x564e2e00be80 0x564e2dff6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.887+0000 7f2fe7e00640 1 -- 172.21.15.28:0/3730792407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e2dfcf7b0 msgr2=0x564e2dff5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:55.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.887+0000 7f2fe7e00640 1 --2- 172.21.15.28:0/3730792407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e2dfcf7b0 0x564e2dff5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.887+0000 7f2fe7e00640 1 -- 172.21.15.28:0/3730792407 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e2e0822d0 con 0x564e2dffa270 2024-04-26T20:58:55.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.887+0000 7f2fed800640 1 --2- 172.21.15.28:0/3730792407 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x564e2e00be80 0x564e2dff6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:55.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.887+0000 7f2fe7e00640 1 --2- 172.21.15.28:0/3730792407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dffa270 0x564e2e081d90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400b760 tx=0x7f2fd4004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52976396d7ba69f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:55.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.887+0000 7f2fce000640 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2024-04-26T20:58:55.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.887+0000 7f2fce000640 1 -- 172.21.15.28:0/3730792407 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fd40044d0 con 0x564e2dffa270 2024-04-26T20:58:55.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.887+0000 7f2fce000640 1 -- 172.21.15.28:0/3730792407 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f2fd400d990 con 0x564e2dffa270 2024-04-26T20:58:55.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.887+0000 7f2ff27e0740 1 -- 172.21.15.28:0/3730792407 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fdc013970 con 0x564e2dffa270 2024-04-26T20:58:55.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.888+0000 7f2fce000640 1 -- 172.21.15.28:0/3730792407 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2fd400db90 con 0x564e2dffa270 2024-04-26T20:58:55.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.888+0000 7f2ff27e0740 10 client.5739 fetch_fsmap learned FSMap version 26 2024-04-26T20:58:55.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.888+0000 7f2ff27e0740 1 -- 172.21.15.28:0/3730792407 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x564e2e08f380 con 0x564e2dffa270 2024-04-26T20:58:55.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.888+0000 7f2fce000640 1 -- 172.21.15.28:0/3730792407 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fdc013970 con 0x564e2dffa270 2024-04-26T20:58:55.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.888+0000 7f2fce000640 1 -- 172.21.15.28:0/3730792407 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 26) ==== 3494+0+0 (secure 0 0 0) 0x7f2fd4013070 con 0x564e2dffa270 2024-04-26T20:58:55.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.888+0000 7f2ff27e0740 10 client.5739 fetch_fsmap finished waiting for FSMap version 26 2024-04-26T20:58:55.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.888+0000 7f2ff27e0740 10 client.5739 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T20:58:55.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.888+0000 7f2ff27e0740 20 client.5739 populate_metadata read hostname 'smithi028' 2024-04-26T20:58:55.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.888+0000 7f2ff27e0740 1 --2- 172.21.15.28:0/3730792407 >> [v2:172.21.15.28:6836/2997047727,v1:172.21.15.28:6837/2997047727] conn(0x564e2e00ac00 0x564e2e00afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:55.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.888+0000 7f2ff27e0740 4 client.5739 mds_command: new command op to 5571 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T20:58:55.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.888+0000 7f2ff27e0740 1 -- 172.21.15.28:0/3730792407 --> [v2:172.21.15.28:6836/2997047727,v1:172.21.15.28:6837/2997047727] -- command(tid 0: {"prefix": "status"}) -- 0x564e2df3b080 con 0x564e2e00ac00 2024-04-26T20:58:55.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.889+0000 7f2fece00640 1 --2- 172.21.15.28:0/3730792407 >> [v2:172.21.15.28:6836/2997047727,v1:172.21.15.28:6837/2997047727] conn(0x564e2e00ac00 0x564e2e00afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:55.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.889+0000 7f2fece00640 1 --2- 172.21.15.28:0/3730792407 >> [v2:172.21.15.28:6836/2997047727,v1:172.21.15.28:6837/2997047727] conn(0x564e2e00ac00 0x564e2e00afe0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32d8801c7c245531 server_cookie=d827dd46e1c4b9bc in_seq=0 out_seq=0 2024-04-26T20:58:55.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.889+0000 7f2fce000640 10 client.5739 ms_handle_connect on v2:172.21.15.28:6836/2997047727 2024-04-26T20:58:55.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.889+0000 7f2fce000640 1 -- 172.21.15.28:0/3730792407 <== mds.0 v2:172.21.15.28:6836/2997047727 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x564e2df3b080 con 0x564e2e00ac00 2024-04-26T20:58:55.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.889+0000 7f2fce000640 10 client.5739 handle_command_reply: tid=0 2024-04-26T20:58:55.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.889+0000 7f2ff27e0740 1 client.5739 shutdown 2024-04-26T20:58:55.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.890+0000 7f2ff27e0740 1 -- 172.21.15.28:0/3730792407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dffa270 msgr2=0x564e2e081d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:55.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.890+0000 7f2ff27e0740 1 --2- 172.21.15.28:0/3730792407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dffa270 0x564e2e081d90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400b760 tx=0x7f2fd4004680 comp rx=0 tx=0).stop 2024-04-26T20:58:55.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.890+0000 7f2ff27e0740 1 -- 172.21.15.28:0/3730792407 shutdown_connections 2024-04-26T20:58:55.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.890+0000 7f2ff27e0740 1 --2- 172.21.15.28:0/3730792407 >> [v2:172.21.15.28:6836/2997047727,v1:172.21.15.28:6837/2997047727] conn(0x564e2e00ac00 0x564e2e00afe0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.890+0000 7f2ff27e0740 1 --2- 172.21.15.28:0/3730792407 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x564e2e00be80 0x564e2dff6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.890+0000 7f2ff27e0740 1 --2- 172.21.15.28:0/3730792407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564e2dffa270 0x564e2e081d90 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.890+0000 7f2ff27e0740 1 --2- 172.21.15.28:0/3730792407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e2dfcf7b0 0x564e2dff5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.890+0000 7f2ff27e0740 1 -- 172.21.15.28:0/3730792407 >> 172.21.15.28:0/3730792407 conn(0x564e2df63d10 msgr2=0x564e2e027990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:55.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.890+0000 7f2fce000640 0 client.5739 ms_handle_reset on v2:172.21.15.28:6836/2997047727 2024-04-26T20:58:55.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.890+0000 7f2ff27e0740 1 -- 172.21.15.28:0/3730792407 shutdown_connections 2024-04-26T20:58:55.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.890+0000 7f2ff27e0740 1 -- 172.21.15.28:0/3730792407 wait complete. 2024-04-26T20:58:55.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.890+0000 7f2ff27e0740 20 client.5739 trim_cache size 0 max 16384 2024-04-26T20:58:55.894 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T20:58:55.894 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "79845bd7-2c94-4fa6-8082-292285d530c9", 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 5571, 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 11.853696365999999, 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 26, 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 45, 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 45, 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 17.229209544 2024-04-26T20:58:55.895 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.891+0000 7f2fee200640 1 -- 172.21.15.28:0/3041359623 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2fc806c8f0 msgr2=0x7f2fc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.891+0000 7f2fee200640 1 --2- 172.21.15.28:0/3041359623 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2fc806c8f0 0x7f2fc806edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc000f60 tx=0x7f2fdc009040 comp rx=0 tx=0).stop 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.891+0000 7f2fee200640 1 -- 172.21.15.28:0/3041359623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fe815f1b0 msgr2=0x7f2fe815c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.891+0000 7f2fee200640 1 --2- 172.21.15.28:0/3041359623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fe815f1b0 0x7f2fe815c600 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009d90 tx=0x7f2fe0004a40 comp rx=0 tx=0).stop 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.891+0000 7f2fee200640 1 -- 172.21.15.28:0/3041359623 shutdown_connections 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.891+0000 7f2fee200640 1 --2- 172.21.15.28:0/3041359623 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2fc806c8f0 0x7f2fc806edb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.891+0000 7f2fee200640 1 --2- 172.21.15.28:0/3041359623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2fe815f1b0 0x7f2fe815c600 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.891+0000 7f2fee200640 1 --2- 172.21.15.28:0/3041359623 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2fe80d5540 0x7f2fe80ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.891+0000 7f2fee200640 1 --2- 172.21.15.28:0/3041359623 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2fe80d4700 0x7f2fe80ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:55.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.891+0000 7f2fee200640 1 -- 172.21.15.28:0/3041359623 >> 172.21.15.28:0/3041359623 conn(0x7f2fe806c770 msgr2=0x7f2fe80b82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:55.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.891+0000 7f2fee200640 1 -- 172.21.15.28:0/3041359623 shutdown_connections 2024-04-26T20:58:55.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:55.891+0000 7f2fee200640 1 -- 172.21.15.28:0/3041359623 wait complete. 2024-04-26T20:58:55.909 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T20:58:56.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.093+0000 7f9869c00640 1 Processor -- start 2024-04-26T20:58:56.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.093+0000 7f9869c00640 1 -- start start 2024-04-26T20:58:56.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9869c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f986415dc90 0x7f9864161650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:56.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9869c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f986415e160 con 0x7f98640721f0 2024-04-26T20:58:56.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9869c00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f986415e440 con 0x7f98641576e0 2024-04-26T20:58:56.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9869c00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9864157ac0 con 0x7f986415dc90 2024-04-26T20:58:56.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9862000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f98640721f0 0x7f986415d580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34172/0 (socket says 172.21.15.28:34172) 2024-04-26T20:58:56.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9862000640 1 -- 172.21.15.28:0/449974395 learned_addr learned my addr 172.21.15.28:0/449974395 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T20:58:56.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9863400640 1 --2- 172.21.15.28:0/449974395 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f986415dc90 0x7f9864161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:56.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9861600640 1 -- 172.21.15.28:0/449974395 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3109930220 0 0) 0x7f986415e160 con 0x7f98640721f0 2024-04-26T20:58:56.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9861600640 1 -- 172.21.15.28:0/449974395 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9848003520 con 0x7f98640721f0 2024-04-26T20:58:56.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9861600640 1 -- 172.21.15.28:0/449974395 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3716690046 0 0) 0x7f9848003520 con 0x7f98640721f0 2024-04-26T20:58:56.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9861600640 1 -- 172.21.15.28:0/449974395 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f986415dc90 msgr2=0x7f9864161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:56.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9861600640 1 --2- 172.21.15.28:0/449974395 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f986415dc90 0x7f9864161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:56.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9861600640 1 -- 172.21.15.28:0/449974395 >> v1:172.21.15.103:6789/0 conn(0x7f98641576e0 legacy=0x7f9864071ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:56.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9861600640 1 -- 172.21.15.28:0/449974395 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9864160b70 con 0x7f98640721f0 2024-04-26T20:58:56.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9861600640 1 -- 172.21.15.28:0/449974395 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f984c0019f0 con 0x7f98640721f0 2024-04-26T20:58:56.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9861600640 1 -- 172.21.15.28:0/449974395 >> v1:172.21.15.28:6789/0 conn(0x7f98640721f0 legacy=0x7f986415d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:56.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9861600640 1 --2- 172.21.15.28:0/449974395 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f986415dc90 0x7f98480039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:56.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9861600640 1 --2- 172.21.15.28:0/449974395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9848003fb0 0x7f9848004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:56.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9861600640 1 --2- 172.21.15.28:0/449974395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848004d40 0x7f98480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:56.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9861600640 1 -- 172.21.15.28:0/449974395 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9864157ac0 con 0x7f9848004d40 2024-04-26T20:58:56.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9861600640 1 -- 172.21.15.28:0/449974395 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f984800bb00 con 0x7f9848003fb0 2024-04-26T20:58:56.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.096+0000 7f9861600640 1 -- 172.21.15.28:0/449974395 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f984800bd00 con 0x7f986415dc90 2024-04-26T20:58:56.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.097+0000 7f9863400640 1 --2- 172.21.15.28:0/449974395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848004d40 0x7f98480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:56.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.097+0000 7f9862a00640 1 --2- 172.21.15.28:0/449974395 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f986415dc90 0x7f98480039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:56.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.097+0000 7f9862000640 1 --2- 172.21.15.28:0/449974395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9848003fb0 0x7f9848004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:56.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.097+0000 7f9863400640 1 -- 172.21.15.28:0/449974395 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f986415dc90 msgr2=0x7f98480039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:56.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.097+0000 7f9863400640 1 --2- 172.21.15.28:0/449974395 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f986415dc90 0x7f98480039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:56.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.097+0000 7f9863400640 1 -- 172.21.15.28:0/449974395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9848003fb0 msgr2=0x7f9848004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T20:58:56.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.097+0000 7f9863400640 1 --2- 172.21.15.28:0/449974395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9848003fb0 0x7f9848004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:56.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.097+0000 7f9863400640 1 -- 172.21.15.28:0/449974395 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9848003520 con 0x7f9848004d40 2024-04-26T20:58:56.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.097+0000 7f9862000640 1 --2- 172.21.15.28:0/449974395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9848003fb0 0x7f9848004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T20:58:56.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.098+0000 7f9863400640 1 --2- 172.21.15.28:0/449974395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848004d40 0x7f98480051c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f985c009a90 tx=0x7f9848005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b634c682acd932f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:56.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.098+0000 7f9861600640 1 -- 172.21.15.28:0/449974395 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c036070 con 0x7f9848004d40 2024-04-26T20:58:56.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.098+0000 7f9861600640 1 -- 172.21.15.28:0/449974395 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f985c031400 con 0x7f9848004d40 2024-04-26T20:58:56.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.098+0000 7f9861600640 1 -- 172.21.15.28:0/449974395 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c009690 con 0x7f9848004d40 2024-04-26T20:58:56.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.099+0000 7f9869c00640 1 -- 172.21.15.28:0/449974395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848004d40 msgr2=0x7f98480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:56.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.099+0000 7f9869c00640 1 --2- 172.21.15.28:0/449974395 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848004d40 0x7f98480051c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f985c009a90 tx=0x7f9848005830 comp rx=0 tx=0).stop 2024-04-26T20:58:56.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.099+0000 7f9863400640 1 -- 172.21.15.28:0/449974395 reap_dead start 2024-04-26T20:58:56.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.099+0000 7f9869c00640 1 -- 172.21.15.28:0/449974395 shutdown_connections 2024-04-26T20:58:56.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.099+0000 7f9869c00640 1 -- 172.21.15.28:0/449974395 >> 172.21.15.28:0/449974395 conn(0x7f98640b4bf0 msgr2=0x7f98640b4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:56.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.099+0000 7f9869c00640 1 -- 172.21.15.28:0/449974395 shutdown_connections 2024-04-26T20:58:56.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.099+0000 7f9869c00640 1 -- 172.21.15.28:0/449974395 wait complete. 2024-04-26T20:58:56.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.101+0000 7f9869c00640 1 Processor -- start 2024-04-26T20:58:56.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.101+0000 7f9869c00640 1 -- start start 2024-04-26T20:58:56.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.103+0000 7f9869c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848004d40 0x7f98641596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:56.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.103+0000 7f9869c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9864159c30 0x7f986415a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:56.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.103+0000 7f9869c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f98640cc270 0x7f98640cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:56.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.103+0000 7f9869c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f98640cfbd0 con 0x7f9848004d40 2024-04-26T20:58:56.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.103+0000 7f9869c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9864161130 con 0x7f9864159c30 2024-04-26T20:58:56.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.103+0000 7f9869c00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9864161650 con 0x7f98640cc270 2024-04-26T20:58:56.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.103+0000 7f9862a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848004d40 0x7f98641596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:56.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.103+0000 7f9862a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848004d40 0x7f98641596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47168/0 (socket says 172.21.15.28:47168) 2024-04-26T20:58:56.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.103+0000 7f9862a00640 1 -- 172.21.15.28:0/386242587 learned_addr learned my addr 172.21.15.28:0/386242587 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T20:58:56.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.103+0000 7f9862a00640 1 -- 172.21.15.28:0/386242587 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f98640cc270 msgr2=0x7f98640cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:56.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.103+0000 7f9862a00640 1 --2- 172.21.15.28:0/386242587 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f98640cc270 0x7f98640cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:56.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.103+0000 7f9862a00640 1 -- 172.21.15.28:0/386242587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9864159c30 msgr2=0x7f986415a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:56.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.103+0000 7f9862a00640 1 --2- 172.21.15.28:0/386242587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9864159c30 0x7f986415a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:56.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.103+0000 7f9862a00640 1 -- 172.21.15.28:0/386242587 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98640ce940 con 0x7f9848004d40 2024-04-26T20:58:56.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.103+0000 7f9862a00640 1 --2- 172.21.15.28:0/386242587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848004d40 0x7f98641596f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f985400e550 tx=0x7f985400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:56.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.103+0000 7f985aa00640 1 -- 172.21.15.28:0/386242587 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98540052e0 con 0x7f9848004d40 2024-04-26T20:58:56.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.103+0000 7f985aa00640 1 -- 172.21.15.28:0/386242587 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9854018070 con 0x7f9848004d40 2024-04-26T20:58:56.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.103+0000 7f985aa00640 1 -- 172.21.15.28:0/386242587 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9854003810 con 0x7f9848004d40 2024-04-26T20:58:56.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.103+0000 7f9869c00640 1 -- 172.21.15.28:0/386242587 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98640cec20 con 0x7f9848004d40 2024-04-26T20:58:56.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.103+0000 7f9869c00640 1 -- 172.21.15.28:0/386242587 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98641591d0 con 0x7f9848004d40 2024-04-26T20:58:56.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.106+0000 7f9869c00640 1 -- 172.21.15.28:0/386242587 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f986406a400 con 0x7f9848004d40 2024-04-26T20:58:56.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.106+0000 7f985aa00640 1 -- 172.21.15.28:0/386242587 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98540039b0 con 0x7f9848004d40 2024-04-26T20:58:56.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.106+0000 7f985aa00640 1 --2- 172.21.15.28:0/386242587 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f984006c920 0x7f984006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T20:58:56.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.107+0000 7f985aa00640 1 -- 172.21.15.28:0/386242587 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f985408b740 con 0x7f9848004d40 2024-04-26T20:58:56.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.107+0000 7f9862000640 1 --2- 172.21.15.28:0/386242587 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f984006c920 0x7f984006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T20:58:56.108 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.107+0000 7f9862000640 1 --2- 172.21.15.28:0/386242587 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f984006c920 0x7f984006ede0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f984c007c40 tx=0x7f984c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T20:58:56.112 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.111+0000 7f985aa00640 1 -- 172.21.15.28:0/386242587 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9854059c00 con 0x7f9848004d40 2024-04-26T20:58:56.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.364+0000 7f9869c00640 1 -- 172.21.15.28:0/386242587 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9864071d90 con 0x7f9848004d40 2024-04-26T20:58:56.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.364+0000 7f985aa00640 1 -- 172.21.15.28:0/386242587 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f9854059500 con 0x7f9848004d40 2024-04-26T20:58:56.365 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T20:58:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.367+0000 7f9869c00640 1 -- 172.21.15.28:0/386242587 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f984006c920 msgr2=0x7f984006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.367+0000 7f9869c00640 1 --2- 172.21.15.28:0/386242587 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f984006c920 0x7f984006ede0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f984c007c40 tx=0x7f984c009040 comp rx=0 tx=0).stop 2024-04-26T20:58:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.367+0000 7f9869c00640 1 -- 172.21.15.28:0/386242587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848004d40 msgr2=0x7f98641596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T20:58:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.367+0000 7f9869c00640 1 --2- 172.21.15.28:0/386242587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848004d40 0x7f98641596f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f985400e550 tx=0x7f985400ea10 comp rx=0 tx=0).stop 2024-04-26T20:58:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.367+0000 7f9869c00640 1 -- 172.21.15.28:0/386242587 shutdown_connections 2024-04-26T20:58:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.367+0000 7f9869c00640 1 --2- 172.21.15.28:0/386242587 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f984006c920 0x7f984006ede0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.367+0000 7f9869c00640 1 --2- 172.21.15.28:0/386242587 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f98640cc270 0x7f98640cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.367+0000 7f9869c00640 1 --2- 172.21.15.28:0/386242587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9864159c30 0x7f986415a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.367+0000 7f9869c00640 1 --2- 172.21.15.28:0/386242587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848004d40 0x7f98641596f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T20:58:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.367+0000 7f9869c00640 1 -- 172.21.15.28:0/386242587 >> 172.21.15.28:0/386242587 conn(0x7f98640b4bf0 msgr2=0x7f986415b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T20:58:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.368+0000 7f9869c00640 1 -- 172.21.15.28:0/386242587 shutdown_connections 2024-04-26T20:58:56.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56.368+0000 7f9869c00640 1 -- 172.21.15.28:0/386242587 wait complete. 2024-04-26T20:58:56.385 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T20:58:56.385 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T20:58:56.385 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2024-04-26T20:58:56.385 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T20:58:56.386 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T20:58:56.386 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T20:58:56.386 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T20:58:56.386 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T20:58:56.386 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:58:56.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:56.386 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2024-04-26T20:58:56.440 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T20:58:56.440 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T20:58:56.441 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T20:58:56.441 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:58:56.441 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2024-04-26T20:58:56.441 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:58:56.441 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:58:56.441 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:58:56.441 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f0 2024-04-26T20:58:56.441 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:58:56.441 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:58:56.441 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp7s0f1 2024-04-26T20:58:56.441 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T20:58:56.441 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2024-04-26T20:58:56.441 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f0 2024-04-26T20:58:56.441 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T20:58:56.442 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42500sec preferred_lft 42500sec 2024-04-26T20:58:56.442 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link noprefixroute 2024-04-26T20:58:56.442 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:58:56.442 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T20:58:56.442 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2024-04-26T20:58:56.442 INFO:teuthology.orchestra.run.smithi028.stdout: altname enp1s0f1 2024-04-26T20:58:56.442 INFO:teuthology.orchestra.run.smithi028.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T20:58:56.442 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 4a:2a:43:26:cb:b2 brd ff:ff:ff:ff:ff:ff 2024-04-26T20:58:56.442 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T20:58:56.442 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:58:56.442 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::608e:e5ff:fecb:cfa2/64 scope link 2024-04-26T20:58:56.442 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:58:56.442 INFO:teuthology.orchestra.run.smithi028.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:58:56.442 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 4a:2a:43:26:cb:b2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:58:56.443 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::482a:43ff:fe26:cbb2/64 scope link 2024-04-26T20:58:56.443 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:58:56.443 INFO:teuthology.orchestra.run.smithi028.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T20:58:56.443 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 56:33:e8:86:4a:fb brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:58:56.443 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::5433:e8ff:fe86:4afb/64 scope link 2024-04-26T20:58:56.443 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2024-04-26T20:58:56.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:56.444 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T20:58:56.520 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-04-26T20:58:56.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:56.525 DEBUG:teuthology.orchestra.run.smithi028:> route 2024-04-26T20:58:56.576 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2024-04-26T20:58:56.576 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T20:58:56.576 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T20:58:56.576 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T20:58:56.576 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T20:58:56.577 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T20:58:56.578 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2024-04-26T20:58:56.578 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T20:58:56.578 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T20:58:56.578 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T20:58:56.578 DEBUG:teuthology.orchestra.run.smithi028:> ') 2024-04-26T20:58:56.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T20:58:56.794 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:58:56.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:56.804 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-04-26T20:58:56.854 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T20:58:56.854 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T20:58:56.855 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:58:56.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:56.855 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2024-04-26T20:58:56.944 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-26T20:58:56.944 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:58:56.944 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T20:58:57.003 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T20:58:57.003 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T20:58:57.003 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:58:57.061 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2024-04-26T20:58:57.145 DEBUG:teuthology.orchestra.run.smithi028:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:58:57.202 INFO:teuthology.orchestra.run.smithi028.stdout:/proc 2024-04-26T20:58:57.202 INFO:teuthology.orchestra.run.smithi028.stdout:/sys 2024-04-26T20:58:57.202 INFO:teuthology.orchestra.run.smithi028.stdout:/dev 2024-04-26T20:58:57.202 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/security 2024-04-26T20:58:57.202 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/shm 2024-04-26T20:58:57.202 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/pts 2024-04-26T20:58:57.203 INFO:teuthology.orchestra.run.smithi028.stdout:/run 2024-04-26T20:58:57.203 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/cgroup 2024-04-26T20:58:57.203 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/pstore 2024-04-26T20:58:57.203 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/bpf 2024-04-26T20:58:57.203 INFO:teuthology.orchestra.run.smithi028.stdout:/ 2024-04-26T20:58:57.203 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/selinux 2024-04-26T20:58:57.203 INFO:teuthology.orchestra.run.smithi028.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:58:57.203 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/mqueue 2024-04-26T20:58:57.203 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/hugepages 2024-04-26T20:58:57.203 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/debug 2024-04-26T20:58:57.203 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/tracing 2024-04-26T20:58:57.203 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/fuse/connections 2024-04-26T20:58:57.203 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:58:57.203 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/config 2024-04-26T20:58:57.203 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:58:57.204 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:58:57.204 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:58:57.204 INFO:teuthology.orchestra.run.smithi028.stdout:/run/user/1000 2024-04-26T20:58:57.204 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph 2024-04-26T20:58:57.204 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:58:57.204 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:58:57.204 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:58:57.204 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:58:57.204 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns 2024-04-26T20:58:57.204 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:58:57.204 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:58:57.204 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:58:57.204 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:58:57.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:57.205 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2024-04-26T20:58:57.260 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T20:58:57.260 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-26T20:58:57.265 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2024-04-26T20:58:57.337 DEBUG:teuthology.orchestra.run.smithi028:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:58:57.383 INFO:teuthology.orchestra.run.smithi028.stdout:/proc 2024-04-26T20:58:57.383 INFO:teuthology.orchestra.run.smithi028.stdout:/sys 2024-04-26T20:58:57.383 INFO:teuthology.orchestra.run.smithi028.stdout:/dev 2024-04-26T20:58:57.383 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/security 2024-04-26T20:58:57.383 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/shm 2024-04-26T20:58:57.383 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/pts 2024-04-26T20:58:57.383 INFO:teuthology.orchestra.run.smithi028.stdout:/run 2024-04-26T20:58:57.383 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/cgroup 2024-04-26T20:58:57.384 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/pstore 2024-04-26T20:58:57.384 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/bpf 2024-04-26T20:58:57.384 INFO:teuthology.orchestra.run.smithi028.stdout:/ 2024-04-26T20:58:57.384 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/selinux 2024-04-26T20:58:57.384 INFO:teuthology.orchestra.run.smithi028.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:58:57.384 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/mqueue 2024-04-26T20:58:57.384 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/hugepages 2024-04-26T20:58:57.384 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/debug 2024-04-26T20:58:57.384 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/tracing 2024-04-26T20:58:57.384 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/fuse/connections 2024-04-26T20:58:57.384 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:58:57.384 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/config 2024-04-26T20:58:57.384 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:58:57.384 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:58:57.385 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:58:57.385 INFO:teuthology.orchestra.run.smithi028.stdout:/run/user/1000 2024-04-26T20:58:57.385 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph 2024-04-26T20:58:57.385 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:58:57.385 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:58:57.385 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:58:57.385 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:58:57.385 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns 2024-04-26T20:58:57.385 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:58:57.385 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:58:57.385 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:58:57.385 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:58:57.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:57.386 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2024-04-26T20:58:57.450 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:2024-04-26T20:58:57.450+0000 7fd0a0b94480 -1 init, newargv = 0x55c7282e48d0 newargc=15 2024-04-26T20:58:57.450 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:ceph-fuse[48828]: starting ceph client 2024-04-26T20:58:57.461 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:ceph-fuse[48828]: starting fuse 2024-04-26T20:58:58.534 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2024-04-26T20:58:58.637 DEBUG:teuthology.orchestra.run.smithi028:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T20:58:58.698 INFO:teuthology.orchestra.run.smithi028.stdout:/proc 2024-04-26T20:58:58.698 INFO:teuthology.orchestra.run.smithi028.stdout:/sys 2024-04-26T20:58:58.698 INFO:teuthology.orchestra.run.smithi028.stdout:/dev 2024-04-26T20:58:58.698 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/security 2024-04-26T20:58:58.698 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/shm 2024-04-26T20:58:58.698 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/pts 2024-04-26T20:58:58.698 INFO:teuthology.orchestra.run.smithi028.stdout:/run 2024-04-26T20:58:58.698 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/cgroup 2024-04-26T20:58:58.698 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/pstore 2024-04-26T20:58:58.698 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/bpf 2024-04-26T20:58:58.698 INFO:teuthology.orchestra.run.smithi028.stdout:/ 2024-04-26T20:58:58.698 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/selinux 2024-04-26T20:58:58.699 INFO:teuthology.orchestra.run.smithi028.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T20:58:58.699 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/mqueue 2024-04-26T20:58:58.699 INFO:teuthology.orchestra.run.smithi028.stdout:/dev/hugepages 2024-04-26T20:58:58.699 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/debug 2024-04-26T20:58:58.699 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/tracing 2024-04-26T20:58:58.699 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/fs/fuse/connections 2024-04-26T20:58:58.699 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T20:58:58.699 INFO:teuthology.orchestra.run.smithi028.stdout:/sys/kernel/config 2024-04-26T20:58:58.699 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T20:58:58.699 INFO:teuthology.orchestra.run.smithi028.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T20:58:58.699 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T20:58:58.699 INFO:teuthology.orchestra.run.smithi028.stdout:/run/user/1000 2024-04-26T20:58:58.699 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph 2024-04-26T20:58:58.699 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T20:58:58.699 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T20:58:58.699 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T20:58:58.699 INFO:teuthology.orchestra.run.smithi028.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T20:58:58.700 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns 2024-04-26T20:58:58.700 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:58:58.700 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T20:58:58.700 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:58:58.700 INFO:teuthology.orchestra.run.smithi028.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T20:58:58.700 INFO:teuthology.orchestra.run.smithi028.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-26T20:58:58.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:58.701 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2024-04-26T20:58:58.758 INFO:teuthology.orchestra.run.smithi028.stdout:41 2024-04-26T20:58:58.758 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-26T20:58:58.758 DEBUG:teuthology.orchestra.run.smithi028:> sudo stdin-killer -- python3 -c ' 2024-04-26T20:58:58.758 DEBUG:teuthology.orchestra.run.smithi028:> import glob 2024-04-26T20:58:58.758 DEBUG:teuthology.orchestra.run.smithi028:> import re 2024-04-26T20:58:58.758 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:58:58.758 DEBUG:teuthology.orchestra.run.smithi028:> import subprocess 2024-04-26T20:58:58.759 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:58:58.759 DEBUG:teuthology.orchestra.run.smithi028:> def _find_admin_socket(client_name): 2024-04-26T20:58:58.759 DEBUG:teuthology.orchestra.run.smithi028:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-26T20:58:58.759 DEBUG:teuthology.orchestra.run.smithi028:> files = glob.glob(asok_path) 2024-04-26T20:58:58.759 DEBUG:teuthology.orchestra.run.smithi028:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-26T20:58:58.759 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:58:58.759 DEBUG:teuthology.orchestra.run.smithi028:> # Given a non-glob path, it better be there 2024-04-26T20:58:58.759 DEBUG:teuthology.orchestra.run.smithi028:> if "*" not in asok_path: 2024-04-26T20:58:58.759 DEBUG:teuthology.orchestra.run.smithi028:> assert(len(files) == 1) 2024-04-26T20:58:58.759 DEBUG:teuthology.orchestra.run.smithi028:> return files[0] 2024-04-26T20:58:58.759 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:58:58.759 DEBUG:teuthology.orchestra.run.smithi028:> for f in files: 2024-04-26T20:58:58.759 DEBUG:teuthology.orchestra.run.smithi028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T20:58:58.759 DEBUG:teuthology.orchestra.run.smithi028:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T20:58:58.759 DEBUG:teuthology.orchestra.run.smithi028:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T20:58:58.759 DEBUG:teuthology.orchestra.run.smithi028:> contents = proc_f.read() 2024-04-26T20:58:58.759 DEBUG:teuthology.orchestra.run.smithi028:> if mountpoint in contents: 2024-04-26T20:58:58.760 DEBUG:teuthology.orchestra.run.smithi028:> return f 2024-04-26T20:58:58.760 DEBUG:teuthology.orchestra.run.smithi028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T20:58:58.760 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:58:58.760 DEBUG:teuthology.orchestra.run.smithi028:> print(_find_admin_socket("client.0")) 2024-04-26T20:58:58.760 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:58:58.932 INFO:teuthology.orchestra.run.smithi028.stdout:/var/run/ceph/ceph-client.0.48828.asok 2024-04-26T20:58:58.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:58:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:58:58.951 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48828.asok 2024-04-26T20:58:58.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:58.951 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48828.asok status 2024-04-26T20:58:59.188 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T20:58:59.188 INFO:teuthology.orchestra.run.smithi028.stdout: "metadata": { 2024-04-26T20:58:59.188 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T20:58:59.188 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T20:58:59.188 INFO:teuthology.orchestra.run.smithi028.stdout: "entity_id": "0", 2024-04-26T20:58:59.188 INFO:teuthology.orchestra.run.smithi028.stdout: "hostname": "smithi028", 2024-04-26T20:58:59.188 INFO:teuthology.orchestra.run.smithi028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-26T20:58:59.188 INFO:teuthology.orchestra.run.smithi028.stdout: "pid": "48828", 2024-04-26T20:58:59.188 INFO:teuthology.orchestra.run.smithi028.stdout: "root": "/" 2024-04-26T20:58:59.189 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:59.189 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_count": 0, 2024-04-26T20:58:59.189 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_pinned_count": 0, 2024-04-26T20:58:59.189 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 5751, 2024-04-26T20:58:59.189 INFO:teuthology.orchestra.run.smithi028.stdout: "inst": { 2024-04-26T20:58:59.189 INFO:teuthology.orchestra.run.smithi028.stdout: "name": { 2024-04-26T20:58:59.189 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "client", 2024-04-26T20:58:59.189 INFO:teuthology.orchestra.run.smithi028.stdout: "num": 5751 2024-04-26T20:58:59.189 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:59.189 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2024-04-26T20:58:59.189 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T20:58:59.189 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:58:59.189 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3114971005 2024-04-26T20:58:59.189 INFO:teuthology.orchestra.run.smithi028.stdout: } 2024-04-26T20:58:59.190 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:59.190 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2024-04-26T20:58:59.190 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2024-04-26T20:58:59.190 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.1:0", 2024-04-26T20:58:59.190 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3114971005 2024-04-26T20:58:59.190 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2024-04-26T20:58:59.190 INFO:teuthology.orchestra.run.smithi028.stdout: "inst_str": "client.5751 192.168.0.1:0/3114971005", 2024-04-26T20:58:59.190 INFO:teuthology.orchestra.run.smithi028.stdout: "addr_str": "192.168.0.1:0/3114971005", 2024-04-26T20:58:59.190 INFO:teuthology.orchestra.run.smithi028.stdout: "inode_count": 1, 2024-04-26T20:58:59.190 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_epoch": 26, 2024-04-26T20:58:59.190 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch": 45, 2024-04-26T20:58:59.190 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch_barrier": 0, 2024-04-26T20:58:59.190 INFO:teuthology.orchestra.run.smithi028.stdout: "blocklisted": false, 2024-04-26T20:58:59.190 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs" 2024-04-26T20:58:59.191 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T20:58:59.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:59.203 DEBUG:teuthology.orchestra.run.smithi028:> stat --file-system '--printf=%T 2024-04-26T20:58:59.203 DEBUG:teuthology.orchestra.run.smithi028:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-26T20:58:59.258 INFO:teuthology.orchestra.run.smithi028.stdout:fuseblk 2024-04-26T20:58:59.259 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-26T20:58:59.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:59.259 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T20:58:59.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:58:59.344 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://github.com/ceph/ceph-qa-suite) 2024-04-26T20:58:59.428 INFO:teuthology.orchestra.run.smithi028.stderr:Cloning into 'ceph-qa-suite'... 2024-04-26T20:59:16.130 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:59:16.140 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:59:32.997 INFO:teuthology.orchestra.run.smithi028.stderr:Updating files: 6% (93/1396) Updating files: 7% (98/1396) Updating files: 8% (112/1396) Updating files: 9% (126/1396) Updating files: 10% (140/1396) Updating files: 10% (148/1396) Updating files: 11% (154/1396) Updating files: 12% (168/1396) Updating files: 13% (182/1396) Updating files: 14% (196/1396) Updating files: 15% (210/1396) Updating files: 15% (212/1396) Updating files: 16% (224/1396) Updating files: 17% (238/1396) Updating files: 18% (252/1396) Updating files: 19% (266/1396) Updating files: 19% (277/1396) Updating files: 20% (280/1396) Updating files: 21% (294/1396) Updating files: 22% (308/1396) Updating files: 23% (322/1396) Updating files: 24% (336/1396) Updating files: 24% (337/1396) Updating files: 25% (349/1396) Updating files: 26% (363/1396) Updating files: 27% (377/1396) Updating files: 28% (391/1396) Updating files: 28% (399/1396) Updating files: 29% (405/1396) Updating files: 30% (419/1396) Updating files: 31% (433/1396) Updating files: 32% (447/1396) Updating files: 32% (457/1396) Updating files: 33% (461/1396) Updating files: 34% (475/1396) Updating files: 35% (489/1396) Updating files: 35% (495/1396) Updating files: 36% (503/1396) Updating files: 37% (517/1396) Updating files: 38% (531/1396) Updating files: 39% (545/1396) Updating files: 39% (554/1396) Updating files: 40% (559/1396) Updating files: 41% (573/1396) Updating files: 42% (587/1396) Updating files: 43% (601/1396) Updating files: 44% (615/1396) Updating files: 44% (621/1396) Updating files: 45% (629/1396) Updating files: 46% (643/1396) Updating files: 47% (657/1396) Updating files: 48% (671/1396) Updating files: 49% (685/1396) Updating files: 50% (698/1396) Updating files: 51% (712/1396) Updating files: 52% (726/1396) Updating files: 53% (740/1396) Updating files: 53% (751/1396) Updating files: 54% (754/1396) Updating files: 55% (768/1396) Updating files: 56% (782/1396) Updating files: 57% (796/1396) Updating files: 58% (810/1396) Updating files: 58% (818/1396) Updating files: 59% (824/1396) Updating files: 60% (838/1396) Updating files: 61% (852/1396) Updating files: 62% (866/1396) Updating files: 63% (880/1396) Updating files: 63% (885/1396) Updating files: 64% (894/1396) Updating files: 65% (908/1396) Updating files: 66% (922/1396) Updating files: 67% (936/1396) Updating files: 67% (948/1396) Updating files: 68% (950/1396) Updating files: 69% (964/1396) Updating files: 70% (978/1396) Updating files: 71% (992/1396) Updating files: 72% (1006/1396) Updating files: 72% (1007/1396) Updating files: 73% (1020/1396) Updating files: 74% (1034/1396) Updating files: 75% (1047/1396) Updating files: 76% (1061/1396) Updating files: 76% (1070/1396) Updating files: 77% (1075/1396) Updating files: 78% (1089/1396) Updating files: 79% (1103/1396) Updating files: 80% (1117/1396) Updating files: 80% (1118/1396) Updating files: 81% (1131/1396) Updating files: 82% (1145/1396) Updating files: 83% (1159/1396) Updating files: 83% (1168/1396) Updating files: 84% (1173/1396) Updating files: 85% (1187/1396) Updating files: 86% (1201/1396) Updating files: 87% (1215/1396) Updating files: 87% (1225/1396) Updating files: 88% (1229/1396) Updating files: 89% (1243/1396) Updating files: 90% (1257/1396) Updating files: 91% (1271/1396) Updating files: 92% (1285/1396) Updating files: 93% (1299/1396) Updating files: 93% (1302/1396) Updating files: 94% (1313/1396) Updating files: 95% (1327/1396) Updating files: 96% (1341/1396) Updating files: 97% (1355/1396) Updating files: 98% (1369/1396) Updating files: 99% (1383/1396) Updating files: 100% (1396/1396) Updating files: 100% (1396/1396), done. 2024-04-26T20:59:33.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T20:59:33.035 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-26T20:59:33.734 INFO:teuthology.orchestra.run.smithi028.stdout:./ 2024-04-26T20:59:33.734 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite 2024-04-26T20:59:33.734 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/erasure-code 2024-04-26T20:59:33.734 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/erasure-code/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml 2024-04-26T20:59:33.734 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/erasure-code/ec-rados-plugin=jerasure-k=3-m=1.yaml 2024-04-26T20:59:33.734 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/erasure-code/ec-rados-plugin=isa-k=2-m=1.yaml 2024-04-26T20:59:33.734 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/erasure-code/ec-rados-plugin=jerasure-k=2-m=1.yaml 2024-04-26T20:59:33.735 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/erasure-code/ec-rados-plugin=shec-k=4-m=3-c=2.yaml 2024-04-26T20:59:33.735 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/erasure-code/ec-feature-plugins-v3.yaml 2024-04-26T20:59:33.735 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/erasure-code/ec-rados-default.yaml 2024-04-26T20:59:33.735 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/erasure-code/ec-rados-sequential.yaml 2024-04-26T20:59:33.735 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/erasure-code/ec-rados-parallel.yaml 2024-04-26T20:59:33.735 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/erasure-code/ec-feature-plugins-v2.yaml 2024-04-26T20:59:33.735 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/releases 2024-04-26T20:59:33.735 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/releases/jewel.yaml 2024-04-26T20:59:33.735 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/releases/kraken.yaml 2024-04-26T20:59:33.735 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/releases/infernalis.yaml 2024-04-26T20:59:33.735 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tox.ini 2024-04-26T20:59:33.735 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/ceph-deploy-overrides 2024-04-26T20:59:33.735 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/ceph-deploy-overrides/enable_dmcrypt_diff_journal_disk.yaml 2024-04-26T20:59:33.735 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/ceph-deploy-overrides/ceph_deploy_dmcrypt.yaml 2024-04-26T20:59:33.735 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/ceph-deploy-overrides/enable_diff_journal_disk.yaml 2024-04-26T20:59:33.736 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/ceph-deploy-overrides/disable_diff_journal_disk.yaml 2024-04-26T20:59:33.736 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/nightlies 2024-04-26T20:59:33.736 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/nightlies/cron_wrapper 2024-04-26T20:59:33.736 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/overrides 2024-04-26T20:59:33.736 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/overrides/3-size-2-min-size.yaml 2024-04-26T20:59:33.736 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/overrides/whitelist_wrongly_marked_down.yaml 2024-04-26T20:59:33.736 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/overrides/2-size-2-min-size.yaml 2024-04-26T20:59:33.736 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/overrides/2-size-1-min-size.yaml 2024-04-26T20:59:33.736 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/overrides/short_pg_log.yaml 2024-04-26T20:59:33.736 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/packages 2024-04-26T20:59:33.736 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/packages/packages.yaml 2024-04-26T20:59:33.736 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/clusters 2024-04-26T20:59:33.736 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/clusters/extra-client.yaml 2024-04-26T20:59:33.736 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/clusters/fixed-2.yaml 2024-04-26T20:59:33.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/clusters/fixed-3.yaml 2024-04-26T20:59:33.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/clusters/fixed-3-cephfs.yaml 2024-04-26T20:59:33.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/clusters/fixed-4.yaml 2024-04-26T20:59:33.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/clusters/fixed-1.yaml 2024-04-26T20:59:33.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs 2024-04-26T20:59:33.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/overrides 2024-04-26T20:59:33.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/overrides/whitelist_wrongly_marked_down.yaml 2024-04-26T20:59:33.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/overrides/debug.yaml 2024-04-26T20:59:33.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/overrides/fuse 2024-04-26T20:59:33.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/overrides/fuse/default-perm 2024-04-26T20:59:33.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/overrides/fuse/default-perm/yes.yaml 2024-04-26T20:59:33.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/overrides/fuse/default-perm/no.yaml 2024-04-26T20:59:33.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/overrides/fuse/default-perm/% 2024-04-26T20:59:33.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/overrides/frag_enable.yaml 2024-04-26T20:59:33.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/clusters 2024-04-26T20:59:33.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/clusters/9-mds.yaml 2024-04-26T20:59:33.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/clusters/3-mds.yaml 2024-04-26T20:59:33.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/clusters/fixed-2-ucephfs.yaml 2024-04-26T20:59:33.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/mount 2024-04-26T20:59:33.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/mount/kclient.yaml 2024-04-26T20:59:33.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/mount/fuse.yaml 2024-04-26T20:59:33.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/begin.yaml 2024-04-26T20:59:33.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/tasks 2024-04-26T20:59:33.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/tasks/libcephfs_interface_tests.yaml 2024-04-26T20:59:33.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/tasks/cfuse_workunit_trivial_sync.yaml 2024-04-26T20:59:33.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/tasks/cfuse_workunit_suites_ffsb.yaml 2024-04-26T20:59:33.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/tasks/cfuse_workunit_suites_blogbench.yaml 2024-04-26T20:59:33.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/tasks/cfuse_workunit_suites_fsstress.yaml 2024-04-26T20:59:33.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/cephfs/tasks/cfuse_workunit_suites_dbench.yaml 2024-04-26T20:59:33.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/objectstore 2024-04-26T20:59:33.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/objectstore/bluestore.yaml 2024-04-26T20:59:33.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/objectstore/filestore.yaml 2024-04-26T20:59:33.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/config 2024-04-26T20:59:33.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/config/rados.yaml 2024-04-26T20:59:33.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/archs 2024-04-26T20:59:33.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/archs/armv7.yaml 2024-04-26T20:59:33.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/archs/i686.yaml 2024-04-26T20:59:33.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/archs/aarch64.yaml 2024-04-26T20:59:33.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/archs/x86_64.yaml 2024-04-26T20:59:33.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/fs 2024-04-26T20:59:33.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/fs/btrfs.yaml 2024-04-26T20:59:33.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/fs/xfs.yaml 2024-04-26T20:59:33.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/fs/ext4.yaml 2024-04-26T20:59:33.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/rgw_pool_type 2024-04-26T20:59:33.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/rgw_pool_type/ec-profile.yaml 2024-04-26T20:59:33.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/rgw_pool_type/ec.yaml 2024-04-26T20:59:33.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/rgw_pool_type/ec-cache.yaml 2024-04-26T20:59:33.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/rgw_pool_type/replicated.yaml 2024-04-26T20:59:33.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git 2024-04-26T20:59:33.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/HEAD 2024-04-26T20:59:33.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/description 2024-04-26T20:59:33.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/index 2024-04-26T20:59:33.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/objects 2024-04-26T20:59:33.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/objects/info 2024-04-26T20:59:33.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/objects/pack 2024-04-26T20:59:33.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/objects/pack/pack-ac547ca4f300164f23b0bf1b8b1e66d5a7352309.idx 2024-04-26T20:59:33.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/objects/pack/pack-ac547ca4f300164f23b0bf1b8b1e66d5a7352309.rev 2024-04-26T20:59:33.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/objects/pack/pack-ac547ca4f300164f23b0bf1b8b1e66d5a7352309.pack 2024-04-26T20:59:33.963 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/hooks 2024-04-26T20:59:33.963 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/hooks/pre-merge-commit.sample 2024-04-26T20:59:33.963 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/hooks/pre-commit.sample 2024-04-26T20:59:33.963 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/hooks/fsmonitor-watchman.sample 2024-04-26T20:59:33.963 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/hooks/commit-msg.sample 2024-04-26T20:59:33.963 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/hooks/pre-rebase.sample 2024-04-26T20:59:33.963 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/hooks/pre-receive.sample 2024-04-26T20:59:33.963 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/hooks/prepare-commit-msg.sample 2024-04-26T20:59:33.963 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/hooks/update.sample 2024-04-26T20:59:33.964 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/hooks/pre-push.sample 2024-04-26T20:59:33.964 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/hooks/sendemail-validate.sample 2024-04-26T20:59:33.964 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/hooks/pre-applypatch.sample 2024-04-26T20:59:33.964 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/hooks/push-to-checkout.sample 2024-04-26T20:59:33.964 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/hooks/post-update.sample 2024-04-26T20:59:33.964 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/hooks/applypatch-msg.sample 2024-04-26T20:59:33.964 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/logs 2024-04-26T20:59:33.964 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/logs/HEAD 2024-04-26T20:59:33.964 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/logs/refs 2024-04-26T20:59:33.964 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/logs/refs/heads 2024-04-26T20:59:33.964 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/logs/refs/heads/master 2024-04-26T20:59:33.964 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/logs/refs/remotes 2024-04-26T20:59:33.965 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/logs/refs/remotes/origin 2024-04-26T20:59:33.965 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/logs/refs/remotes/origin/HEAD 2024-04-26T20:59:33.965 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/packed-refs 2024-04-26T20:59:33.965 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/config 2024-04-26T20:59:33.965 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/info 2024-04-26T20:59:33.965 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/info/exclude 2024-04-26T20:59:33.965 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/refs 2024-04-26T20:59:33.965 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/refs/tags 2024-04-26T20:59:33.965 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/refs/heads 2024-04-26T20:59:33.965 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/refs/heads/master 2024-04-26T20:59:33.965 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/refs/remotes 2024-04-26T20:59:33.965 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/refs/remotes/origin 2024-04-26T20:59:33.965 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/refs/remotes/origin/HEAD 2024-04-26T20:59:33.965 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.git/branches 2024-04-26T20:59:33.966 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/config_options 2024-04-26T20:59:33.966 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/config_options/cephdeploy_conf.yaml 2024-04-26T20:59:33.966 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros 2024-04-26T20:59:33.966 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/a-supported-distro.yaml 2024-04-26T20:59:33.966 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/supported 2024-04-26T20:59:33.966 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/supported/ubuntu_16.04.yaml 2024-04-26T20:59:33.966 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/supported/centos_7.2.yaml 2024-04-26T20:59:33.966 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/supported/ubuntu_14.04.yaml 2024-04-26T20:59:33.966 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all 2024-04-26T20:59:33.966 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/fedora_17.yaml 2024-04-26T20:59:33.966 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/centos_6.4.yaml 2024-04-26T20:59:33.966 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/debian_7.0.yaml 2024-04-26T20:59:33.966 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/debian_6.0.yaml 2024-04-26T20:59:33.966 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/fedora_19.yaml 2024-04-26T20:59:33.967 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/ubuntu_16.04.yaml 2024-04-26T20:59:33.967 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/opensuse_13.2.yaml 2024-04-26T20:59:33.967 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/opensuse_12.2.yaml 2024-04-26T20:59:33.967 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/rhel_6.3.yaml 2024-04-26T20:59:33.967 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/sles_11-sp2.yaml 2024-04-26T20:59:33.967 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/centos_7.2.yaml 2024-04-26T20:59:33.967 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/rhel_6.4.yaml 2024-04-26T20:59:33.967 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/centos_7.0.yaml 2024-04-26T20:59:33.967 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/ubuntu_14.04_i686.yaml 2024-04-26T20:59:33.967 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/opensuse_42.1.yaml 2024-04-26T20:59:33.967 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/debian_8.0.yaml 2024-04-26T20:59:33.967 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/fedora_18.yaml 2024-04-26T20:59:33.967 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/rhel_7.0.yaml 2024-04-26T20:59:33.967 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/ubuntu_14.04.yaml 2024-04-26T20:59:33.968 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/centos_7.1.yaml 2024-04-26T20:59:33.968 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/ubuntu_12.10.yaml 2024-04-26T20:59:33.968 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/ubuntu_12.04.yaml 2024-04-26T20:59:33.968 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/centos_6.3.yaml 2024-04-26T20:59:33.968 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/rhel_6.5.yaml 2024-04-26T20:59:33.968 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/ubuntu_14.04_aarch64.yaml 2024-04-26T20:59:33.968 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/distros/all/centos_6.5.yaml 2024-04-26T20:59:33.968 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites 2024-04-26T20:59:33.968 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress 2024-04-26T20:59:33.968 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/bench 2024-04-26T20:59:33.968 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/bench/clusters 2024-04-26T20:59:33.968 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/bench/clusters/fixed-3-cephfs.yaml 2024-04-26T20:59:33.968 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/bench/% 2024-04-26T20:59:33.968 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/bench/tasks 2024-04-26T20:59:33.969 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/bench/tasks/kclient_workunit_suites_fsx.yaml 2024-04-26T20:59:33.969 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/bench/tasks/cfuse_workunit_snaps.yaml 2024-04-26T20:59:33.969 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/thrash 2024-04-26T20:59:33.969 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/thrash/clusters 2024-04-26T20:59:33.969 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/thrash/clusters/16-osd.yaml 2024-04-26T20:59:33.969 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/thrash/clusters/3-osd-1-machine.yaml 2024-04-26T20:59:33.969 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/thrash/clusters/8-osd.yaml 2024-04-26T20:59:33.969 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/thrash/fs 2024-04-26T20:59:33.969 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/thrash/fs/none.yaml 2024-04-26T20:59:33.969 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/thrash/fs/btrfs.yaml 2024-04-26T20:59:33.969 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/thrash/fs/xfs.yaml 2024-04-26T20:59:33.969 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/thrash/workloads 2024-04-26T20:59:33.969 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/thrash/workloads/readwrite.yaml 2024-04-26T20:59:33.969 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/thrash/workloads/radosbench.yaml 2024-04-26T20:59:33.969 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/thrash/workloads/iozone_cfuse.yaml 2024-04-26T20:59:34.217 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/thrash/workloads/bonnie_cfuse.yaml 2024-04-26T20:59:34.217 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/thrash/% 2024-04-26T20:59:34.217 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/thrash/thrashers 2024-04-26T20:59:34.217 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/thrash/thrashers/default.yaml 2024-04-26T20:59:34.217 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/thrash/thrashers/more-down.yaml 2024-04-26T20:59:34.217 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/stress/thrash/thrashers/fast.yaml 2024-04-26T20:59:34.217 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/mgr 2024-04-26T20:59:34.217 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/mgr/basic 2024-04-26T20:59:34.217 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/mgr/basic/clusters 2024-04-26T20:59:34.218 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/mgr/basic/clusters/2-node-mgr.yaml 2024-04-26T20:59:34.218 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/mgr/basic/fs 2024-04-26T20:59:34.218 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/mgr/basic/fs/xfs.yaml 2024-04-26T20:59:34.218 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/mgr/basic/% 2024-04-26T20:59:34.218 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/mgr/basic/tasks 2024-04-26T20:59:34.218 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/mgr/basic/tasks/failover.yaml 2024-04-26T20:59:34.218 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/mgr/basic/debug 2024-04-26T20:59:34.218 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/mgr/basic/debug/mgr.yaml 2024-04-26T20:59:34.218 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle 2024-04-26T20:59:34.218 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd 2024-04-26T20:59:34.218 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/powercycle 2024-04-26T20:59:34.218 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/powercycle/default.yaml 2024-04-26T20:59:34.218 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/clusters 2024-04-26T20:59:34.218 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/clusters/3osd-1per-target.yaml 2024-04-26T20:59:34.219 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/objectstore 2024-04-26T20:59:34.219 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/fs 2024-04-26T20:59:34.219 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/fs/btrfs.yaml 2024-04-26T20:59:34.219 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/fs/xfs.yaml 2024-04-26T20:59:34.219 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/% 2024-04-26T20:59:34.219 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/tasks 2024-04-26T20:59:34.219 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/tasks/readwrite.yaml 2024-04-26T20:59:34.219 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/tasks/cfuse_workunit_suites_fsync.yaml 2024-04-26T20:59:34.219 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/tasks/radosbench.yaml 2024-04-26T20:59:34.219 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/tasks/cfuse_workunit_suites_pjd.yaml 2024-04-26T20:59:34.219 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/tasks/cfuse_workunit_suites_ffsb.yaml 2024-04-26T20:59:34.219 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/tasks/cfuse_workunit_suites_fsx.yaml 2024-04-26T20:59:34.219 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/tasks/snaps-few-objects.yaml 2024-04-26T20:59:34.219 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/tasks/cfuse_workunit_misc.yaml 2024-04-26T20:59:34.219 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/tasks/admin_socket_objecter_requests.yaml 2024-04-26T20:59:34.219 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/tasks/cfuse_workunit_suites_truncate_delay.yaml 2024-04-26T20:59:34.219 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/tasks/snaps-many-objects.yaml 2024-04-26T20:59:34.220 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/tasks/cfuse_workunit_kernel_untar_build.yaml 2024-04-26T20:59:34.220 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/tasks/rados_api_tests.yaml 2024-04-26T20:59:34.220 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/powercycle/osd/tasks/cfuse_workunit_suites_fsstress.yaml 2024-04-26T20:59:34.220 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd 2024-04-26T20:59:34.220 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli 2024-04-26T20:59:34.220 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/features 2024-04-26T20:59:34.220 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/features/defaults.yaml 2024-04-26T20:59:34.220 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/features/journaling.yaml 2024-04-26T20:59:34.220 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/features/layering.yaml 2024-04-26T20:59:34.220 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/features/format-1.yaml 2024-04-26T20:59:34.220 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/base 2024-04-26T20:59:34.220 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/base/install.yaml 2024-04-26T20:59:34.220 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/pool 2024-04-26T20:59:34.220 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/pool/small-cache-pool.yaml 2024-04-26T20:59:34.220 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/pool/replicated-data-pool.yaml 2024-04-26T20:59:34.220 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/pool/ec-data-pool.yaml 2024-04-26T20:59:34.221 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/pool/none.yaml 2024-04-26T20:59:34.221 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/clusters 2024-04-26T20:59:34.221 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/msgr-failures 2024-04-26T20:59:34.221 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/msgr-failures/many.yaml 2024-04-26T20:59:34.221 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/msgr-failures/few.yaml 2024-04-26T20:59:34.221 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/objectstore 2024-04-26T20:59:34.221 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/fs 2024-04-26T20:59:34.221 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/workloads 2024-04-26T20:59:34.221 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/workloads/rbd_cli_import_export.yaml 2024-04-26T20:59:34.221 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/workloads/rbd_cli_copy.yaml 2024-04-26T20:59:34.221 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/cli/% 2024-04-26T20:59:34.221 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/maintenance 2024-04-26T20:59:34.221 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/maintenance/base 2024-04-26T20:59:34.221 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/maintenance/base/install.yaml 2024-04-26T20:59:34.221 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/maintenance/clusters 2024-04-26T20:59:34.222 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/maintenance/clusters/openstack.yaml 2024-04-26T20:59:34.222 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/maintenance/clusters/+ 2024-04-26T20:59:34.222 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/maintenance/clusters/fixed-3.yaml 2024-04-26T20:59:34.222 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/maintenance/objectstore 2024-04-26T20:59:34.222 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/maintenance/workloads 2024-04-26T20:59:34.222 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/maintenance/workloads/dynamic_features.yaml 2024-04-26T20:59:34.222 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/maintenance/workloads/rebuild_object_map.yaml 2024-04-26T20:59:34.408 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/maintenance/% 2024-04-26T20:59:34.408 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/maintenance/xfs.yaml 2024-04-26T20:59:34.408 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/maintenance/qemu 2024-04-26T20:59:34.409 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/maintenance/qemu/xfstests.yaml 2024-04-26T20:59:34.409 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/singleton 2024-04-26T20:59:34.409 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/singleton/openstack.yaml 2024-04-26T20:59:34.409 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/singleton/objectstore 2024-04-26T20:59:34.409 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/singleton/% 2024-04-26T20:59:34.409 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/singleton/all 2024-04-26T20:59:34.409 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/singleton/all/rbd-vs-unmanaged-snaps.yaml 2024-04-26T20:59:34.409 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/singleton/all/rbd_mirror.yaml 2024-04-26T20:59:34.409 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/singleton/all/verify_pool.yaml 2024-04-26T20:59:34.409 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/singleton/all/formatted-output.yaml 2024-04-26T20:59:34.409 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/singleton/all/rbdmap_RBDMAPFILE.yaml 2024-04-26T20:59:34.409 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/singleton/all/qemu-iotests-writeback.yaml 2024-04-26T20:59:34.409 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/singleton/all/read-flags-writeback.yaml 2024-04-26T20:59:34.409 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/singleton/all/permissions.yaml 2024-04-26T20:59:34.410 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/singleton/all/read-flags-no-cache.yaml 2024-04-26T20:59:34.410 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/singleton/all/qemu-iotests-no-cache.yaml 2024-04-26T20:59:34.410 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/singleton/all/qemu-iotests-writethrough.yaml 2024-04-26T20:59:34.410 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/singleton/all/read-flags-writethrough.yaml 2024-04-26T20:59:34.410 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/singleton/all/admin_socket.yaml 2024-04-26T20:59:34.410 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/singleton/all/merge_diff.yaml 2024-04-26T20:59:34.410 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic 2024-04-26T20:59:34.410 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/base 2024-04-26T20:59:34.410 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/base/install.yaml 2024-04-26T20:59:34.410 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/cachepool 2024-04-26T20:59:34.410 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/cachepool/none.yaml 2024-04-26T20:59:34.410 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/cachepool/small.yaml 2024-04-26T20:59:34.410 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/clusters 2024-04-26T20:59:34.410 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/clusters/openstack.yaml 2024-04-26T20:59:34.410 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/clusters/+ 2024-04-26T20:59:34.411 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/clusters/fixed-1.yaml 2024-04-26T20:59:34.411 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/msgr-failures 2024-04-26T20:59:34.411 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/msgr-failures/many.yaml 2024-04-26T20:59:34.411 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/msgr-failures/few.yaml 2024-04-26T20:59:34.411 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/objectstore 2024-04-26T20:59:34.411 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/fs 2024-04-26T20:59:34.411 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/fs/xfs.yaml 2024-04-26T20:59:34.411 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/% 2024-04-26T20:59:34.411 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/tasks 2024-04-26T20:59:34.411 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/tasks/rbd_lock_and_fence.yaml 2024-04-26T20:59:34.411 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/tasks/rbd_cli_tests.yaml 2024-04-26T20:59:34.411 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/tasks/rbd_api_tests_old_format.yaml 2024-04-26T20:59:34.411 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/tasks/rbd_cls_tests.yaml 2024-04-26T20:59:34.411 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/basic/tasks/rbd_python_api_tests_old_format.yaml 2024-04-26T20:59:34.411 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd 2024-04-26T20:59:34.411 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/pool 2024-04-26T20:59:34.412 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/pool/small-cache-pool.yaml 2024-04-26T20:59:34.412 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/pool/replicated-data-pool.yaml 2024-04-26T20:59:34.412 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/pool/ec-data-pool.yaml 2024-04-26T20:59:34.412 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/pool/none.yaml 2024-04-26T20:59:34.412 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/clusters 2024-04-26T20:59:34.412 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/clusters/openstack.yaml 2024-04-26T20:59:34.412 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/clusters/+ 2024-04-26T20:59:34.412 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/clusters/fixed-3.yaml 2024-04-26T20:59:34.412 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/msgr-failures 2024-04-26T20:59:34.412 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/msgr-failures/few.yaml 2024-04-26T20:59:34.412 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/objectstore 2024-04-26T20:59:34.412 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/config 2024-04-26T20:59:34.412 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/config/skip-partial-discard.yaml 2024-04-26T20:59:34.412 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/config/none.yaml 2024-04-26T20:59:34.412 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/config/copy-on-read.yaml 2024-04-26T20:59:34.412 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/fs 2024-04-26T20:59:34.412 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/workloads 2024-04-26T20:59:34.413 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/workloads/c_api_tests.yaml 2024-04-26T20:59:34.413 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/workloads/rbd_fio.yaml 2024-04-26T20:59:34.413 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/workloads/c_api_tests_with_defaults.yaml 2024-04-26T20:59:34.413 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/workloads/python_api_tests.yaml 2024-04-26T20:59:34.413 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/workloads/fsx.yaml 2024-04-26T20:59:34.413 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/workloads/python_api_tests_with_journaling.yaml 2024-04-26T20:59:34.413 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/workloads/c_api_tests_with_journaling.yaml 2024-04-26T20:59:34.413 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/workloads/python_api_tests_with_defaults.yaml 2024-04-26T20:59:34.413 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/cache 2024-04-26T20:59:34.413 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/cache/writeback.yaml 2024-04-26T20:59:34.413 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/cache/writethrough.yaml 2024-04-26T20:59:34.638 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/cache/none.yaml 2024-04-26T20:59:34.638 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/librbd/% 2024-04-26T20:59:34.639 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/valgrind 2024-04-26T20:59:34.639 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/valgrind/base 2024-04-26T20:59:34.639 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/valgrind/base/install.yaml 2024-04-26T20:59:34.639 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/valgrind/clusters 2024-04-26T20:59:34.639 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/valgrind/objectstore 2024-04-26T20:59:34.639 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/valgrind/fs 2024-04-26T20:59:34.639 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/valgrind/workloads 2024-04-26T20:59:34.639 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/valgrind/workloads/rbd_mirror.yaml 2024-04-26T20:59:34.639 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/valgrind/workloads/c_api_tests.yaml 2024-04-26T20:59:34.639 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/valgrind/workloads/c_api_tests_with_defaults.yaml 2024-04-26T20:59:34.639 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/valgrind/workloads/python_api_tests.yaml 2024-04-26T20:59:34.639 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/valgrind/workloads/fsx.yaml 2024-04-26T20:59:34.639 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/valgrind/workloads/python_api_tests_with_journaling.yaml 2024-04-26T20:59:34.640 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/valgrind/workloads/c_api_tests_with_journaling.yaml 2024-04-26T20:59:34.640 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/valgrind/workloads/python_api_tests_with_defaults.yaml 2024-04-26T20:59:34.640 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/valgrind/validator 2024-04-26T20:59:34.640 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/valgrind/validator/memcheck.yaml 2024-04-26T20:59:34.640 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/valgrind/% 2024-04-26T20:59:34.640 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/nbd 2024-04-26T20:59:34.640 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/nbd/base 2024-04-26T20:59:34.640 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/nbd/cluster 2024-04-26T20:59:34.640 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/nbd/cluster/openstack.yaml 2024-04-26T20:59:34.640 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/nbd/cluster/+ 2024-04-26T20:59:34.640 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/nbd/cluster/fixed-3.yaml 2024-04-26T20:59:34.640 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/nbd/msgr-failures 2024-04-26T20:59:34.640 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/nbd/objectstore 2024-04-26T20:59:34.640 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/nbd/fs 2024-04-26T20:59:34.641 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/nbd/workloads 2024-04-26T20:59:34.641 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/nbd/workloads/rbd_nbd.yaml 2024-04-26T20:59:34.641 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/nbd/workloads/rbd_fsx_nbd.yaml 2024-04-26T20:59:34.641 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/nbd/% 2024-04-26T20:59:34.641 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/nbd/thrashers 2024-04-26T20:59:34.641 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash 2024-04-26T20:59:34.641 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/base 2024-04-26T20:59:34.641 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/base/install.yaml 2024-04-26T20:59:34.641 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/clusters 2024-04-26T20:59:34.641 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/clusters/openstack.yaml 2024-04-26T20:59:34.641 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/clusters/fixed-2.yaml 2024-04-26T20:59:34.641 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/clusters/+ 2024-04-26T20:59:34.641 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/msgr-failures 2024-04-26T20:59:34.641 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/msgr-failures/few.yaml 2024-04-26T20:59:34.641 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/objectstore 2024-04-26T20:59:34.642 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/fs 2024-04-26T20:59:34.642 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/fs/xfs.yaml 2024-04-26T20:59:34.642 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/workloads 2024-04-26T20:59:34.642 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/workloads/rbd_fsx_cache_writeback.yaml 2024-04-26T20:59:34.642 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/workloads/rbd_fsx_copy_on_read.yaml 2024-04-26T20:59:34.642 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/workloads/rbd_fsx_journal.yaml 2024-04-26T20:59:34.642 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/workloads/rbd_api_tests_journaling.yaml 2024-04-26T20:59:34.642 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/workloads/rbd_fsx_nocache.yaml 2024-04-26T20:59:34.642 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/workloads/rbd_api_tests.yaml 2024-04-26T20:59:34.642 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/workloads/journal.yaml 2024-04-26T20:59:34.642 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/workloads/rbd_api_tests_copy_on_read.yaml 2024-04-26T20:59:34.642 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/workloads/rbd_api_tests_no_locking.yaml 2024-04-26T20:59:34.642 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/workloads/rbd_fsx_cache_writethrough.yaml 2024-04-26T20:59:34.643 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/% 2024-04-26T20:59:34.643 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/thrashers 2024-04-26T20:59:34.643 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/thrashers/default.yaml 2024-04-26T20:59:34.643 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/thrash/thrashers/cache.yaml 2024-04-26T20:59:34.643 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/mirror 2024-04-26T20:59:34.643 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/mirror/base 2024-04-26T20:59:34.643 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/mirror/base/install.yaml 2024-04-26T20:59:34.643 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/mirror/cluster 2024-04-26T20:59:34.643 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/mirror/cluster/openstack.yaml 2024-04-26T20:59:34.643 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/mirror/cluster/2-node.yaml 2024-04-26T20:59:34.643 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/mirror/cluster/+ 2024-04-26T20:59:34.643 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/mirror/msgr-failures 2024-04-26T20:59:34.643 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/mirror/objectstore 2024-04-26T20:59:34.643 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/mirror/fs 2024-04-26T20:59:34.644 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/mirror/workloads 2024-04-26T20:59:34.644 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/mirror/workloads/rbd-mirror-stress-workunit.yaml 2024-04-26T20:59:34.644 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/mirror/workloads/rbd-mirror-workunit.yaml 2024-04-26T20:59:34.644 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/mirror/% 2024-04-26T20:59:34.644 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/mirror/rbd-mirror 2024-04-26T20:59:34.644 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/mirror/rbd-mirror/one-per-cluster.yaml 2024-04-26T20:59:34.644 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu 2024-04-26T20:59:34.644 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/features 2024-04-26T20:59:34.931 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/features/defaults.yaml 2024-04-26T20:59:34.932 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/features/journaling.yaml 2024-04-26T20:59:34.932 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/pool 2024-04-26T20:59:34.932 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/pool/small-cache-pool.yaml 2024-04-26T20:59:34.932 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/pool/replicated-data-pool.yaml 2024-04-26T20:59:34.932 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/pool/ec-cache-pool.yaml 2024-04-26T20:59:34.932 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/pool/ec-data-pool.yaml 2024-04-26T20:59:34.932 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/pool/none.yaml 2024-04-26T20:59:34.932 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/clusters 2024-04-26T20:59:34.932 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/clusters/openstack.yaml 2024-04-26T20:59:34.932 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/clusters/+ 2024-04-26T20:59:34.932 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/clusters/fixed-3.yaml 2024-04-26T20:59:34.932 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/msgr-failures 2024-04-26T20:59:34.932 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/msgr-failures/few.yaml 2024-04-26T20:59:34.933 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/objectstore 2024-04-26T20:59:34.933 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/fs 2024-04-26T20:59:34.933 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/workloads 2024-04-26T20:59:34.933 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/workloads/qemu_iozone.yaml.disabled 2024-04-26T20:59:34.933 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/workloads/#qemu_xfstests.yaml# 2024-04-26T20:59:34.933 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/workloads/qemu_xfstests.yaml 2024-04-26T20:59:34.933 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/workloads/qemu_fsstress.yaml 2024-04-26T20:59:34.933 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/workloads/qemu_bonnie.yaml 2024-04-26T20:59:34.933 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/cache 2024-04-26T20:59:34.933 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/cache/writeback.yaml 2024-04-26T20:59:34.933 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/cache/writethrough.yaml 2024-04-26T20:59:34.933 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/cache/none.yaml 2024-04-26T20:59:34.933 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rbd/qemu/% 2024-04-26T20:59:34.934 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rest 2024-04-26T20:59:34.934 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rest/basic 2024-04-26T20:59:34.934 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rest/basic/tasks 2024-04-26T20:59:34.934 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rest/basic/tasks/rest_test.yaml 2024-04-26T20:59:34.934 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw 2024-04-26T20:59:34.934 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/multifs 2024-04-26T20:59:34.934 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/multifs/clusters 2024-04-26T20:59:34.934 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/multifs/clusters/fixed-2.yaml 2024-04-26T20:59:34.935 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/multifs/objectstore 2024-04-26T20:59:34.935 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/multifs/fs 2024-04-26T20:59:34.935 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/multifs/fs/xfs.yaml 2024-04-26T20:59:34.935 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/multifs/rgw_pool_type 2024-04-26T20:59:34.935 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/multifs/frontend 2024-04-26T20:59:34.935 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/multifs/frontend/apache.yaml 2024-04-26T20:59:34.936 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/multifs/frontend/civetweb.yaml 2024-04-26T20:59:34.936 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/multifs/overrides.yaml 2024-04-26T20:59:34.936 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/multifs/% 2024-04-26T20:59:34.936 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/multifs/tasks 2024-04-26T20:59:34.936 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/multifs/tasks/rgw_roundtrip.yaml 2024-04-26T20:59:34.936 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/multifs/tasks/rgw_multipart_upload.yaml 2024-04-26T20:59:34.936 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/multifs/tasks/rgw_user_quota.yaml 2024-04-26T20:59:34.936 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/multifs/tasks/rgw_readwrite.yaml 2024-04-26T20:59:34.936 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/multifs/tasks/rgw_swift.yaml 2024-04-26T20:59:34.936 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/multifs/tasks/rgw_s3tests.yaml 2024-04-26T20:59:34.936 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/multifs/tasks/rgw_bucket_quota.yaml 2024-04-26T20:59:34.936 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/singleton 2024-04-26T20:59:34.937 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/singleton/objectstore 2024-04-26T20:59:34.937 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/singleton/fs 2024-04-26T20:59:34.937 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/singleton/fs/xfs.yaml 2024-04-26T20:59:34.937 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/singleton/rgw_pool_type 2024-04-26T20:59:34.937 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/singleton/frontend 2024-04-26T20:59:34.937 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/singleton/frontend/apache.yaml 2024-04-26T20:59:34.937 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/singleton/frontend/civetweb.yaml 2024-04-26T20:59:34.937 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/singleton/overrides.yaml 2024-04-26T20:59:34.937 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/singleton/% 2024-04-26T20:59:34.937 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/singleton/xfs.yaml 2024-04-26T20:59:34.937 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/singleton/all 2024-04-26T20:59:34.937 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/singleton/all/radosgw-admin-multi-region.yaml 2024-04-26T20:59:34.937 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/singleton/all/radosgw-admin.yaml 2024-04-26T20:59:34.937 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/singleton/all/radosgw-admin-data-sync.yaml 2024-04-26T20:59:34.938 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/singleton/all/radosgw-convert-to-region.yaml 2024-04-26T20:59:34.938 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify 2024-04-26T20:59:34.938 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify/validater 2024-04-26T20:59:34.938 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify/validater/valgrind.yaml 2024-04-26T20:59:34.938 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify/validater/lockdep.yaml 2024-04-26T20:59:34.938 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify/clusters 2024-04-26T20:59:34.938 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify/clusters/fixed-2.yaml 2024-04-26T20:59:34.938 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify/msgr-failures 2024-04-26T20:59:34.938 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify/msgr-failures/few.yaml 2024-04-26T20:59:34.938 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify/objectstore 2024-04-26T20:59:34.938 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify/fs 2024-04-26T20:59:34.938 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify/fs/xfs.yaml 2024-04-26T20:59:34.938 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify/rgw_pool_type 2024-04-26T20:59:34.938 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify/frontend 2024-04-26T20:59:34.939 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify/frontend/apache.yaml 2024-04-26T20:59:35.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify/frontend/civetweb.yaml 2024-04-26T20:59:35.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify/overrides.yaml 2024-04-26T20:59:35.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify/% 2024-04-26T20:59:35.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify/tasks 2024-04-26T20:59:35.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify/tasks/rgw_s3tests_multiregion.yaml 2024-04-26T20:59:35.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify/tasks/rgw_swift.yaml 2024-04-26T20:59:35.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rgw/verify/tasks/rgw_s3tests.yaml 2024-04-26T20:59:35.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-deploy 2024-04-26T20:59:35.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-deploy/basic 2024-04-26T20:59:35.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-deploy/basic/ceph-deploy-overrides 2024-04-26T20:59:35.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-deploy/basic/python_versions 2024-04-26T20:59:35.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-deploy/basic/python_versions/python_3.yaml 2024-04-26T20:59:35.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-deploy/basic/python_versions/python_2.yaml 2024-04-26T20:59:35.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-deploy/basic/config_options 2024-04-26T20:59:35.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-deploy/basic/distros 2024-04-26T20:59:35.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-deploy/basic/% 2024-04-26T20:59:35.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-deploy/basic/tasks 2024-04-26T20:59:35.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-deploy/basic/tasks/ceph-admin-commands.yaml 2024-04-26T20:59:35.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt 2024-04-26T20:59:35.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic 2024-04-26T20:59:35.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/clusters 2024-04-26T20:59:35.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/clusters/fixed-3.yaml 2024-04-26T20:59:35.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/msgr-failures 2024-04-26T20:59:35.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/msgr-failures/many.yaml 2024-04-26T20:59:35.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/msgr-failures/few.yaml 2024-04-26T20:59:35.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/fs 2024-04-26T20:59:35.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/fs/btrfs.yaml 2024-04-26T20:59:35.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/% 2024-04-26T20:59:35.210 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/tasks 2024-04-26T20:59:35.210 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/tasks/iozone-sync.yaml 2024-04-26T20:59:35.210 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/tasks/dbench-short.yaml 2024-04-26T20:59:35.210 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/tasks/blogbench.yaml 2024-04-26T20:59:35.210 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/tasks/ffsb.yaml 2024-04-26T20:59:35.210 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/tasks/pjd.yaml 2024-04-26T20:59:35.210 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/tasks/fsync-tester.yaml 2024-04-26T20:59:35.210 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/tasks/iogen.yaml 2024-04-26T20:59:35.210 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/tasks/fsx.yaml 2024-04-26T20:59:35.210 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/tasks/bonnie.yaml 2024-04-26T20:59:35.210 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/tasks/fio.yaml 2024-04-26T20:59:35.210 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/tasks/iozone.yaml 2024-04-26T20:59:35.210 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/tasks/dbench.yaml 2024-04-26T20:59:35.211 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/tgt/basic/tasks/fsstress.yaml 2024-04-26T20:59:35.211 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/samba 2024-04-26T20:59:35.211 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/samba/install 2024-04-26T20:59:35.211 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/samba/install/install.yaml 2024-04-26T20:59:35.211 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/samba/clusters 2024-04-26T20:59:35.211 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/samba/clusters/samba-basic.yaml 2024-04-26T20:59:35.211 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/samba/workload 2024-04-26T20:59:35.211 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/samba/workload/cifs-kernel-build.yaml.disabled 2024-04-26T20:59:35.211 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/samba/workload/cifs-fsstress.yaml 2024-04-26T20:59:35.211 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/samba/workload/smbtorture.yaml 2024-04-26T20:59:35.211 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/samba/workload/cifs-dbench.yaml 2024-04-26T20:59:35.211 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/samba/fs 2024-04-26T20:59:35.211 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/samba/fs/xfs.yaml 2024-04-26T20:59:35.211 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/samba/mount 2024-04-26T20:59:35.212 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/samba/mount/noceph.yaml 2024-04-26T20:59:35.212 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/samba/mount/kclient.yaml 2024-04-26T20:59:35.212 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/samba/mount/fuse.yaml 2024-04-26T20:59:35.212 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/samba/mount/native.yaml 2024-04-26T20:59:35.212 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/samba/% 2024-04-26T20:59:35.212 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible 2024-04-26T20:59:35.212 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke 2024-04-26T20:59:35.213 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/5-tests 2024-04-26T20:59:35.213 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/5-tests/cls.yaml 2024-04-26T20:59:35.213 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/5-tests/rbd_cli_tests.yaml.disabled 2024-04-26T20:59:35.213 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/5-tests/rbd_import_export.yaml 2024-04-26T20:59:35.213 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/5-tests/ceph-admin-commands.yaml 2024-04-26T20:59:35.213 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/1-distros 2024-04-26T20:59:35.213 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/0-clusters 2024-04-26T20:59:35.213 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/0-clusters/single_mon_osd.yaml 2024-04-26T20:59:35.213 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/0-clusters/3-node.yaml 2024-04-26T20:59:35.214 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/3-common 2024-04-26T20:59:35.214 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/3-common/os_tuning 2024-04-26T20:59:35.214 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/3-common/os_tuning/vm_friendly.yaml 2024-04-26T20:59:35.214 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/3-common/source 2024-04-26T20:59:35.214 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/3-common/source/upstream_stable.yaml 2024-04-26T20:59:35.214 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/3-common/% 2024-04-26T20:59:35.214 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/% 2024-04-26T20:59:35.214 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/4-osd 2024-04-26T20:59:35.214 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/4-osd/journal 2024-04-26T20:59:35.548 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/4-osd/journal/collocated.yaml 2024-04-26T20:59:35.548 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/4-osd/% 2024-04-26T20:59:35.548 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/4-osd/devices 2024-04-26T20:59:35.548 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/4-osd/devices/osd_auto_discovery.yaml 2024-04-26T20:59:35.548 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/4-osd/devices/from_teuthology.yaml 2024-04-26T20:59:35.548 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/2-setup 2024-04-26T20:59:35.548 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-ansible/smoke/2-setup/ceph_ansible.yaml 2024-04-26T20:59:35.548 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/big 2024-04-26T20:59:35.549 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/big/rados-thrash 2024-04-26T20:59:35.549 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/big/rados-thrash/clusters 2024-04-26T20:59:35.549 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/big/rados-thrash/clusters/medium.yaml 2024-04-26T20:59:35.549 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/big/rados-thrash/clusters/big.yaml 2024-04-26T20:59:35.549 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/big/rados-thrash/clusters/small.yaml 2024-04-26T20:59:35.549 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/big/rados-thrash/fs 2024-04-26T20:59:35.549 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/big/rados-thrash/fs/btrfs.yaml 2024-04-26T20:59:35.549 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/big/rados-thrash/fs/xfs.yaml 2024-04-26T20:59:35.549 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/big/rados-thrash/workloads 2024-04-26T20:59:35.549 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/big/rados-thrash/workloads/snaps-few-objects.yaml 2024-04-26T20:59:35.549 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/big/rados-thrash/ceph 2024-04-26T20:59:35.549 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/big/rados-thrash/ceph/ceph.yaml 2024-04-26T20:59:35.549 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/big/rados-thrash/% 2024-04-26T20:59:35.549 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/big/rados-thrash/thrashers 2024-04-26T20:59:35.550 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/big/rados-thrash/thrashers/default.yaml 2024-04-26T20:59:35.550 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/mixed-clients 2024-04-26T20:59:35.550 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/mixed-clients/basic 2024-04-26T20:59:35.550 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/mixed-clients/basic/clusters 2024-04-26T20:59:35.550 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/mixed-clients/basic/clusters/fixed-3.yaml 2024-04-26T20:59:35.550 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/mixed-clients/basic/fs 2024-04-26T20:59:35.550 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/mixed-clients/basic/fs/btrfs.yaml 2024-04-26T20:59:35.550 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/mixed-clients/basic/tasks 2024-04-26T20:59:35.550 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/mixed-clients/basic/tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml 2024-04-26T20:59:35.550 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/mixed-clients/basic/tasks/kernel_cfuse_workunits_dbench_iozone.yaml 2024-04-26T20:59:35.550 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs 2024-04-26T20:59:35.551 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless 2024-04-26T20:59:35.551 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/traceless 2024-04-26T20:59:35.551 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/traceless/50pc.yaml 2024-04-26T20:59:35.551 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/overrides 2024-04-26T20:59:35.551 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/overrides/whitelist_wrongly_marked_down.yaml 2024-04-26T20:59:35.551 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/overrides/debug.yaml 2024-04-26T20:59:35.551 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/overrides/+ 2024-04-26T20:59:35.551 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/overrides/frag_enable.yaml 2024-04-26T20:59:35.551 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/clusters 2024-04-26T20:59:35.551 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/clusters/fixed-2-ucephfs.yaml 2024-04-26T20:59:35.551 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/objectstore 2024-04-26T20:59:35.551 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/fs 2024-04-26T20:59:35.551 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/fs/xfs.yaml 2024-04-26T20:59:35.551 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/mount 2024-04-26T20:59:35.551 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/mount/fuse.yaml 2024-04-26T20:59:35.552 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/begin.yaml 2024-04-26T20:59:35.552 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/% 2024-04-26T20:59:35.552 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/tasks 2024-04-26T20:59:35.552 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/tasks/cfuse_workunit_suites_ffsb.yaml 2024-04-26T20:59:35.552 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/tasks/cfuse_workunit_suites_blogbench.yaml 2024-04-26T20:59:35.552 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/tasks/cfuse_workunit_suites_fsstress.yaml 2024-04-26T20:59:35.552 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/traceless/tasks/cfuse_workunit_suites_dbench.yaml 2024-04-26T20:59:35.552 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multifs 2024-04-26T20:59:35.552 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multifs/overrides 2024-04-26T20:59:35.552 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multifs/overrides/whitelist_wrongly_marked_down.yaml 2024-04-26T20:59:35.552 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multifs/overrides/mon-debug.yaml 2024-04-26T20:59:35.552 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multifs/overrides/debug.yaml 2024-04-26T20:59:35.552 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multifs/overrides/+ 2024-04-26T20:59:35.552 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multifs/overrides/frag_enable.yaml 2024-04-26T20:59:35.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multifs/clusters 2024-04-26T20:59:35.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multifs/clusters/2-remote-clients.yaml 2024-04-26T20:59:35.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multifs/objectstore 2024-04-26T20:59:35.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multifs/fs 2024-04-26T20:59:35.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multifs/fs/xfs.yaml 2024-04-26T20:59:35.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multifs/mount 2024-04-26T20:59:35.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multifs/mount/fuse.yaml 2024-04-26T20:59:35.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multifs/begin.yaml 2024-04-26T20:59:35.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multifs/% 2024-04-26T20:59:35.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multifs/tasks 2024-04-26T20:59:35.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multifs/tasks/failover.yaml 2024-04-26T20:59:35.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/snaps 2024-04-26T20:59:35.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/snaps/overrides 2024-04-26T20:59:35.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/snaps/overrides/whitelist_wrongly_marked_down.yaml 2024-04-26T20:59:35.780 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/snaps/overrides/debug.yaml 2024-04-26T20:59:35.780 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/snaps/overrides/+ 2024-04-26T20:59:35.780 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/snaps/overrides/frag_enable.yaml 2024-04-26T20:59:35.780 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/snaps/clusters 2024-04-26T20:59:35.780 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/snaps/clusters/fixed-2-ucephfs.yaml 2024-04-26T20:59:35.780 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/snaps/objectstore 2024-04-26T20:59:35.780 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/snaps/fs 2024-04-26T20:59:35.780 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/snaps/fs/xfs.yaml 2024-04-26T20:59:35.780 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/snaps/mount 2024-04-26T20:59:35.780 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/snaps/mount/fuse.yaml 2024-04-26T20:59:35.780 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/snaps/begin.yaml 2024-04-26T20:59:35.780 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/snaps/% 2024-04-26T20:59:35.781 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/snaps/tasks 2024-04-26T20:59:35.781 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/snaps/tasks/snaptests.yaml 2024-04-26T20:59:35.781 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic 2024-04-26T20:59:35.781 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/overrides 2024-04-26T20:59:35.781 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/overrides/whitelist_wrongly_marked_down.yaml 2024-04-26T20:59:35.781 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/overrides/debug.yaml 2024-04-26T20:59:35.781 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/overrides/+ 2024-04-26T20:59:35.781 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/overrides/frag_enable.yaml 2024-04-26T20:59:35.781 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/clusters 2024-04-26T20:59:35.781 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/clusters/fixed-2-ucephfs.yaml 2024-04-26T20:59:35.781 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/objectstore 2024-04-26T20:59:35.781 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/fs 2024-04-26T20:59:35.781 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/fs/xfs.yaml 2024-04-26T20:59:35.781 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/mount 2024-04-26T20:59:35.782 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/mount/fuse.yaml 2024-04-26T20:59:35.782 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/inline 2024-04-26T20:59:35.782 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/inline/yes.yaml 2024-04-26T20:59:35.782 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/inline/no.yaml 2024-04-26T20:59:35.782 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/begin.yaml 2024-04-26T20:59:35.782 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/% 2024-04-26T20:59:35.782 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks 2024-04-26T20:59:35.782 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_suites_fsync.yaml 2024-04-26T20:59:35.782 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/libcephfs_interface_tests.yaml 2024-04-26T20:59:35.782 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_suites_pjd.yaml 2024-04-26T20:59:35.782 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_norstats.yaml 2024-04-26T20:59:35.782 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_trivial_sync.yaml 2024-04-26T20:59:35.782 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_suites_ffsb.yaml 2024-04-26T20:59:35.783 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_suites_fsx.yaml 2024-04-26T20:59:35.783 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/libcephfs_java.yaml 2024-04-26T20:59:35.783 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_misc.yaml 2024-04-26T20:59:35.783 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_misc_test_o_trunc.yaml 2024-04-26T20:59:35.783 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/mds_creation_retry.yaml 2024-04-26T20:59:35.783 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/asok_dump_tree.yaml 2024-04-26T20:59:35.783 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_quota.yaml 2024-04-26T20:59:35.783 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_suites_truncate_delay.yaml 2024-04-26T20:59:35.783 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_kernel_untar_build.yaml 2024-04-26T20:59:35.783 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_suites_blogbench.yaml 2024-04-26T20:59:35.783 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/libcephfs_python.yaml 2024-04-26T20:59:35.783 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_suites_fsstress.yaml 2024-04-26T20:59:35.783 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/cephfs_scrub_tests.yaml 2024-04-26T20:59:35.783 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_suites_iogen.yaml 2024-04-26T20:59:35.783 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_suites_dbench.yaml 2024-04-26T20:59:35.783 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_suites_iozone.yaml 2024-04-26T20:59:35.784 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery 2024-04-26T20:59:35.784 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/overrides 2024-04-26T20:59:35.784 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/overrides/whitelist_wrongly_marked_down.yaml 2024-04-26T20:59:35.784 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/overrides/fuse-unmounted.yaml 2024-04-26T20:59:35.784 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/overrides/debug.yaml 2024-04-26T20:59:35.784 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/overrides/+ 2024-04-26T20:59:35.784 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/overrides/frag_enable.yaml 2024-04-26T20:59:35.784 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/clusters 2024-04-26T20:59:35.784 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/clusters/4-remote-clients.yaml 2024-04-26T20:59:35.784 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/objectstore 2024-04-26T20:59:35.784 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/fs 2024-04-26T20:59:35.784 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/fs/xfs.yaml 2024-04-26T20:59:35.784 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/mount 2024-04-26T20:59:35.784 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/mount/fuse.yaml 2024-04-26T20:59:35.784 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/begin.yaml 2024-04-26T20:59:35.785 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/% 2024-04-26T20:59:35.785 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/tasks 2024-04-26T20:59:35.785 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/tasks/backtrace.yaml 2024-04-26T20:59:35.785 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/tasks/forward-scrub.yaml 2024-04-26T20:59:35.785 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/tasks/client-readahad.yaml 2024-04-26T20:59:35.785 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/tasks/volume-client.yaml 2024-04-26T20:59:36.017 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/tasks/config-commands.yaml 2024-04-26T20:59:36.017 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/tasks/mds-full.yaml 2024-04-26T20:59:36.017 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/tasks/sessionmap.yaml 2024-04-26T20:59:36.017 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/tasks/data-scan.yaml 2024-04-26T20:59:36.017 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/tasks/cap-flush.yaml 2024-04-26T20:59:36.017 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/tasks/mds-flush.yaml 2024-04-26T20:59:36.017 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/tasks/auto-repair.yaml 2024-04-26T20:59:36.017 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/tasks/damage.yaml 2024-04-26T20:59:36.017 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/tasks/client-recovery.yaml 2024-04-26T20:59:36.017 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/tasks/journal-repair.yaml 2024-04-26T20:59:36.017 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/tasks/client-limits.yaml 2024-04-26T20:59:36.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/tasks/strays.yaml 2024-04-26T20:59:36.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/tasks/fragment.yaml 2024-04-26T20:59:36.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/recovery/tasks/pool-perm.yaml 2024-04-26T20:59:36.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/permission 2024-04-26T20:59:36.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/permission/overrides 2024-04-26T20:59:36.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/permission/overrides/whitelist_wrongly_marked_down.yaml 2024-04-26T20:59:36.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/permission/overrides/debug.yaml 2024-04-26T20:59:36.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/permission/overrides/+ 2024-04-26T20:59:36.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/permission/overrides/frag_enable.yaml 2024-04-26T20:59:36.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/permission/clusters 2024-04-26T20:59:36.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/permission/clusters/fixed-2-ucephfs.yaml 2024-04-26T20:59:36.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/permission/objectstore 2024-04-26T20:59:36.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/permission/fs 2024-04-26T20:59:36.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/permission/fs/xfs.yaml 2024-04-26T20:59:36.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/permission/mount 2024-04-26T20:59:36.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/permission/mount/fuse.yaml 2024-04-26T20:59:36.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/permission/begin.yaml 2024-04-26T20:59:36.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/permission/% 2024-04-26T20:59:36.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/permission/tasks 2024-04-26T20:59:36.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/permission/tasks/cfuse_workunit_suites_pjd.yaml 2024-04-26T20:59:36.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/permission/tasks/cfuse_workunit_misc.yaml 2024-04-26T20:59:36.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/standbyreplay 2024-04-26T20:59:36.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/standbyreplay/overrides 2024-04-26T20:59:36.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/standbyreplay/overrides/whitelist_wrongly_marked_down.yaml 2024-04-26T20:59:36.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/standbyreplay/overrides/debug.yaml 2024-04-26T20:59:36.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/standbyreplay/overrides/+ 2024-04-26T20:59:36.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/standbyreplay/overrides/frag_enable.yaml 2024-04-26T20:59:36.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/standbyreplay/clusters 2024-04-26T20:59:36.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/standbyreplay/clusters/standby-replay.yaml 2024-04-26T20:59:36.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/standbyreplay/objectstore 2024-04-26T20:59:36.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/standbyreplay/fs 2024-04-26T20:59:36.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/standbyreplay/fs/xfs.yaml 2024-04-26T20:59:36.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/standbyreplay/mount 2024-04-26T20:59:36.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/standbyreplay/mount/fuse.yaml 2024-04-26T20:59:36.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/standbyreplay/begin.yaml 2024-04-26T20:59:36.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/standbyreplay/% 2024-04-26T20:59:36.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/standbyreplay/tasks 2024-04-26T20:59:36.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/standbyreplay/tasks/migration.yaml 2024-04-26T20:59:36.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/32bits 2024-04-26T20:59:36.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/32bits/overrides 2024-04-26T20:59:36.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/32bits/overrides/whitelist_wrongly_marked_down.yaml 2024-04-26T20:59:36.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/32bits/overrides/debug.yaml 2024-04-26T20:59:36.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/32bits/overrides/+ 2024-04-26T20:59:36.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/32bits/overrides/faked-ino.yaml 2024-04-26T20:59:36.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/32bits/overrides/frag_enable.yaml 2024-04-26T20:59:36.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/32bits/clusters 2024-04-26T20:59:36.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/32bits/clusters/fixed-2-ucephfs.yaml 2024-04-26T20:59:36.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/32bits/objectstore 2024-04-26T20:59:36.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/32bits/fs 2024-04-26T20:59:36.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/32bits/fs/xfs.yaml 2024-04-26T20:59:36.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/32bits/mount 2024-04-26T20:59:36.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/32bits/mount/fuse.yaml 2024-04-26T20:59:36.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/32bits/begin.yaml 2024-04-26T20:59:36.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/32bits/% 2024-04-26T20:59:36.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/32bits/tasks 2024-04-26T20:59:36.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/32bits/tasks/cfuse_workunit_suites_pjd.yaml 2024-04-26T20:59:36.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/32bits/tasks/cfuse_workunit_suites_fsstress.yaml 2024-04-26T20:59:36.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash 2024-04-26T20:59:36.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/ceph-thrash 2024-04-26T20:59:36.022 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/ceph-thrash/default.yaml 2024-04-26T20:59:36.022 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/overrides 2024-04-26T20:59:36.022 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/overrides/whitelist_wrongly_marked_down.yaml 2024-04-26T20:59:36.022 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/overrides/debug.yaml 2024-04-26T20:59:36.022 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/overrides/+ 2024-04-26T20:59:36.022 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/overrides/frag_enable.yaml 2024-04-26T20:59:36.022 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/clusters 2024-04-26T20:59:36.291 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/clusters/mds-1active-1standby.yaml 2024-04-26T20:59:36.291 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/msgr-failures 2024-04-26T20:59:36.292 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/msgr-failures/osd-mds-delay.yaml 2024-04-26T20:59:36.292 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/msgr-failures/none.yaml 2024-04-26T20:59:36.292 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/objectstore 2024-04-26T20:59:36.292 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/fs 2024-04-26T20:59:36.292 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/fs/xfs.yaml 2024-04-26T20:59:36.292 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/mount 2024-04-26T20:59:36.292 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/mount/fuse.yaml 2024-04-26T20:59:36.292 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/begin.yaml 2024-04-26T20:59:36.292 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/% 2024-04-26T20:59:36.292 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/tasks 2024-04-26T20:59:36.292 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/tasks/cfuse_workunit_suites_pjd.yaml 2024-04-26T20:59:36.292 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/tasks/cfuse_workunit_trivial_sync.yaml 2024-04-26T20:59:36.292 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/tasks/cfuse_workunit_snaptests.yaml 2024-04-26T20:59:36.292 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/thrash/tasks/cfuse_workunit_suites_fsstress.yaml 2024-04-26T20:59:36.293 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify 2024-04-26T20:59:36.293 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/validater 2024-04-26T20:59:36.293 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-26T20:59:36.293 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/validater/lockdep.yaml 2024-04-26T20:59:36.293 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/overrides 2024-04-26T20:59:36.293 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/overrides/whitelist_wrongly_marked_down.yaml 2024-04-26T20:59:36.293 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/overrides/mon-debug.yaml 2024-04-26T20:59:36.293 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/overrides/debug.yaml 2024-04-26T20:59:36.293 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/overrides/+ 2024-04-26T20:59:36.293 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/overrides/frag_enable.yaml 2024-04-26T20:59:36.293 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/clusters 2024-04-26T20:59:36.294 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/clusters/fixed-2-ucephfs.yaml 2024-04-26T20:59:36.294 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/objectstore 2024-04-26T20:59:36.294 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/fs 2024-04-26T20:59:36.294 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/fs/xfs.yaml 2024-04-26T20:59:36.294 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/mount 2024-04-26T20:59:36.294 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/mount/fuse.yaml 2024-04-26T20:59:36.294 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/begin.yaml 2024-04-26T20:59:36.294 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/% 2024-04-26T20:59:36.295 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/tasks 2024-04-26T20:59:36.295 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/tasks/libcephfs_interface_tests.yaml 2024-04-26T20:59:36.295 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/tasks/cfuse_workunit_suites_fsstress.yaml 2024-04-26T20:59:36.295 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/verify/tasks/cfuse_workunit_suites_dbench.yaml 2024-04-26T20:59:36.295 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient 2024-04-26T20:59:36.295 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/overrides 2024-04-26T20:59:36.295 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/overrides/whitelist_wrongly_marked_down.yaml 2024-04-26T20:59:36.295 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/overrides/debug.yaml 2024-04-26T20:59:36.295 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/overrides/+ 2024-04-26T20:59:36.295 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/overrides/frag_enable.yaml 2024-04-26T20:59:36.295 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/clusters 2024-04-26T20:59:36.295 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/clusters/three_clients.yaml 2024-04-26T20:59:36.295 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/clusters/two_clients.yaml 2024-04-26T20:59:36.295 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/objectstore 2024-04-26T20:59:36.296 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/fs 2024-04-26T20:59:36.296 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/fs/xfs.yaml 2024-04-26T20:59:36.297 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/mount 2024-04-26T20:59:36.298 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/mount/fuse.yaml 2024-04-26T20:59:36.298 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/mount/kclient.yaml.disabled 2024-04-26T20:59:36.298 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/begin.yaml 2024-04-26T20:59:36.298 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/% 2024-04-26T20:59:36.298 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/tasks 2024-04-26T20:59:36.298 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/tasks/fsx-mpi.yaml.disabled 2024-04-26T20:59:36.298 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/tasks/ior-shared-file.yaml 2024-04-26T20:59:36.298 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/tasks/cephfs_misc_tests.yaml 2024-04-26T20:59:36.298 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/fs/multiclient/tasks/mdtest.yaml 2024-04-26T20:59:36.298 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-disk 2024-04-26T20:59:36.298 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-disk/basic 2024-04-26T20:59:36.299 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-disk/basic/distros 2024-04-26T20:59:36.299 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-disk/basic/% 2024-04-26T20:59:36.299 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-disk/basic/tasks 2024-04-26T20:59:36.299 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/ceph-disk/basic/tasks/ceph-disk.yaml 2024-04-26T20:59:36.299 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd 2024-04-26T20:59:36.299 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/unmap 2024-04-26T20:59:36.299 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/unmap/clusters 2024-04-26T20:59:36.299 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/unmap/clusters/separate-client.yaml 2024-04-26T20:59:36.299 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/unmap/ceph 2024-04-26T20:59:36.299 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/unmap/ceph/ceph.yaml 2024-04-26T20:59:36.299 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/unmap/kernels 2024-04-26T20:59:36.299 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/unmap/kernels/pre-single-major.yaml 2024-04-26T20:59:36.299 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/unmap/kernels/single-major-on.yaml 2024-04-26T20:59:36.300 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/unmap/kernels/single-major-off.yaml 2024-04-26T20:59:36.300 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/unmap/conf.yaml 2024-04-26T20:59:36.300 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/unmap/% 2024-04-26T20:59:36.552 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/unmap/xfs.yaml 2024-04-26T20:59:36.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/unmap/tasks 2024-04-26T20:59:36.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/unmap/tasks/unmap.yaml 2024-04-26T20:59:36.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd 2024-04-26T20:59:36.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd/clusters 2024-04-26T20:59:36.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd/clusters/fixed-3.yaml 2024-04-26T20:59:36.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd/msgr-failures 2024-04-26T20:59:36.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd/msgr-failures/many.yaml 2024-04-26T20:59:36.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd/msgr-failures/few.yaml 2024-04-26T20:59:36.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd/fs 2024-04-26T20:59:36.553 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd/fs/btrfs.yaml 2024-04-26T20:59:36.554 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd/conf.yaml 2024-04-26T20:59:36.554 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd/% 2024-04-26T20:59:36.554 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd/tasks 2024-04-26T20:59:36.554 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd/tasks/rbd_workunit_suites_fsstress.yaml 2024-04-26T20:59:36.554 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd/tasks/rbd_fio.yaml 2024-04-26T20:59:36.554 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd/tasks/rbd_workunit_suites_ffsb.yaml 2024-04-26T20:59:36.554 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd/tasks/rbd_workunit_suites_fsstress_ext4.yaml 2024-04-26T20:59:36.554 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd/tasks/rbd_workunit_suites_iozone.yaml 2024-04-26T20:59:36.554 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd/tasks/rbd_workunit_suites_dbench.yaml 2024-04-26T20:59:36.554 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd/tasks/rbd_workunit_kernel_untar_build.yaml 2024-04-26T20:59:36.554 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd/tasks/rbd_workunit_trivial_sync.yaml 2024-04-26T20:59:36.554 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd/tasks/rbd_workunit_suites_fsstress_btrfs.yaml 2024-04-26T20:59:36.554 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd/tasks/rbd_workunit_suites_fsx.yaml 2024-04-26T20:59:36.554 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/singleton 2024-04-26T20:59:36.555 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/singleton/msgr-failures 2024-04-26T20:59:36.555 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/singleton/msgr-failures/many.yaml 2024-04-26T20:59:36.555 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/singleton/msgr-failures/few.yaml 2024-04-26T20:59:36.555 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/singleton/fs 2024-04-26T20:59:36.555 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/singleton/fs/btrfs.yaml 2024-04-26T20:59:36.555 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/singleton/conf.yaml 2024-04-26T20:59:36.555 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/singleton/% 2024-04-26T20:59:36.555 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/singleton/tasks 2024-04-26T20:59:36.555 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/singleton/tasks/rbd_xfstests.yaml 2024-04-26T20:59:36.555 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount 2024-04-26T20:59:36.556 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount/install 2024-04-26T20:59:36.556 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount/install/ceph.yaml 2024-04-26T20:59:36.556 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount/clusters 2024-04-26T20:59:36.556 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount/clusters/fixed-3.yaml 2024-04-26T20:59:36.556 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount/msgr-failures 2024-04-26T20:59:36.556 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount/msgr-failures/many.yaml 2024-04-26T20:59:36.556 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount/msgr-failures/few.yaml 2024-04-26T20:59:36.557 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount/fs 2024-04-26T20:59:36.557 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount/fs/btrfs.yaml 2024-04-26T20:59:36.557 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount/conf.yaml 2024-04-26T20:59:36.557 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount/% 2024-04-26T20:59:36.557 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount/tasks 2024-04-26T20:59:36.557 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount/tasks/rbd_map_unmap.yaml 2024-04-26T20:59:36.557 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount/tasks/rbd_kfsx.yaml 2024-04-26T20:59:36.557 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount/tasks/rbd_image_read.yaml 2024-04-26T20:59:36.557 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount/tasks/rbd_simple_big.yaml 2024-04-26T20:59:36.557 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount/tasks/rbd_concurrent.yaml 2024-04-26T20:59:36.557 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount/tasks/rbd_map_snapshot_io.yaml 2024-04-26T20:59:36.557 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount/tasks/rbd_huge_tickets.yaml 2024-04-26T20:59:36.557 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/rbd-nomount/tasks/rbd_kernel.yaml 2024-04-26T20:59:36.557 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/thrash 2024-04-26T20:59:36.557 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/thrash/clusters 2024-04-26T20:59:36.558 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/thrash/clusters/fixed-3.yaml 2024-04-26T20:59:36.558 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/thrash/fs 2024-04-26T20:59:36.558 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/thrash/fs/btrfs.yaml 2024-04-26T20:59:36.558 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/thrash/workloads 2024-04-26T20:59:36.558 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/thrash/workloads/rbd_fio.yaml 2024-04-26T20:59:36.558 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/thrash/workloads/rbd_workunit_suites_ffsb.yaml 2024-04-26T20:59:36.558 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/thrash/workloads/rbd_workunit_suites_iozone.yaml.disabled 2024-04-26T20:59:36.558 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/thrash/conf.yaml 2024-04-26T20:59:36.558 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/thrash/% 2024-04-26T20:59:36.558 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/thrash/thrashers 2024-04-26T20:59:36.558 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/thrash/thrashers/default.yaml 2024-04-26T20:59:36.558 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/krbd/thrash/thrashers/mon-thrasher.yaml 2024-04-26T20:59:36.558 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/buildpackages 2024-04-26T20:59:36.559 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/buildpackages/tests 2024-04-26T20:59:36.559 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/buildpackages/tests/distros 2024-04-26T20:59:36.559 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/buildpackages/tests/% 2024-04-26T20:59:36.559 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/buildpackages/tests/tasks 2024-04-26T20:59:36.559 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/buildpackages/tests/tasks/release.yaml 2024-04-26T20:59:36.559 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/buildpackages/any 2024-04-26T20:59:36.559 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/buildpackages/any/distros 2024-04-26T20:59:36.736 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/buildpackages/any/% 2024-04-26T20:59:36.736 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/buildpackages/any/tasks 2024-04-26T20:59:36.736 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/buildpackages/any/tasks/release.yaml 2024-04-26T20:59:36.736 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados 2024-04-26T20:59:36.736 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/objectstore 2024-04-26T20:59:36.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/objectstore/filestore-idempotent-aio-journal.yaml 2024-04-26T20:59:36.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/objectstore/ceph_objectstore_tool.yaml 2024-04-26T20:59:36.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/objectstore/fusestore.yaml 2024-04-26T20:59:36.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/objectstore/keyvaluedb.yaml 2024-04-26T20:59:36.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/objectstore/objectstore.yaml 2024-04-26T20:59:36.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/objectstore/alloc-hint.yaml 2024-04-26T20:59:36.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/objectstore/filejournal.yaml 2024-04-26T20:59:36.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/objectstore/objectcacher-stress.yaml 2024-04-26T20:59:36.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/objectstore/filestore-idempotent.yaml 2024-04-26T20:59:36.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton 2024-04-26T20:59:36.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/msgr 2024-04-26T20:59:36.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/msgr-failures 2024-04-26T20:59:36.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/msgr-failures/many.yaml 2024-04-26T20:59:36.737 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/msgr-failures/few.yaml 2024-04-26T20:59:36.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/objectstore 2024-04-26T20:59:36.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/fs 2024-04-26T20:59:36.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/fs/xfs.yaml 2024-04-26T20:59:36.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/% 2024-04-26T20:59:36.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all 2024-04-26T20:59:36.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/divergent_priors.yaml 2024-04-26T20:59:36.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/watch-notify-same-primary.yaml 2024-04-26T20:59:36.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/ec-lost-unfound.yaml 2024-04-26T20:59:36.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/rest-api.yaml 2024-04-26T20:59:36.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/mon-config-keys.yaml 2024-04-26T20:59:36.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/mon-thrasher.yaml 2024-04-26T20:59:36.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/osd-recovery-incomplete.yaml 2024-04-26T20:59:36.738 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/osd-recovery.yaml 2024-04-26T20:59:36.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/admin-socket.yaml 2024-04-26T20:59:36.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/reg11184.yaml 2024-04-26T20:59:36.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/mon-seesaw.yaml 2024-04-26T20:59:36.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/radostool.yaml 2024-04-26T20:59:36.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/thrash_cache_writeback_proxy_none.yaml 2024-04-26T20:59:36.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/dump-stuck.yaml 2024-04-26T20:59:36.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/resolve_stuck_peering.yaml 2024-04-26T20:59:36.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/osd-backfill.yaml 2024-04-26T20:59:36.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/divergent_priors2.yaml 2024-04-26T20:59:36.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/peer.yaml 2024-04-26T20:59:36.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/test_envlibrados_for_rocksdb.yaml 2024-04-26T20:59:36.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/thrash-rados.yaml 2024-04-26T20:59:36.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/lost-unfound.yaml 2024-04-26T20:59:36.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/cephtool.yaml 2024-04-26T20:59:36.739 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/pg-removal-interruption.yaml 2024-04-26T20:59:36.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/rebuild-mondb.yaml 2024-04-26T20:59:36.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/all/lost-unfound-delete.yaml 2024-04-26T20:59:36.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton/rados.yaml 2024-04-26T20:59:36.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic 2024-04-26T20:59:36.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/msgr 2024-04-26T20:59:36.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/msgr/async.yaml 2024-04-26T20:59:36.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/msgr/simple.yaml 2024-04-26T20:59:36.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/msgr/random.yaml 2024-04-26T20:59:36.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/clusters 2024-04-26T20:59:36.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/clusters/openstack.yaml 2024-04-26T20:59:36.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/clusters/fixed-2.yaml 2024-04-26T20:59:36.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/clusters/+ 2024-04-26T20:59:36.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/msgr-failures 2024-04-26T20:59:36.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/msgr-failures/many.yaml 2024-04-26T20:59:36.740 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/msgr-failures/few.yaml 2024-04-26T20:59:36.741 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/objectstore 2024-04-26T20:59:36.741 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/fs 2024-04-26T20:59:36.741 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/fs/btrfs.yaml 2024-04-26T20:59:36.741 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/fs/xfs.yaml 2024-04-26T20:59:36.741 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/% 2024-04-26T20:59:36.741 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/mon_kv_backend 2024-04-26T20:59:36.741 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/tasks 2024-04-26T20:59:36.741 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/tasks/readwrite.yaml 2024-04-26T20:59:36.741 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/tasks/rados_cls_all.yaml 2024-04-26T20:59:36.741 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/tasks/rados_workunit_loadgen_mostlyread.yaml 2024-04-26T20:59:36.741 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/tasks/rados_stress_watch.yaml 2024-04-26T20:59:36.853 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/tasks/rados_workunit_loadgen_big.yaml 2024-04-26T20:59:36.853 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/tasks/scrub_test.yaml 2024-04-26T20:59:36.853 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/tasks/rados_workunit_loadgen_mix.yaml 2024-04-26T20:59:36.853 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/tasks/rados_python.yaml 2024-04-26T20:59:36.853 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/tasks/repair_test.yaml 2024-04-26T20:59:36.854 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/tasks/rados_api_tests.yaml 2024-04-26T20:59:36.854 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/tasks/rgw_snaps.yaml 2024-04-26T20:59:36.854 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/tasks/rados_striper.yaml 2024-04-26T20:59:36.854 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/basic/rados.yaml 2024-04-26T20:59:36.854 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-shec 2024-04-26T20:59:36.854 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/leveldb.yaml 2024-04-26T20:59:36.854 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/clusters 2024-04-26T20:59:36.854 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/clusters/openstack.yaml 2024-04-26T20:59:36.854 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/clusters/+ 2024-04-26T20:59:36.854 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/clusters/fixed-4.yaml 2024-04-26T20:59:36.854 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/msgr-failures 2024-04-26T20:59:36.854 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/objectstore 2024-04-26T20:59:36.854 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/fs 2024-04-26T20:59:36.855 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/fs/xfs.yaml 2024-04-26T20:59:36.855 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/workloads 2024-04-26T20:59:36.855 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml 2024-04-26T20:59:36.855 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/% 2024-04-26T20:59:36.855 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/thrashers 2024-04-26T20:59:36.855 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/thrashers/default.yaml 2024-04-26T20:59:36.855 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/rados.yaml 2024-04-26T20:59:36.855 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code 2024-04-26T20:59:36.855 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/fast 2024-04-26T20:59:36.855 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/fast/normal.yaml 2024-04-26T20:59:36.855 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/fast/fast.yaml 2024-04-26T20:59:36.855 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/leveldb.yaml 2024-04-26T20:59:36.856 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/clusters 2024-04-26T20:59:36.856 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/msgr-failures 2024-04-26T20:59:36.856 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/objectstore 2024-04-26T20:59:36.856 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/fs 2024-04-26T20:59:36.856 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/workloads 2024-04-26T20:59:36.856 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/workloads/ec-small-objects-fast-read-overwrites.yaml 2024-04-26T20:59:36.856 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/workloads/ec-small-objects-overwrites.yaml 2024-04-26T20:59:36.856 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml 2024-04-26T20:59:36.856 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/workloads/ec-snaps-few-objects-overwrites.yaml 2024-04-26T20:59:36.856 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/workloads/ec-radosbench.yaml 2024-04-26T20:59:36.856 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/workloads/ec-rados-plugin=jerasure-k=2-m=1.yaml 2024-04-26T20:59:36.856 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/workloads/ec-small-objects.yaml 2024-04-26T20:59:36.856 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/workloads/ec-small-objects-fast-read.yaml 2024-04-26T20:59:36.856 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/workloads/ec-pool-snaps-few-objects-overwrites.yaml 2024-04-26T20:59:36.856 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/% 2024-04-26T20:59:36.856 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/thrashers 2024-04-26T20:59:36.857 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/thrashers/default.yaml 2024-04-26T20:59:36.857 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/thrashers/fastread.yaml 2024-04-26T20:59:36.857 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/thrashers/morepggrow.yaml 2024-04-26T20:59:36.857 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/thrashers/pggrow.yaml 2024-04-26T20:59:36.857 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/thrashers/mapgap.yaml 2024-04-26T20:59:36.857 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code/rados.yaml 2024-04-26T20:59:36.857 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton-nomsgr 2024-04-26T20:59:36.857 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton-nomsgr/% 2024-04-26T20:59:36.857 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton-nomsgr/all 2024-04-26T20:59:36.857 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton-nomsgr/all/multi-backfill-reject.yaml 2024-04-26T20:59:36.857 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton-nomsgr/all/full-tiering.yaml 2024-04-26T20:59:36.857 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton-nomsgr/all/cache-fs-trunc.yaml 2024-04-26T20:59:36.857 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton-nomsgr/all/msgr.yaml 2024-04-26T20:59:36.857 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton-nomsgr/all/ceph-post-file.yaml 2024-04-26T20:59:37.053 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton-nomsgr/all/export-after-evict.yaml 2024-04-26T20:59:37.053 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton-nomsgr/all/valgrind-leaks.yaml 2024-04-26T20:59:37.053 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/singleton-nomsgr/rados.yaml 2024-04-26T20:59:37.053 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade 2024-04-26T20:59:37.053 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton 2024-04-26T20:59:37.053 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/6-next-mon 2024-04-26T20:59:37.053 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/6-next-mon/monb.yaml 2024-04-26T20:59:37.053 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/8-next-mon 2024-04-26T20:59:37.054 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/8-next-mon/monc.yaml 2024-04-26T20:59:37.054 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/1-jewel-install 2024-04-26T20:59:37.054 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/1-jewel-install/jewel.yaml 2024-04-26T20:59:37.054 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/5-workload 2024-04-26T20:59:37.054 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/5-workload/readwrite.yaml 2024-04-26T20:59:37.054 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/5-workload/snaps-few-objects.yaml 2024-04-26T20:59:37.054 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/5-workload/rbd-cls.yaml 2024-04-26T20:59:37.054 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/5-workload/rbd-import-export.yaml 2024-04-26T20:59:37.054 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/5-workload/+ 2024-04-26T20:59:37.054 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/4-mon 2024-04-26T20:59:37.054 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/4-mon/mona.yaml 2024-04-26T20:59:37.054 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/% 2024-04-26T20:59:37.054 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/9-workload 2024-04-26T20:59:37.055 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/9-workload/rbd-python.yaml 2024-04-26T20:59:37.055 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/9-workload/rgw-swift.yaml 2024-04-26T20:59:37.055 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/9-workload/snaps-many-objects.yaml 2024-04-26T20:59:37.055 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/9-workload/+ 2024-04-26T20:59:37.055 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/2-partial-upgrade 2024-04-26T20:59:37.055 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/2-partial-upgrade/firsthalf.yaml 2024-04-26T20:59:37.055 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/3-thrash 2024-04-26T20:59:37.055 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/3-thrash/default.yaml 2024-04-26T20:59:37.055 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/7-workload 2024-04-26T20:59:37.055 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/7-workload/radosbench.yaml 2024-04-26T20:59:37.056 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/7-workload/rbd_api.yaml 2024-04-26T20:59:37.056 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/7-workload/+ 2024-04-26T20:59:37.056 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/0-cluster 2024-04-26T20:59:37.056 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/0-cluster/openstack.yaml 2024-04-26T20:59:37.056 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/0-cluster/+ 2024-04-26T20:59:37.056 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/0-cluster/start.yaml 2024-04-26T20:59:37.056 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash 2024-04-26T20:59:37.056 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/msgr 2024-04-26T20:59:37.056 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/clusters 2024-04-26T20:59:37.056 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/clusters/3-mons.yaml 2024-04-26T20:59:37.056 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/clusters/9-mons.yaml 2024-04-26T20:59:37.056 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/msgr-failures 2024-04-26T20:59:37.056 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/msgr-failures/mon-delay.yaml 2024-04-26T20:59:37.056 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/msgr-failures/few.yaml 2024-04-26T20:59:37.056 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/objectstore 2024-04-26T20:59:37.057 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/fs 2024-04-26T20:59:37.057 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/fs/xfs.yaml 2024-04-26T20:59:37.057 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/workloads 2024-04-26T20:59:37.057 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/workloads/pool-create-delete.yaml 2024-04-26T20:59:37.057 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/workloads/snaps-few-objects.yaml 2024-04-26T20:59:37.057 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/workloads/rados_mon_workunits.yaml 2024-04-26T20:59:37.057 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/workloads/rados_5925.yaml 2024-04-26T20:59:37.057 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/workloads/rados_api_tests.yaml 2024-04-26T20:59:37.057 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/ceph 2024-04-26T20:59:37.057 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/ceph/ceph.yaml 2024-04-26T20:59:37.057 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/% 2024-04-26T20:59:37.057 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/mon_kv_backend 2024-04-26T20:59:37.058 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/thrashers 2024-04-26T20:59:37.058 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/thrashers/many.yaml 2024-04-26T20:59:37.058 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/thrashers/one.yaml 2024-04-26T20:59:37.058 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/thrashers/sync.yaml 2024-04-26T20:59:37.058 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/thrashers/sync-many.yaml 2024-04-26T20:59:37.230 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/thrashers/force-sync-many.yaml 2024-04-26T20:59:37.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/monthrash/rados.yaml 2024-04-26T20:59:37.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash 2024-04-26T20:59:37.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/msgr 2024-04-26T20:59:37.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/clusters 2024-04-26T20:59:37.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/clusters/openstack.yaml 2024-04-26T20:59:37.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/clusters/fixed-2.yaml 2024-04-26T20:59:37.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/clusters/+ 2024-04-26T20:59:37.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/1-pg-log-overrides 2024-04-26T20:59:37.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/1-pg-log-overrides/normal_pg_log.yaml 2024-04-26T20:59:37.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/1-pg-log-overrides/short_pg_log.yaml 2024-04-26T20:59:37.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/msgr-failures 2024-04-26T20:59:37.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/msgr-failures/few.yaml 2024-04-26T20:59:37.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/msgr-failures/fastclose.yaml 2024-04-26T20:59:37.232 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/msgr-failures/osd-delay.yaml 2024-04-26T20:59:37.232 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/0-size-min-size-overrides 2024-04-26T20:59:37.232 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/0-size-min-size-overrides/3-size-2-min-size.yaml 2024-04-26T20:59:37.232 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/0-size-min-size-overrides/2-size-2-min-size.yaml 2024-04-26T20:59:37.232 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/0-size-min-size-overrides/2-size-1-min-size.yaml 2024-04-26T20:59:37.232 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/objectstore 2024-04-26T20:59:37.232 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/fs 2024-04-26T20:59:37.232 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/fs/btrfs.yaml 2024-04-26T20:59:37.232 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/fs/xfs.yaml 2024-04-26T20:59:37.232 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/workloads 2024-04-26T20:59:37.232 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/workloads/radosbench.yaml 2024-04-26T20:59:37.232 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/workloads/cache-agent-big.yaml 2024-04-26T20:59:37.232 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/workloads/cache.yaml 2024-04-26T20:59:37.232 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/workloads/cache-agent-small.yaml 2024-04-26T20:59:37.232 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/workloads/snaps-few-objects.yaml 2024-04-26T20:59:37.233 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/workloads/write_fadvise_dontneed.yaml 2024-04-26T20:59:37.233 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/workloads/cache-pool-snaps.yaml 2024-04-26T20:59:37.233 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/workloads/admin_socket_objecter_requests.yaml 2024-04-26T20:59:37.233 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/workloads/cache-pool-snaps-readproxy.yaml 2024-04-26T20:59:37.233 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/workloads/cache-snaps.yaml 2024-04-26T20:59:37.233 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/workloads/pool-snaps-few-objects.yaml 2024-04-26T20:59:37.233 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/workloads/rados_api_tests.yaml 2024-04-26T20:59:37.233 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/workloads/small-objects.yaml 2024-04-26T20:59:37.233 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/hobj-sort.yaml 2024-04-26T20:59:37.233 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/% 2024-04-26T20:59:37.233 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/thrashers 2024-04-26T20:59:37.233 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/thrashers/default.yaml 2024-04-26T20:59:37.233 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/thrashers/morepggrow.yaml 2024-04-26T20:59:37.233 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/thrashers/pggrow.yaml 2024-04-26T20:59:37.233 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/thrashers/mapgap.yaml 2024-04-26T20:59:37.233 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/rocksdb.yaml 2024-04-26T20:59:37.233 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash/rados.yaml 2024-04-26T20:59:37.234 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify 2024-04-26T20:59:37.234 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/msgr 2024-04-26T20:59:37.234 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/validater 2024-04-26T20:59:37.234 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/validater/valgrind.yaml 2024-04-26T20:59:37.234 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/validater/lockdep.yaml 2024-04-26T20:59:37.234 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/clusters 2024-04-26T20:59:37.234 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/clusters/openstack.yaml 2024-04-26T20:59:37.234 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/clusters/fixed-2.yaml 2024-04-26T20:59:37.234 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/clusters/+ 2024-04-26T20:59:37.234 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/msgr-failures 2024-04-26T20:59:37.234 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/msgr-failures/few.yaml 2024-04-26T20:59:37.234 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/objectstore 2024-04-26T20:59:37.234 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/fs 2024-04-26T20:59:37.234 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/fs/btrfs.yaml 2024-04-26T20:59:37.234 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/1thrash 2024-04-26T20:59:37.235 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/1thrash/default.yaml 2024-04-26T20:59:37.235 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/1thrash/none.yaml 2024-04-26T20:59:37.235 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/% 2024-04-26T20:59:37.235 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/mon_kv_backend 2024-04-26T20:59:37.235 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/tasks 2024-04-26T20:59:37.235 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/tasks/rados_cls_all.yaml 2024-04-26T20:59:37.235 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/tasks/mon_recovery.yaml 2024-04-26T20:59:37.235 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/tasks/rados_api_tests.yaml 2024-04-26T20:59:37.235 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/verify/rados.yaml 2024-04-26T20:59:37.235 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/multimon 2024-04-26T20:59:37.235 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/multimon/msgr 2024-04-26T20:59:37.425 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/multimon/clusters 2024-04-26T20:59:37.425 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/multimon/clusters/21.yaml 2024-04-26T20:59:37.425 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/multimon/clusters/9.yaml 2024-04-26T20:59:37.425 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/multimon/clusters/3.yaml 2024-04-26T20:59:37.425 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/multimon/clusters/6.yaml 2024-04-26T20:59:37.426 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/multimon/msgr-failures 2024-04-26T20:59:37.426 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/multimon/msgr-failures/many.yaml 2024-04-26T20:59:37.426 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/multimon/msgr-failures/few.yaml 2024-04-26T20:59:37.426 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/multimon/objectstore 2024-04-26T20:59:37.426 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/multimon/fs 2024-04-26T20:59:37.426 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/multimon/fs/xfs.yaml 2024-04-26T20:59:37.426 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/multimon/% 2024-04-26T20:59:37.426 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/multimon/mon_kv_backend 2024-04-26T20:59:37.426 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/multimon/tasks 2024-04-26T20:59:37.426 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/multimon/tasks/mon_clock_with_skews.yaml 2024-04-26T20:59:37.426 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/multimon/tasks/mon_recovery.yaml 2024-04-26T20:59:37.426 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/multimon/tasks/mon_clock_no_skews.yaml 2024-04-26T20:59:37.426 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/multimon/rados.yaml 2024-04-26T20:59:37.426 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-isa 2024-04-26T20:59:37.427 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/leveldb.yaml 2024-04-26T20:59:37.427 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/clusters 2024-04-26T20:59:37.427 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/msgr-failures 2024-04-26T20:59:37.427 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/objectstore 2024-04-26T20:59:37.427 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/supported 2024-04-26T20:59:37.427 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/fs 2024-04-26T20:59:37.427 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/workloads 2024-04-26T20:59:37.427 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/workloads/ec-rados-plugin=isa-k=2-m=1.yaml 2024-04-26T20:59:37.427 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/arch 2024-04-26T20:59:37.427 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/arch/x86_64.yaml 2024-04-26T20:59:37.427 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/% 2024-04-26T20:59:37.427 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/thrashers 2024-04-26T20:59:37.427 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/rados.yaml 2024-04-26T20:59:37.427 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-big 2024-04-26T20:59:37.427 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-big/cluster 2024-04-26T20:59:37.428 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-big/cluster/openstack.yaml 2024-04-26T20:59:37.428 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-big/cluster/12-osds.yaml 2024-04-26T20:59:37.428 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-big/cluster/+ 2024-04-26T20:59:37.428 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-big/leveldb.yaml 2024-04-26T20:59:37.428 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-big/msgr-failures 2024-04-26T20:59:37.428 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-big/objectstore 2024-04-26T20:59:37.428 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-big/fs 2024-04-26T20:59:37.428 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-big/workloads 2024-04-26T20:59:37.428 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-big/workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml 2024-04-26T20:59:37.428 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-big/% 2024-04-26T20:59:37.428 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-big/thrashers 2024-04-26T20:59:37.428 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-big/thrashers/default.yaml 2024-04-26T20:59:37.428 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-big/thrashers/fastread.yaml 2024-04-26T20:59:37.428 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-big/thrashers/morepggrow.yaml 2024-04-26T20:59:37.428 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-big/thrashers/pggrow.yaml 2024-04-26T20:59:37.429 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-big/thrashers/mapgap.yaml 2024-04-26T20:59:37.429 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/rados/thrash-erasure-code-big/rados.yaml 2024-04-26T20:59:37.429 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/knfs 2024-04-26T20:59:37.429 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/knfs/basic 2024-04-26T20:59:37.429 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/knfs/basic/clusters 2024-04-26T20:59:37.429 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/knfs/basic/clusters/extra-client.yaml 2024-04-26T20:59:37.429 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/knfs/basic/fs 2024-04-26T20:59:37.429 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/knfs/basic/fs/xfs.yaml 2024-04-26T20:59:37.429 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/knfs/basic/mount 2024-04-26T20:59:37.429 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/knfs/basic/mount/v4.yaml 2024-04-26T20:59:37.429 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/knfs/basic/mount/v3.yaml 2024-04-26T20:59:37.429 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/knfs/basic/ceph 2024-04-26T20:59:37.429 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/knfs/basic/ceph/base.yaml 2024-04-26T20:59:37.429 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/knfs/basic/% 2024-04-26T20:59:37.429 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/knfs/basic/tasks 2024-04-26T20:59:37.429 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/knfs/basic/tasks/nfs_workunit_suites_dbench.yaml 2024-04-26T20:59:37.430 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/knfs/basic/tasks/nfs-workunit-kernel-untar-build.yaml 2024-04-26T20:59:37.430 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/knfs/basic/tasks/nfs_workunit_suites_ffsb.yaml 2024-04-26T20:59:37.430 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/knfs/basic/tasks/nfs_workunit_suites_blogbench.yaml 2024-04-26T20:59:37.430 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/knfs/basic/tasks/nfs_workunit_misc.yaml 2024-04-26T20:59:37.803 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/knfs/basic/tasks/nfs_workunit_suites_fsstress.yaml 2024-04-26T20:59:37.803 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/knfs/basic/tasks/nfs_workunit_suites_iozone.yaml 2024-04-26T20:59:37.803 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/experimental 2024-04-26T20:59:37.803 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/experimental/multimds 2024-04-26T20:59:37.804 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/experimental/multimds/clusters 2024-04-26T20:59:37.804 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/experimental/multimds/clusters/7-multimds.yaml 2024-04-26T20:59:37.804 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/experimental/multimds/% 2024-04-26T20:59:37.804 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/experimental/multimds/tasks 2024-04-26T20:59:37.804 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/experimental/multimds/tasks/fsstress_thrash_subtrees.yaml 2024-04-26T20:59:37.804 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal 2024-04-26T20:59:37.804 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/fs-misc 2024-04-26T20:59:37.804 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/fs-misc/clusters 2024-04-26T20:59:37.804 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/fs-misc/clusters/two_clients.yaml 2024-04-26T20:59:37.804 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/fs-misc/fs 2024-04-26T20:59:37.804 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/fs-misc/fs/btrfs.yaml 2024-04-26T20:59:37.805 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/fs-misc/% 2024-04-26T20:59:37.805 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/fs-misc/tasks 2024-04-26T20:59:37.805 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/fs-misc/tasks/locktest.yaml 2024-04-26T20:59:37.805 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/basic 2024-04-26T20:59:37.805 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/basic/clusters 2024-04-26T20:59:37.805 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/basic/clusters/fixed-3.yaml 2024-04-26T20:59:37.805 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/basic/fs 2024-04-26T20:59:37.805 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/basic/fs/btrfs.yaml 2024-04-26T20:59:37.805 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/basic/% 2024-04-26T20:59:37.805 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/basic/tasks 2024-04-26T20:59:37.805 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/basic/tasks/kclient_workunit_suites_blogbench.yaml 2024-04-26T20:59:37.805 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/basic/tasks/kclient_workunit_suites_fsx.yaml 2024-04-26T20:59:37.805 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/mds_restart 2024-04-26T20:59:37.806 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/mds_restart/clusters 2024-04-26T20:59:37.806 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/mds_restart/clusters/one_mds.yaml 2024-04-26T20:59:37.806 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/mds_restart/% 2024-04-26T20:59:37.806 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/mds_restart/tasks 2024-04-26T20:59:37.806 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/mds_restart/tasks/restart-workunit-backtraces.yaml 2024-04-26T20:59:37.806 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds 2024-04-26T20:59:37.806 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds/clusters 2024-04-26T20:59:37.806 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds/clusters/3-node-3-mds.yaml 2024-04-26T20:59:37.806 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds/clusters/3-node-9-mds.yaml 2024-04-26T20:59:37.806 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds/fs 2024-04-26T20:59:37.806 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds/fs/btrfs.yaml 2024-04-26T20:59:37.807 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds/% 2024-04-26T20:59:37.807 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds/mounts 2024-04-26T20:59:37.807 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds/mounts/ceph-fuse.yaml 2024-04-26T20:59:37.807 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds/mounts/kclient.yaml 2024-04-26T20:59:37.807 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds/tasks 2024-04-26T20:59:37.807 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds/tasks/workunit_suites_pjd.yaml 2024-04-26T20:59:37.807 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds/tasks/workunit_suites_fsstress.yaml 2024-04-26T20:59:37.807 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds/tasks/workunit_suites_truncate_delay.yaml 2024-04-26T20:59:37.807 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds/tasks/workunit_misc.yaml 2024-04-26T20:59:37.807 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds/tasks/workunit_suites_blogbench.yaml 2024-04-26T20:59:37.807 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds/tasks/workunit_suites_fsync.yaml 2024-04-26T20:59:37.807 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds/tasks/workunit_suites_dbench.yaml 2024-04-26T20:59:37.808 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds/thrash 2024-04-26T20:59:37.808 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds/thrash/normal.yaml 2024-04-26T20:59:37.808 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/marginal/multimds/thrash/exports.yaml 2024-04-26T20:59:37.808 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs 2024-04-26T20:59:37.808 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs 2024-04-26T20:59:37.808 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/clusters 2024-04-26T20:59:37.808 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/clusters/fixed-3-cephfs.yaml 2024-04-26T20:59:37.808 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/fs 2024-04-26T20:59:37.808 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/fs/xfs.yaml 2024-04-26T20:59:37.808 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/inline 2024-04-26T20:59:37.808 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/inline/yes.yaml 2024-04-26T20:59:37.808 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/inline/no.yaml 2024-04-26T20:59:37.809 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/conf.yaml 2024-04-26T20:59:37.809 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/% 2024-04-26T20:59:37.809 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/tasks 2024-04-26T20:59:37.809 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_misc.yaml 2024-04-26T20:59:37.809 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_suites_fsstress.yaml 2024-04-26T20:59:37.809 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_trivial_sync.yaml 2024-04-26T20:59:37.809 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_snaps.yaml 2024-04-26T20:59:37.809 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_direct_io.yaml 2024-04-26T20:59:37.809 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_suites_fsx.yaml 2024-04-26T20:59:38.017 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_kernel_untar_build.yaml 2024-04-26T20:59:38.017 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_suites_ffsb.yaml 2024-04-26T20:59:38.017 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_suites_dbench.yaml 2024-04-26T20:59:38.017 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_suites_fsync.yaml 2024-04-26T20:59:38.017 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_suites_pjd.yaml 2024-04-26T20:59:38.017 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_suites_iozone.yaml 2024-04-26T20:59:38.017 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_o_trunc.yaml 2024-04-26T20:59:38.017 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/mixed-clients 2024-04-26T20:59:38.017 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/mixed-clients/clusters 2024-04-26T20:59:38.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/mixed-clients/clusters/2-clients.yaml 2024-04-26T20:59:38.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/mixed-clients/fs 2024-04-26T20:59:38.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/mixed-clients/fs/xfs.yaml 2024-04-26T20:59:38.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/mixed-clients/conf.yaml 2024-04-26T20:59:38.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/mixed-clients/% 2024-04-26T20:59:38.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/mixed-clients/tasks 2024-04-26T20:59:38.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/mixed-clients/tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml 2024-04-26T20:59:38.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/mixed-clients/tasks/kernel_cfuse_workunits_dbench_iozone.yaml 2024-04-26T20:59:38.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery 2024-04-26T20:59:38.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/dirfrag 2024-04-26T20:59:38.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/dirfrag/frag_enable.yaml 2024-04-26T20:59:38.018 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/clusters 2024-04-26T20:59:38.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/clusters/4-remote-clients.yaml 2024-04-26T20:59:38.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/% 2024-04-26T20:59:38.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/mounts 2024-04-26T20:59:38.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/mounts/kmounts.yaml 2024-04-26T20:59:38.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/xfs.yaml 2024-04-26T20:59:38.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/tasks 2024-04-26T20:59:38.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/tasks/backtrace.yaml 2024-04-26T20:59:38.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/tasks/forward-scrub.yaml 2024-04-26T20:59:38.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/tasks/volume-client.yaml 2024-04-26T20:59:38.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/tasks/config-commands.yaml 2024-04-26T20:59:38.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/tasks/sessionmap.yaml 2024-04-26T20:59:38.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/tasks/failover.yaml.disabled 2024-04-26T20:59:38.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/tasks/data-scan.yaml 2024-04-26T20:59:38.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/tasks/mds-flush.yaml 2024-04-26T20:59:38.019 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/tasks/auto-repair.yaml 2024-04-26T20:59:38.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/tasks/damage.yaml 2024-04-26T20:59:38.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/tasks/client-recovery.yaml 2024-04-26T20:59:38.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/tasks/journal-repair.yaml 2024-04-26T20:59:38.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/tasks/client-limits.yaml 2024-04-26T20:59:38.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/tasks/strays.yaml.disabled 2024-04-26T20:59:38.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/tasks/pool-perm.yaml 2024-04-26T20:59:38.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/debug 2024-04-26T20:59:38.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/recovery/debug/mds_client.yaml 2024-04-26T20:59:38.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/thrash 2024-04-26T20:59:38.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/thrash/clusters 2024-04-26T20:59:38.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/thrash/clusters/fixed-3-cephfs.yaml 2024-04-26T20:59:38.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/thrash/fs 2024-04-26T20:59:38.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/thrash/fs/xfs.yaml 2024-04-26T20:59:38.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/thrash/workloads 2024-04-26T20:59:38.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/thrash/workloads/kclient_workunit_suites_ffsb.yaml 2024-04-26T20:59:38.020 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/thrash/workloads/kclient_workunit_suites_iozone.yaml 2024-04-26T20:59:38.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/thrash/conf.yaml 2024-04-26T20:59:38.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/thrash/% 2024-04-26T20:59:38.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/thrash/thrashers 2024-04-26T20:59:38.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/thrash/thrashers/default.yaml 2024-04-26T20:59:38.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/thrash/thrashers/mon.yaml 2024-04-26T20:59:38.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/kcephfs/thrash/thrashers/mds.yaml 2024-04-26T20:59:38.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade 2024-04-26T20:59:38.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x 2024-04-26T20:59:38.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/point-to-point-x 2024-04-26T20:59:38.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/point-to-point-x/point-to-point-upgrade.yaml 2024-04-26T20:59:38.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/point-to-point-x/distros 2024-04-26T20:59:38.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/point-to-point-x/distros/centos_7.2.yaml 2024-04-26T20:59:38.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/point-to-point-x/distros/ubuntu_14.04.yaml 2024-04-26T20:59:38.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/point-to-point-x/% 2024-04-26T20:59:38.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel 2024-04-26T20:59:38.021 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/kraken.yaml 2024-04-26T20:59:38.182 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/1-jewel-install 2024-04-26T20:59:38.182 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/1-jewel-install/jewel.yaml 2024-04-26T20:59:38.182 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/4-kraken.yaml 2024-04-26T20:59:38.183 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/3-upgrade-sequence 2024-04-26T20:59:38.183 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/3-upgrade-sequence/upgrade-all.yaml 2024-04-26T20:59:38.183 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/3-upgrade-sequence/upgrade-mon-osd-mds.yaml 2024-04-26T20:59:38.183 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/distros 2024-04-26T20:59:38.183 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/% 2024-04-26T20:59:38.183 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/2-workload 2024-04-26T20:59:38.183 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/2-workload/test_rbd_python.yaml 2024-04-26T20:59:38.183 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/2-workload/blogbench.yaml 2024-04-26T20:59:38.183 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/2-workload/rados_api.yaml 2024-04-26T20:59:38.183 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/2-workload/rados_loadgenbig.yaml 2024-04-26T20:59:38.183 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/2-workload/ec-rados-default.yaml 2024-04-26T20:59:38.183 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/2-workload/+ 2024-04-26T20:59:38.183 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/2-workload/test_rbd_api.yaml 2024-04-26T20:59:38.183 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/0-cluster 2024-04-26T20:59:38.184 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/0-cluster/openstack.yaml 2024-04-26T20:59:38.184 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/0-cluster/+ 2024-04-26T20:59:38.184 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/0-cluster/start.yaml 2024-04-26T20:59:38.184 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/5-final-workload 2024-04-26T20:59:38.184 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/5-final-workload/blogbench.yaml 2024-04-26T20:59:38.184 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/5-final-workload/rados-snaps-few-objects.yaml 2024-04-26T20:59:38.184 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/5-final-workload/+ 2024-04-26T20:59:38.184 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/5-final-workload/rados_loadgenmix.yaml 2024-04-26T20:59:38.184 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/5-final-workload/rbd_cls.yaml 2024-04-26T20:59:38.184 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/5-final-workload/rados_mon_thrash.yaml 2024-04-26T20:59:38.184 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/5-final-workload/rgw_swift.yaml 2024-04-26T20:59:38.184 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/parallel/5-final-workload/rbd_import_export.yaml 2024-04-26T20:59:38.184 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split 2024-04-26T20:59:38.184 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/6-next-mon 2024-04-26T20:59:38.185 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/6-next-mon/monb.yaml 2024-04-26T20:59:38.185 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/8-next-mon 2024-04-26T20:59:38.185 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/8-next-mon/monc.yaml 2024-04-26T20:59:38.185 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/1-jewel-install 2024-04-26T20:59:38.185 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/1-jewel-install/jewel.yaml 2024-04-26T20:59:38.185 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/5-workload 2024-04-26T20:59:38.185 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/5-workload/readwrite.yaml 2024-04-26T20:59:38.185 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/5-workload/snaps-few-objects.yaml 2024-04-26T20:59:38.185 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/5-workload/rbd-cls.yaml 2024-04-26T20:59:38.185 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/5-workload/rbd-import-export.yaml 2024-04-26T20:59:38.185 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/5-workload/+ 2024-04-26T20:59:38.185 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/4-mon 2024-04-26T20:59:38.185 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/4-mon/mona.yaml 2024-04-26T20:59:38.185 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/distros 2024-04-26T20:59:38.186 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/% 2024-04-26T20:59:38.186 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/9-workload 2024-04-26T20:59:38.186 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/9-workload/rbd-python.yaml 2024-04-26T20:59:38.186 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/9-workload/rgw-swift.yaml 2024-04-26T20:59:38.186 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/9-workload/snaps-many-objects.yaml 2024-04-26T20:59:38.186 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/9-workload/+ 2024-04-26T20:59:38.186 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/2-partial-upgrade 2024-04-26T20:59:38.186 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/2-partial-upgrade/firsthalf.yaml 2024-04-26T20:59:38.186 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/3-thrash 2024-04-26T20:59:38.186 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/3-thrash/default.yaml 2024-04-26T20:59:38.186 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/7-workload 2024-04-26T20:59:38.186 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/7-workload/radosbench.yaml 2024-04-26T20:59:38.358 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/7-workload/rbd_api.yaml 2024-04-26T20:59:38.358 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/7-workload/+ 2024-04-26T20:59:38.358 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/0-cluster 2024-04-26T20:59:38.358 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/0-cluster/openstack.yaml 2024-04-26T20:59:38.358 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/0-cluster/+ 2024-04-26T20:59:38.358 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/0-cluster/start.yaml 2024-04-26T20:59:38.359 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64 2024-04-26T20:59:38.359 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/6-next-mon 2024-04-26T20:59:38.359 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/8-next-mon 2024-04-26T20:59:38.359 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/1-jewel-install 2024-04-26T20:59:38.359 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/5-workload 2024-04-26T20:59:38.359 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/5-workload/ec-rados-default.yaml 2024-04-26T20:59:38.359 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/4-mon 2024-04-26T20:59:38.359 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/% 2024-04-26T20:59:38.359 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/0-x86_64.yaml 2024-04-26T20:59:38.359 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/9-workload 2024-04-26T20:59:38.359 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/9-workload/ec-rados-plugin=jerasure-k=3-m=1.yaml 2024-04-26T20:59:38.359 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/2-partial-upgrade 2024-04-26T20:59:38.359 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/3-thrash 2024-04-26T20:59:38.359 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/3-thrash/default.yaml 2024-04-26T20:59:38.360 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/0-cluster 2024-04-26T20:59:38.360 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code 2024-04-26T20:59:38.360 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/6-next-mon 2024-04-26T20:59:38.360 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/8-next-mon 2024-04-26T20:59:38.360 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/1-jewel-install 2024-04-26T20:59:38.360 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/5-workload 2024-04-26T20:59:38.360 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/5-workload/ec-rados-default.yaml 2024-04-26T20:59:38.360 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/4-mon 2024-04-26T20:59:38.360 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/distros 2024-04-26T20:59:38.360 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/% 2024-04-26T20:59:38.360 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/9-workload 2024-04-26T20:59:38.360 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/9-workload/ec-rados-plugin=jerasure-k=3-m=1.yaml 2024-04-26T20:59:38.360 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/2-partial-upgrade 2024-04-26T20:59:38.360 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/3-thrash 2024-04-26T20:59:38.361 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/3-thrash/default.yaml 2024-04-26T20:59:38.361 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/0-cluster 2024-04-26T20:59:38.361 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x 2024-04-26T20:59:38.361 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel 2024-04-26T20:59:38.361 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/8-kraken.yaml 2024-04-26T20:59:38.361 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/1-hammer-jewel-install 2024-04-26T20:59:38.361 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/1-hammer-jewel-install/hammer-jewel.yaml 2024-04-26T20:59:38.361 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/4-jewel.yaml 2024-04-26T20:59:38.361 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/3.5-finish.yaml 2024-04-26T20:59:38.361 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/3-upgrade-sequence 2024-04-26T20:59:38.361 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/3-upgrade-sequence/upgrade-osd-mds-mon.yaml 2024-04-26T20:59:38.361 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/3-upgrade-sequence/upgrade-all.yaml 2024-04-26T20:59:38.361 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/7-upgrade-sequence 2024-04-26T20:59:38.362 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/7-upgrade-sequence/upgrade-by-daemon.yaml 2024-04-26T20:59:38.362 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/7-upgrade-sequence/upgrade-all.yaml 2024-04-26T20:59:38.362 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/distros 2024-04-26T20:59:38.362 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/distros/centos_7.2.yaml 2024-04-26T20:59:38.362 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/distros/ubuntu_14.04.yaml 2024-04-26T20:59:38.523 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/% 2024-04-26T20:59:38.523 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/9-final-workload 2024-04-26T20:59:38.523 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/9-final-workload/rados-snaps-few-objects.yaml 2024-04-26T20:59:38.523 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/9-final-workload/+ 2024-04-26T20:59:38.523 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/9-final-workload/rados_loadgenmix.yaml 2024-04-26T20:59:38.523 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/9-final-workload/rbd_cls.yaml 2024-04-26T20:59:38.523 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/9-final-workload/rados_mon_thrash.yaml 2024-04-26T20:59:38.523 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/9-final-workload/rbd_import_export.yaml 2024-04-26T20:59:38.523 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/9-final-workload/rgw_s3tests.yaml 2024-04-26T20:59:38.524 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/2-workload 2024-04-26T20:59:38.524 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/2-workload/test_rbd_python.yaml 2024-04-26T20:59:38.524 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/2-workload/rados_api.yaml 2024-04-26T20:59:38.524 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/2-workload/rados_loadgenbig.yaml 2024-04-26T20:59:38.524 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/2-workload/ec-rados-default.yaml 2024-04-26T20:59:38.524 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/2-workload/+ 2024-04-26T20:59:38.524 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/2-workload/test_rbd_api.yaml 2024-04-26T20:59:38.524 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/6-workload 2024-04-26T20:59:38.524 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/6-workload/test_rbd_python.yaml 2024-04-26T20:59:38.524 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/6-workload/rados_api.yaml 2024-04-26T20:59:38.524 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/6-workload/rados_loadgenbig.yaml 2024-04-26T20:59:38.524 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/6-workload/ec-rados-default.yaml 2024-04-26T20:59:38.525 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/6-workload/+ 2024-04-26T20:59:38.525 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/6-workload/test_rbd_api.yaml 2024-04-26T20:59:38.525 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/5-hammer-jewel-x-upgrade 2024-04-26T20:59:38.525 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/5-hammer-jewel-x-upgrade/hammer-jewel-x.yaml 2024-04-26T20:59:38.525 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/0-cluster 2024-04-26T20:59:38.525 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/0-cluster/start.yaml 2024-04-26T20:59:38.525 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split 2024-04-26T20:59:38.525 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/6-next-mon 2024-04-26T20:59:38.525 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/8-next-mon 2024-04-26T20:59:38.525 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/5-workload 2024-04-26T20:59:38.525 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/4-mon 2024-04-26T20:59:38.525 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/distros 2024-04-26T20:59:38.525 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/distros/centos_7.2.yaml 2024-04-26T20:59:38.525 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/distros/ubuntu_14.04.yaml 2024-04-26T20:59:38.526 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/% 2024-04-26T20:59:38.526 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/9-workload 2024-04-26T20:59:38.526 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/1-hammer-install-and-upgrade-to-jewel 2024-04-26T20:59:38.526 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/1-hammer-install-and-upgrade-to-jewel/hammer-to-jewel.yaml 2024-04-26T20:59:38.526 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/2-partial-upgrade 2024-04-26T20:59:38.526 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/3-thrash 2024-04-26T20:59:38.526 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/7-workload 2024-04-26T20:59:38.526 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/0-cluster 2024-04-26T20:59:38.526 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade 2024-04-26T20:59:38.526 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x 2024-04-26T20:59:38.526 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/rbd 2024-04-26T20:59:38.526 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/rbd/1-install 2024-04-26T20:59:38.526 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/rbd/1-install/hammer-client-x.yaml 2024-04-26T20:59:38.526 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/rbd/% 2024-04-26T20:59:38.527 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/rbd/2-workload 2024-04-26T20:59:38.527 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/rbd/2-workload/rbd_notification_tests.yaml 2024-04-26T20:59:38.527 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/rbd/0-cluster 2024-04-26T20:59:38.886 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/rbd/0-cluster/start.yaml 2024-04-26T20:59:38.886 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/basic 2024-04-26T20:59:38.886 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/basic/1-install 2024-04-26T20:59:38.887 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/basic/1-install/hammer-client-x.yaml 2024-04-26T20:59:38.887 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/basic/% 2024-04-26T20:59:38.887 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/basic/2-workload 2024-04-26T20:59:38.887 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/basic/2-workload/rbd_cli_import_export.yaml 2024-04-26T20:59:38.887 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/basic/2-workload/rbd_api_tests.yaml 2024-04-26T20:59:38.887 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/basic/0-cluster 2024-04-26T20:59:38.887 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/basic/0-cluster/start.yaml 2024-04-26T20:59:38.887 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x 2024-04-26T20:59:38.887 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/rbd 2024-04-26T20:59:38.887 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/rbd/1-install 2024-04-26T20:59:38.887 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/rbd/1-install/jewel-client-x.yaml 2024-04-26T20:59:38.887 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/rbd/% 2024-04-26T20:59:38.887 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/rbd/2-workload 2024-04-26T20:59:38.888 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/rbd/2-workload/rbd_notification_tests.yaml 2024-04-26T20:59:38.888 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/rbd/0-cluster 2024-04-26T20:59:38.888 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/rbd/0-cluster/start.yaml 2024-04-26T20:59:38.888 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/basic 2024-04-26T20:59:38.888 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/basic/1-install 2024-04-26T20:59:38.888 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/basic/1-install/jewel-client-x.yaml 2024-04-26T20:59:38.888 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/basic/% 2024-04-26T20:59:38.888 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/basic/2-workload 2024-04-26T20:59:38.888 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/basic/2-workload/rbd_cli_import_export.yaml 2024-04-26T20:59:38.888 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/basic/2-workload/rbd_api_tests.yaml 2024-04-26T20:59:38.888 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/basic/0-cluster 2024-04-26T20:59:38.888 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/basic/0-cluster/start.yaml 2024-04-26T20:59:38.888 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/dummy 2024-04-26T20:59:38.889 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/dummy/% 2024-04-26T20:59:38.889 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/dummy/all 2024-04-26T20:59:38.889 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/dummy/all/nop.yaml 2024-04-26T20:59:38.889 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/hadoop 2024-04-26T20:59:38.889 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/hadoop/basic 2024-04-26T20:59:38.889 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/hadoop/basic/clusters 2024-04-26T20:59:38.889 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/hadoop/basic/clusters/fixed-3.yaml 2024-04-26T20:59:38.889 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/hadoop/basic/% 2024-04-26T20:59:38.889 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/hadoop/basic/xfs.yaml 2024-04-26T20:59:38.889 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/hadoop/basic/tasks 2024-04-26T20:59:38.889 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/hadoop/basic/tasks/wordcount.yaml 2024-04-26T20:59:38.889 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/hadoop/basic/tasks/repl.yaml 2024-04-26T20:59:38.889 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/hadoop/basic/tasks/terasort.yaml 2024-04-26T20:59:38.889 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology 2024-04-26T20:59:38.890 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/integration.yaml 2024-04-26T20:59:38.890 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/nop 2024-04-26T20:59:38.890 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/nop/% 2024-04-26T20:59:38.890 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/nop/all 2024-04-26T20:59:38.890 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/nop/all/nop.yaml 2024-04-26T20:59:38.890 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/rgw 2024-04-26T20:59:38.890 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/rgw/distros 2024-04-26T20:59:38.890 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/rgw/% 2024-04-26T20:59:38.890 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/rgw/tasks 2024-04-26T20:59:38.890 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/rgw/tasks/s3tests-fastcgi.yaml 2024-04-26T20:59:38.890 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/rgw/tasks/s3tests-fcgi.yaml 2024-04-26T20:59:38.890 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/rgw/tasks/s3tests-civetweb.yaml 2024-04-26T20:59:38.890 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/no-ceph 2024-04-26T20:59:38.890 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/no-ceph/clusters 2024-04-26T20:59:38.891 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/no-ceph/clusters/single.yaml 2024-04-26T20:59:38.891 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/no-ceph/% 2024-04-26T20:59:38.891 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/no-ceph/tasks 2024-04-26T20:59:38.891 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/no-ceph/tasks/teuthology.yaml 2024-04-26T20:59:38.891 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/ceph 2024-04-26T20:59:38.891 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/ceph/clusters 2024-04-26T20:59:38.891 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/ceph/clusters/single.yaml 2024-04-26T20:59:38.891 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/ceph/distros 2024-04-26T20:59:38.891 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/ceph/% 2024-04-26T20:59:38.891 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/ceph/tasks 2024-04-26T20:59:39.106 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/ceph/tasks/teuthology.yaml 2024-04-26T20:59:39.106 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/buildpackages 2024-04-26T20:59:39.106 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/buildpackages/distros 2024-04-26T20:59:39.107 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/buildpackages/% 2024-04-26T20:59:39.107 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/buildpackages/tasks 2024-04-26T20:59:39.107 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/buildpackages/tasks/default.yaml 2024-04-26T20:59:39.107 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/buildpackages/tasks/tag.yaml 2024-04-26T20:59:39.107 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/buildpackages/tasks/branch.yaml 2024-04-26T20:59:39.107 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/multi-cluster 2024-04-26T20:59:39.107 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/multi-cluster/fs 2024-04-26T20:59:39.107 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/multi-cluster/fs/xfs.yaml 2024-04-26T20:59:39.107 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/multi-cluster/% 2024-04-26T20:59:39.107 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/multi-cluster/all 2024-04-26T20:59:39.107 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/multi-cluster/all/thrashosds.yaml 2024-04-26T20:59:39.107 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/multi-cluster/all/ceph.yaml 2024-04-26T20:59:39.107 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/multi-cluster/all/upgrade.yaml 2024-04-26T20:59:39.107 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/multi-cluster/all/workunit.yaml 2024-04-26T20:59:39.108 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/workunits 2024-04-26T20:59:39.108 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/teuthology/workunits/yes.yaml 2024-04-26T20:59:39.108 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds 2024-04-26T20:59:39.108 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/basic 2024-04-26T20:59:39.108 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/basic/overrides 2024-04-26T20:59:39.108 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/basic/overrides/fuse-default-perm-no.yaml 2024-04-26T20:59:39.108 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/basic/overrides/basic 2024-04-26T20:59:39.108 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/basic/overrides/% 2024-04-26T20:59:39.108 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/basic/clusters 2024-04-26T20:59:39.108 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/basic/clusters/9-mds.yaml 2024-04-26T20:59:39.108 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/basic/clusters/3-mds.yaml 2024-04-26T20:59:39.108 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/basic/fs 2024-04-26T20:59:39.108 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/basic/mount 2024-04-26T20:59:39.108 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/basic/mount/kclient.yaml 2024-04-26T20:59:39.109 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/basic/mount/fuse.yaml 2024-04-26T20:59:39.109 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/basic/inline 2024-04-26T20:59:39.109 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/basic/begin.yaml 2024-04-26T20:59:39.109 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/basic/% 2024-04-26T20:59:39.109 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/basic/tasks 2024-04-26T20:59:39.109 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/verify 2024-04-26T20:59:39.109 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/verify/validater 2024-04-26T20:59:39.109 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/verify/overrides 2024-04-26T20:59:39.109 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/verify/overrides/fuse-default-perm-no.yaml 2024-04-26T20:59:39.109 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/verify/overrides/% 2024-04-26T20:59:39.109 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/verify/overrides/verify 2024-04-26T20:59:39.109 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/verify/clusters 2024-04-26T20:59:39.109 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/verify/clusters/9-mds.yaml 2024-04-26T20:59:39.109 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/verify/clusters/3-mds.yaml 2024-04-26T20:59:39.110 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/verify/fs 2024-04-26T20:59:39.110 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/verify/mount 2024-04-26T20:59:39.110 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/verify/mount/kclient.yaml 2024-04-26T20:59:39.110 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/verify/mount/fuse.yaml 2024-04-26T20:59:39.110 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/verify/begin.yaml 2024-04-26T20:59:39.110 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/verify/% 2024-04-26T20:59:39.110 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/multimds/verify/tasks 2024-04-26T20:59:39.110 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke 2024-04-26T20:59:39.110 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/1node 2024-04-26T20:59:39.110 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/1node/clusters 2024-04-26T20:59:39.110 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/1node/clusters/openstack.yaml 2024-04-26T20:59:39.110 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/1node/clusters/+ 2024-04-26T20:59:39.110 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/1node/clusters/fixed-1.yaml 2024-04-26T20:59:39.110 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/1node/% 2024-04-26T20:59:39.111 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/1node/tasks 2024-04-26T20:59:39.111 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/1node/tasks/ceph-deploy.yaml 2024-04-26T20:59:39.111 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/systemd 2024-04-26T20:59:39.111 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/systemd/distro 2024-04-26T20:59:39.111 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/systemd/distro/ubuntu_16.04.yaml 2024-04-26T20:59:39.111 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/systemd/distro/centos_7.2.yaml 2024-04-26T20:59:39.111 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/systemd/% 2024-04-26T20:59:39.111 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/systemd/tasks 2024-04-26T20:59:39.111 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/systemd/tasks/systemd.yaml 2024-04-26T20:59:39.111 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic 2024-04-26T20:59:39.111 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/clusters 2024-04-26T20:59:39.111 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/clusters/fixed-3-cephfs.yaml 2024-04-26T20:59:39.111 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/fs 2024-04-26T20:59:39.111 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/fs/btrfs.yaml 2024-04-26T20:59:39.111 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/% 2024-04-26T20:59:39.112 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks 2024-04-26T20:59:39.112 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/libcephfs_interface_tests.yaml 2024-04-26T20:59:39.112 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/rados_cache_snaps.yaml 2024-04-26T20:59:39.112 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/rados_cls_all.yaml 2024-04-26T20:59:39.112 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/rbd_cli_import_export.yaml 2024-04-26T20:59:39.203 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/kclient_workunit_suites_fsstress.yaml 2024-04-26T20:59:39.203 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/cfuse_workunit_suites_pjd.yaml 2024-04-26T20:59:39.203 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/mon_thrash.yaml 2024-04-26T20:59:39.203 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/kclient_workunit_direct_io.yaml 2024-04-26T20:59:39.203 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/rbd_fsx.yaml 2024-04-26T20:59:39.203 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/rbd_python_api_tests.yaml 2024-04-26T20:59:39.203 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/rbd_workunit_suites_iozone.yaml 2024-04-26T20:59:39.203 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/kclient_workunit_suites_dbench.yaml 2024-04-26T20:59:39.204 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/rbd_api_tests.yaml 2024-04-26T20:59:39.204 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/cfuse_workunit_suites_blogbench.yaml 2024-04-26T20:59:39.204 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/rados_ec_snaps.yaml 2024-04-26T20:59:39.204 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/rgw_ec_s3tests.yaml 2024-04-26T20:59:39.204 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/kclient_workunit_suites_pjd.yaml 2024-04-26T20:59:39.204 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/rados_workunit_loadgen_mix.yaml 2024-04-26T20:59:39.204 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/rados_python.yaml 2024-04-26T20:59:39.204 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/rgw_swift.yaml 2024-04-26T20:59:39.204 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/rados_bench.yaml 2024-04-26T20:59:39.204 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/rados_api_tests.yaml 2024-04-26T20:59:39.204 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/cfuse_workunit_suites_fsstress.yaml 2024-04-26T20:59:39.204 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/rgw_s3tests.yaml 2024-04-26T20:59:39.204 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/suites/smoke/basic/tasks/cfuse_workunit_suites_iozone.yaml 2024-04-26T20:59:39.204 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/machine_types 2024-04-26T20:59:39.205 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/machine_types/vps.yaml 2024-04-26T20:59:39.205 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/machine_types/schedule_rados_ovh.sh 2024-04-26T20:59:39.205 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/machine_types/schedule_rados.sh 2024-04-26T20:59:39.205 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/mon_kv_backend 2024-04-26T20:59:39.205 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/mon_kv_backend/leveldb.yaml 2024-04-26T20:59:39.205 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/mon_kv_backend/rocksdb.yaml 2024-04-26T20:59:39.205 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/README 2024-04-26T20:59:39.205 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks 2024-04-26T20:59:39.205 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/rebuild_mondb.py 2024-04-26T20:59:39.205 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/workunit.py 2024-04-26T20:59:39.205 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/osd_backfill.py 2024-04-26T20:59:39.205 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/calamari_setup.py 2024-04-26T20:59:39.206 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/object_source_down.py 2024-04-26T20:59:39.206 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/populate_rbd_pool.py 2024-04-26T20:59:39.206 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/util 2024-04-26T20:59:39.206 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/util/rgw.py 2024-04-26T20:59:39.206 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/util/__init__.py 2024-04-26T20:59:39.206 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/util/rados.py 2024-04-26T20:59:39.206 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/util/test 2024-04-26T20:59:39.206 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/util/test/__init__.py 2024-04-26T20:59:39.206 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/util/test/test_rados.py 2024-04-26T20:59:39.206 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/osd_failsafe_enospc.py 2024-04-26T20:59:39.206 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/osd_recovery.py 2024-04-26T20:59:39.206 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/die_on_err.py 2024-04-26T20:59:39.206 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/calamari_nosetests.py 2024-04-26T20:59:39.206 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/admin_socket.py 2024-04-26T20:59:39.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/mgr 2024-04-26T20:59:39.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/mgr/__init__.py 2024-04-26T20:59:39.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/mgr/test_failover.py 2024-04-26T20:59:39.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/mgr/mgr_test_case.py 2024-04-26T20:59:39.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/thrash_pool_snaps.py 2024-04-26T20:59:39.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/userdata_teardown.yaml 2024-04-26T20:59:39.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/radosbenchsweep.py 2024-04-26T20:59:39.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/mds_thrash.py 2024-04-26T20:59:39.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/rbd.py 2024-04-26T20:59:39.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/rgw.py 2024-04-26T20:59:39.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/teuthology_integration.py 2024-04-26T20:59:39.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/mon_seesaw.py 2024-04-26T20:59:39.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/repair_test.py 2024-04-26T20:59:39.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/watch_notify_same_primary.py 2024-04-26T20:59:39.207 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/peer.py 2024-04-26T20:59:39.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/restart.py 2024-04-26T20:59:39.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/ceph.py 2024-04-26T20:59:39.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/tests 2024-04-26T20:59:39.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/tests/test_buildpackages.py 2024-04-26T20:59:39.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/tests/test_radosgw_admin.py 2024-04-26T20:59:39.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/tests/test_devstack.py 2024-04-26T20:59:39.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/tests/__init__.py 2024-04-26T20:59:39.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/rgw_logsocket.py 2024-04-26T20:59:39.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/radosgw_admin.py 2024-04-26T20:59:39.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/ceph_client.py 2024-04-26T20:59:39.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/__init__.py 2024-04-26T20:59:39.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/aver.py 2024-04-26T20:59:39.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cram.py 2024-04-26T20:59:39.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs 2024-04-26T20:59:39.208 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_misc.py 2024-04-26T20:59:39.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_client_limits.py 2024-04-26T20:59:39.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_pool_perm.py 2024-04-26T20:59:39.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_data_scan.py 2024-04-26T20:59:39.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_client_recovery.py 2024-04-26T20:59:39.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_journal_migration.py 2024-04-26T20:59:39.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_dump_tree.py 2024-04-26T20:59:39.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_mantle.py 2024-04-26T20:59:39.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_sessionmap.py 2024-04-26T20:59:39.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/__init__.py 2024-04-26T20:59:39.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_journal_repair.py 2024-04-26T20:59:39.209 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/kernel_mount.py 2024-04-26T20:59:39.224 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_cap_flush.py 2024-04-26T20:59:39.224 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_readahead.py 2024-04-26T20:59:39.224 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_backtrace.py 2024-04-26T20:59:39.225 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_auto_repair.py 2024-04-26T20:59:39.225 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_strays.py 2024-04-26T20:59:39.225 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_volume_client.py 2024-04-26T20:59:39.225 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/cephfs_test_case.py 2024-04-26T20:59:39.225 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_fragment.py 2024-04-26T20:59:39.225 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_failover.py 2024-04-26T20:59:39.225 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_full.py 2024-04-26T20:59:39.225 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/filesystem.py 2024-04-26T20:59:39.225 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/mount.py 2024-04-26T20:59:39.225 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_damage.py 2024-04-26T20:59:39.226 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_scrub_checks.py 2024-04-26T20:59:39.226 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_config_commands.py 2024-04-26T20:59:39.226 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_flush.py 2024-04-26T20:59:39.226 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/fuse_mount.py 2024-04-26T20:59:39.226 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs/test_forward_scrub.py 2024-04-26T20:59:39.226 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/recovery_bench.py 2024-04-26T20:59:39.226 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/multibench.py 2024-04-26T20:59:39.226 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/devstack.py 2024-04-26T20:59:39.226 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cephfs_test_runner.py 2024-04-26T20:59:39.226 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/filestore_idempotent.py 2024-04-26T20:59:39.226 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/mod_proxy_fcgi.tcp.conf.template 2024-04-26T20:59:39.226 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/scrub.py 2024-04-26T20:59:39.226 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/divergent_priors.py 2024-04-26T20:59:39.226 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/samba.py 2024-04-26T20:59:39.227 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/watch_notify_stress.py 2024-04-26T20:59:39.227 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/boto.cfg.template 2024-04-26T20:59:39.227 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/buildpackages.py 2024-04-26T20:59:39.227 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/ceph_test_case.py 2024-04-26T20:59:39.227 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/s3roundtrip.py 2024-04-26T20:59:39.227 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/qemu.py 2024-04-26T20:59:39.227 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/locktest.py 2024-04-26T20:59:39.227 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/manypools.py 2024-04-26T20:59:39.227 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/mds_creation_failure.py 2024-04-26T20:59:39.227 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/apache.conf.template 2024-04-26T20:59:39.227 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/ec_lost_unfound.py 2024-04-26T20:59:39.227 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/buildpackages 2024-04-26T20:59:39.227 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/buildpackages/ubuntu-16.04-user-data.txt 2024-04-26T20:59:39.227 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/buildpackages/ubuntu-14.04-user-data.txt 2024-04-26T20:59:39.227 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/buildpackages/centos-6.5-user-data.txt 2024-04-26T20:59:39.228 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/buildpackages/centos-7.1-user-data.txt 2024-04-26T20:59:39.228 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/buildpackages/make-deb.sh 2024-04-26T20:59:39.228 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/buildpackages/centos-7.0-user-data.txt 2024-04-26T20:59:39.228 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/buildpackages/make-rpm.sh 2024-04-26T20:59:39.228 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/buildpackages/opensuse-42.1-user-data.txt 2024-04-26T20:59:39.228 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/buildpackages/Makefile 2024-04-26T20:59:39.228 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/buildpackages/ubuntu-12.04-user-data.txt 2024-04-26T20:59:39.228 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/buildpackages/common.sh 2024-04-26T20:59:39.228 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/buildpackages/user-data.txt 2024-04-26T20:59:39.228 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/buildpackages/debian-8.0-user-data.txt 2024-04-26T20:59:39.228 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/buildpackages/centos-7.2-user-data.txt 2024-04-26T20:59:39.228 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/rbd_mirror.py 2024-04-26T20:59:39.228 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/dump_stuck.py 2024-04-26T20:59:39.229 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/ceph_objectstore_tool.py 2024-04-26T20:59:39.229 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/rbd_fsx.py 2024-04-26T20:59:39.229 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/radosgw_agent.py 2024-04-26T20:59:39.229 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/tgt.py 2024-04-26T20:59:39.229 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/vstart_runner.py 2024-04-26T20:59:39.229 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/resolve_stuck_peering.py 2024-04-26T20:59:39.229 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/rados.py 2024-04-26T20:59:39.229 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/s3tests.py 2024-04-26T20:59:39.229 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/userdata_setup.yaml 2024-04-26T20:59:39.229 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/lost_unfound.py 2024-04-26T20:59:39.229 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/s3readwrite.py 2024-04-26T20:59:39.229 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/ceph_fuse.py 2024-04-26T20:59:39.229 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/rest_api.py 2024-04-26T20:59:39.229 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/reg11184.py 2024-04-26T20:59:39.229 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/rep_lost_unfound_delete.py 2024-04-26T20:59:39.230 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/mod_proxy_fcgi.uds.conf.template 2024-04-26T20:59:39.230 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/ceph_deploy.py 2024-04-26T20:59:39.230 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/ceph_manager.py 2024-04-26T20:59:39.230 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/radosbench.py 2024-04-26T20:59:39.230 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/scrub_test.py 2024-04-26T20:59:39.230 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/mon_clock_skew_check.py 2024-04-26T20:59:39.230 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/kclient.py 2024-04-26T20:59:39.230 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/omapbench.py 2024-04-26T20:59:39.230 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/autotest.py 2024-04-26T20:59:39.230 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/mod_fastcgi.conf.template 2024-04-26T20:59:39.230 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/peering_speed_test.py 2024-04-26T20:59:39.230 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/mon_recovery.py 2024-04-26T20:59:39.230 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/cifs_mount.py 2024-04-26T20:59:39.230 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/rbd_fio.py 2024-04-26T20:59:39.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/logrotate.conf 2024-04-26T20:59:39.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/mon_thrash.py 2024-04-26T20:59:39.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/metadata.yaml 2024-04-26T20:59:39.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/divergent_priors2.py 2024-04-26T20:59:39.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/blktrace.py 2024-04-26T20:59:39.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/thrashosds.py 2024-04-26T20:59:39.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/systemd.py 2024-04-26T20:59:39.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/radosgw_admin_rest.py 2024-04-26T20:59:39.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/tasks/create_verify_lfn_objects.py 2024-04-26T20:59:39.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/timezone 2024-04-26T20:59:39.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/timezone/random.yaml 2024-04-26T20:59:39.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/timezone/eastern.yaml 2024-04-26T20:59:39.231 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/timezone/pacific.yaml 2024-04-26T20:59:39.236 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/.gitignore 2024-04-26T20:59:39.236 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/debug 2024-04-26T20:59:39.236 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/debug/openstack-30G.yaml 2024-04-26T20:59:39.236 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/debug/buildpackages.yaml 2024-04-26T20:59:39.236 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/debug/mds_client.yaml 2024-04-26T20:59:39.236 INFO:teuthology.orchestra.run.smithi028.stdout:./ceph-qa-suite/debug/openstack-15G.yaml 2024-04-26T20:59:39.239 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:39.239 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:39.239 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:39.239 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:39.239 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-26T20:59:39.240 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:39.391 INFO:teuthology.orchestra.run.smithi028.stdout:1 2024-04-26T20:59:39.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:39.405 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:39.405 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:39.405 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:39.405 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:39.405 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite").st_ino) 2024-04-26T20:59:39.405 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:39.599 INFO:teuthology.orchestra.run.smithi028.stdout:1099511627776 2024-04-26T20:59:39.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:39.614 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:39.614 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:39.614 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:39.614 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:39.615 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/erasure-code").st_ino) 2024-04-26T20:59:39.615 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:39.811 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628424 2024-04-26T20:59:39.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:39.826 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:39.826 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:39.826 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:39.826 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:39.826 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/erasure-code/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml").st_ino) 2024-04-26T20:59:39.826 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:40.022 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628432 2024-04-26T20:59:40.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:40.037 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:40.037 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:40.037 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:40.037 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:40.038 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/erasure-code/ec-rados-plugin=jerasure-k=3-m=1.yaml").st_ino) 2024-04-26T20:59:40.038 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:40.229 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628431 2024-04-26T20:59:40.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:40.244 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:40.244 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:40.244 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:40.244 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:40.244 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/erasure-code/ec-rados-plugin=isa-k=2-m=1.yaml").st_ino) 2024-04-26T20:59:40.244 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:40.439 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628429 2024-04-26T20:59:40.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:40.455 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:40.455 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:40.455 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:40.455 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:40.455 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/erasure-code/ec-rados-plugin=jerasure-k=2-m=1.yaml").st_ino) 2024-04-26T20:59:40.455 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:40.643 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628430 2024-04-26T20:59:40.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:40.657 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:40.658 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:40.658 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:40.658 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:40.658 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/erasure-code/ec-rados-plugin=shec-k=4-m=3-c=2.yaml").st_ino) 2024-04-26T20:59:40.658 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:40.853 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628433 2024-04-26T20:59:40.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:40.868 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:40.868 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:40.868 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:40.868 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:40.868 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/erasure-code/ec-feature-plugins-v3.yaml").st_ino) 2024-04-26T20:59:40.868 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:41.067 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628426 2024-04-26T20:59:41.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:41.081 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:41.081 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:41.081 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:41.081 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:41.082 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/erasure-code/ec-rados-default.yaml").st_ino) 2024-04-26T20:59:41.082 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:41.277 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628427 2024-04-26T20:59:41.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:41.292 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:41.292 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:41.292 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:41.293 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:41.293 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/erasure-code/ec-rados-sequential.yaml").st_ino) 2024-04-26T20:59:41.293 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:41.489 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628434 2024-04-26T20:59:41.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:41.505 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:41.505 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:41.505 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:41.505 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:41.505 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/erasure-code/ec-rados-parallel.yaml").st_ino) 2024-04-26T20:59:41.505 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:41.701 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628428 2024-04-26T20:59:41.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:41.715 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:41.715 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:41.715 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:41.715 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:41.716 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/erasure-code/ec-feature-plugins-v2.yaml").st_ino) 2024-04-26T20:59:41.716 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:41.912 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628425 2024-04-26T20:59:41.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:41.925 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:41.925 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:41.925 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:41.925 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:41.926 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/releases").st_ino) 2024-04-26T20:59:41.926 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:42.124 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628459 2024-04-26T20:59:42.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:42.140 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:42.140 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:42.140 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:42.140 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:42.140 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/releases/jewel.yaml").st_ino) 2024-04-26T20:59:42.141 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:42.321 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628461 2024-04-26T20:59:42.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:42.334 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:42.335 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:42.335 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:42.335 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:42.335 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/releases/kraken.yaml").st_ino) 2024-04-26T20:59:42.335 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:42.529 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628462 2024-04-26T20:59:42.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:42.544 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:42.544 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:42.544 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:42.544 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:42.544 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/releases/infernalis.yaml").st_ino) 2024-04-26T20:59:42.544 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:42.739 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628460 2024-04-26T20:59:42.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:42.754 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:42.754 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:42.754 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:42.755 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:42.755 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tox.ini").st_ino) 2024-04-26T20:59:42.755 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:42.949 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630232 2024-04-26T20:59:42.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:42.963 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:42.963 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:42.963 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:42.963 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:42.964 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/ceph-deploy-overrides").st_ino) 2024-04-26T20:59:42.964 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:43.159 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628345 2024-04-26T20:59:43.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:43.174 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:43.174 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:43.174 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:43.174 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:43.174 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/ceph-deploy-overrides/enable_dmcrypt_diff_journal_disk.yaml").st_ino) 2024-04-26T20:59:43.174 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:43.371 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628349 2024-04-26T20:59:43.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:43.385 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:43.386 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:43.386 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:43.386 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:43.386 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/ceph-deploy-overrides/ceph_deploy_dmcrypt.yaml").st_ino) 2024-04-26T20:59:43.386 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:43.581 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628346 2024-04-26T20:59:43.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:43.595 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:43.595 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:43.595 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:43.596 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:43.596 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/ceph-deploy-overrides/enable_diff_journal_disk.yaml").st_ino) 2024-04-26T20:59:43.596 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:43.794 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628348 2024-04-26T20:59:43.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:43.809 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:43.809 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:43.809 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:43.809 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:43.809 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/ceph-deploy-overrides/disable_diff_journal_disk.yaml").st_ino) 2024-04-26T20:59:43.809 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:44.005 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628347 2024-04-26T20:59:44.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:44.019 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:44.019 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:44.020 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:44.020 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:44.020 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/nightlies").st_ino) 2024-04-26T20:59:44.020 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:44.216 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628446 2024-04-26T20:59:44.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:44.231 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:44.231 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:44.231 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:44.231 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:44.231 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/nightlies/cron_wrapper").st_ino) 2024-04-26T20:59:44.231 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:44.434 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628447 2024-04-26T20:59:44.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:44.449 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:44.450 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:44.450 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:44.450 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:44.450 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/overrides").st_ino) 2024-04-26T20:59:44.450 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:44.644 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628451 2024-04-26T20:59:44.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:44.658 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:44.659 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:44.659 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:44.659 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:44.659 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/overrides/3-size-2-min-size.yaml").st_ino) 2024-04-26T20:59:44.659 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:44.853 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628454 2024-04-26T20:59:44.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:44.867 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:44.867 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:44.868 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:44.868 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:44.868 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/overrides/whitelist_wrongly_marked_down.yaml").st_ino) 2024-04-26T20:59:44.868 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:45.053 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628456 2024-04-26T20:59:45.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:45.067 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:45.067 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:45.068 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:45.068 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:45.068 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/overrides/2-size-2-min-size.yaml").st_ino) 2024-04-26T20:59:45.068 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:45.263 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628453 2024-04-26T20:59:45.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:45.277 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:45.277 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:45.277 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:45.277 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:45.277 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/overrides/2-size-1-min-size.yaml").st_ino) 2024-04-26T20:59:45.277 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:45.475 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628452 2024-04-26T20:59:45.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:45.490 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:45.490 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:45.490 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:45.490 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:45.490 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/overrides/short_pg_log.yaml").st_ino) 2024-04-26T20:59:45.490 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:45.685 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628455 2024-04-26T20:59:45.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:45.699 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:45.699 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:45.699 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:45.699 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:45.699 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/packages").st_ino) 2024-04-26T20:59:45.700 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:45.895 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628457 2024-04-26T20:59:45.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:45.910 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:45.910 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:45.910 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:45.911 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:45.911 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/packages/packages.yaml").st_ino) 2024-04-26T20:59:45.911 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:46.105 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628458 2024-04-26T20:59:46.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:46.120 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:46.120 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:46.120 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:46.120 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:46.120 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/clusters").st_ino) 2024-04-26T20:59:46.120 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:46.253 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:59:46.261 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T20:59:46.302 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628375 2024-04-26T20:59:46.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:46.316 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:46.317 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:46.317 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:46.317 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:46.317 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/clusters/extra-client.yaml").st_ino) 2024-04-26T20:59:46.317 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:46.450 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628376 2024-04-26T20:59:46.454 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:46.464 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:46.464 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:46.464 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:46.465 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:46.465 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/clusters/fixed-2.yaml").st_ino) 2024-04-26T20:59:46.465 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:46.614 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628378 2024-04-26T20:59:46.619 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:46.629 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:46.629 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:46.629 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:46.629 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:46.629 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/clusters/fixed-3.yaml").st_ino) 2024-04-26T20:59:46.629 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:46.824 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628380 2024-04-26T20:59:46.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:46.838 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:46.838 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:46.838 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:46.838 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:46.839 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/clusters/fixed-3-cephfs.yaml").st_ino) 2024-04-26T20:59:46.839 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:47.034 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628379 2024-04-26T20:59:47.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:47.048 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:47.048 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:47.049 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:47.049 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:47.049 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/clusters/fixed-4.yaml").st_ino) 2024-04-26T20:59:47.049 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:47.244 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628381 2024-04-26T20:59:47.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:47.258 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:47.259 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:47.259 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:47.259 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:47.259 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/clusters/fixed-1.yaml").st_ino) 2024-04-26T20:59:47.259 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:47.442 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628377 2024-04-26T20:59:47.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:47.456 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:47.456 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:47.456 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:47.457 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:47.457 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs").st_ino) 2024-04-26T20:59:47.457 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:47.651 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628350 2024-04-26T20:59:47.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:47.666 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:47.666 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:47.666 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:47.666 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:47.666 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/overrides").st_ino) 2024-04-26T20:59:47.666 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:47.859 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628359 2024-04-26T20:59:47.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:47.873 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:47.873 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:47.874 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:47.874 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:47.874 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/overrides/whitelist_wrongly_marked_down.yaml").st_ino) 2024-04-26T20:59:47.874 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:48.076 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628367 2024-04-26T20:59:48.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:48.093 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:48.093 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:48.093 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:48.093 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:48.093 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/overrides/debug.yaml").st_ino) 2024-04-26T20:59:48.093 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:48.274 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628360 2024-04-26T20:59:48.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:48.289 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:48.289 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:48.289 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:48.289 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:48.289 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/overrides/fuse").st_ino) 2024-04-26T20:59:48.289 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:48.485 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628362 2024-04-26T20:59:48.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:48.501 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:48.501 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:48.501 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:48.501 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:48.501 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/overrides/fuse/default-perm").st_ino) 2024-04-26T20:59:48.502 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:48.698 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628363 2024-04-26T20:59:48.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:48.713 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:48.713 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:48.713 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:48.713 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:48.713 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/overrides/fuse/default-perm/yes.yaml").st_ino) 2024-04-26T20:59:48.713 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:48.909 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628366 2024-04-26T20:59:48.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:48.923 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:48.923 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:48.923 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:48.923 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:48.923 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/overrides/fuse/default-perm/no.yaml").st_ino) 2024-04-26T20:59:48.923 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:49.109 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628365 2024-04-26T20:59:49.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:49.125 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:49.125 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:49.125 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:49.125 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:49.125 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/overrides/fuse/default-perm/%").st_ino) 2024-04-26T20:59:49.125 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:49.320 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628364 2024-04-26T20:59:49.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:49.335 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:49.335 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:49.336 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:49.336 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:49.336 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/overrides/frag_enable.yaml").st_ino) 2024-04-26T20:59:49.336 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:49.533 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628361 2024-04-26T20:59:49.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:49.548 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:49.548 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:49.548 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:49.548 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:49.548 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/clusters").st_ino) 2024-04-26T20:59:49.548 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:49.744 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628352 2024-04-26T20:59:49.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:49.759 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:49.759 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:49.760 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:49.760 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:49.760 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/clusters/9-mds.yaml").st_ino) 2024-04-26T20:59:49.760 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:49.959 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628354 2024-04-26T20:59:49.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:49.974 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:49.974 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:49.974 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:49.974 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:49.974 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/clusters/3-mds.yaml").st_ino) 2024-04-26T20:59:49.974 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:50.167 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628353 2024-04-26T20:59:50.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:50.182 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:50.182 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:50.182 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:50.183 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:50.183 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/clusters/fixed-2-ucephfs.yaml").st_ino) 2024-04-26T20:59:50.183 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:50.380 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628355 2024-04-26T20:59:50.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:50.395 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:50.395 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:50.395 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:50.395 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:50.395 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/mount").st_ino) 2024-04-26T20:59:50.395 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:50.591 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628356 2024-04-26T20:59:50.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:50.606 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:50.607 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:50.607 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:50.607 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:50.607 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/mount/kclient.yaml").st_ino) 2024-04-26T20:59:50.607 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:50.794 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628358 2024-04-26T20:59:50.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:50.809 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:50.809 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:50.809 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:50.809 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:50.809 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/mount/fuse.yaml").st_ino) 2024-04-26T20:59:50.809 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:51.005 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628357 2024-04-26T20:59:51.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:51.020 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:51.020 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:51.020 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:51.020 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:51.020 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/begin.yaml").st_ino) 2024-04-26T20:59:51.021 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:51.203 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628351 2024-04-26T20:59:51.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:51.217 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:51.218 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:51.218 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:51.218 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:51.218 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/tasks").st_ino) 2024-04-26T20:59:51.218 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:51.415 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628368 2024-04-26T20:59:51.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:51.430 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:51.430 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:51.430 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:51.430 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:51.430 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/tasks/libcephfs_interface_tests.yaml").st_ino) 2024-04-26T20:59:51.430 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:51.627 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628374 2024-04-26T20:59:51.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:51.643 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:51.643 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:51.643 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:51.643 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:51.643 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/tasks/cfuse_workunit_trivial_sync.yaml").st_ino) 2024-04-26T20:59:51.643 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:51.837 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628373 2024-04-26T20:59:51.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:51.853 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:51.853 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:51.853 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:51.853 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:51.853 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/tasks/cfuse_workunit_suites_ffsb.yaml").st_ino) 2024-04-26T20:59:51.853 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:52.050 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628371 2024-04-26T20:59:52.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:52.065 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:52.065 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:52.065 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:52.065 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:52.065 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/tasks/cfuse_workunit_suites_blogbench.yaml").st_ino) 2024-04-26T20:59:52.066 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:52.265 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628369 2024-04-26T20:59:52.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:52.280 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:52.280 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:52.280 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:52.280 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:52.280 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/tasks/cfuse_workunit_suites_fsstress.yaml").st_ino) 2024-04-26T20:59:52.280 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:52.467 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628372 2024-04-26T20:59:52.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:52.482 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:52.483 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:52.483 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:52.483 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:52.483 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/cephfs/tasks/cfuse_workunit_suites_dbench.yaml").st_ino) 2024-04-26T20:59:52.483 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:52.667 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628370 2024-04-26T20:59:52.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:52.682 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:52.682 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:52.682 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:52.682 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:52.682 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/objectstore").st_ino) 2024-04-26T20:59:52.682 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:52.874 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628448 2024-04-26T20:59:52.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:52.888 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:52.888 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:52.888 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:52.888 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:52.888 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/objectstore/bluestore.yaml").st_ino) 2024-04-26T20:59:52.888 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:53.085 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628449 2024-04-26T20:59:53.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:53.100 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:53.100 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:53.100 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:53.100 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:53.100 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/objectstore/filestore.yaml").st_ino) 2024-04-26T20:59:53.101 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:53.291 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628450 2024-04-26T20:59:53.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:53.306 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:53.306 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:53.306 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:53.306 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:53.306 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/config").st_ino) 2024-04-26T20:59:53.306 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:53.500 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628382 2024-04-26T20:59:53.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:53.515 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:53.515 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:53.515 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:53.515 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:53.515 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/config/rados.yaml").st_ino) 2024-04-26T20:59:53.515 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:53.703 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628383 2024-04-26T20:59:53.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:53.718 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:53.718 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:53.718 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:53.718 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:53.718 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/archs").st_ino) 2024-04-26T20:59:53.718 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:53.912 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628340 2024-04-26T20:59:53.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:53.928 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:53.928 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:53.928 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:53.928 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:53.928 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/archs/armv7.yaml").st_ino) 2024-04-26T20:59:53.928 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:54.125 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628342 2024-04-26T20:59:54.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:54.141 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:54.141 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:54.141 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:54.141 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:54.141 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/archs/i686.yaml").st_ino) 2024-04-26T20:59:54.141 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:54.335 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628343 2024-04-26T20:59:54.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:54.350 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:54.350 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:54.351 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:54.351 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:54.351 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/archs/aarch64.yaml").st_ino) 2024-04-26T20:59:54.351 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:54.546 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628341 2024-04-26T20:59:54.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:54.562 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:54.562 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:54.562 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:54.562 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:54.562 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/archs/x86_64.yaml").st_ino) 2024-04-26T20:59:54.562 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:54.756 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628344 2024-04-26T20:59:54.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:54.771 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:54.771 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:54.771 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:54.771 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:54.772 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/fs").st_ino) 2024-04-26T20:59:54.772 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:54.966 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628435 2024-04-26T20:59:54.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:54.981 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:54.981 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:54.981 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:54.981 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:54.981 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/fs/btrfs.yaml").st_ino) 2024-04-26T20:59:54.982 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:55.165 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628436 2024-04-26T20:59:55.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:55.180 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:55.180 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:55.180 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:55.180 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:55.180 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/fs/xfs.yaml").st_ino) 2024-04-26T20:59:55.181 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:55.381 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628438 2024-04-26T20:59:55.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:55.395 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:55.395 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:55.395 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:55.396 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:55.396 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/fs/ext4.yaml").st_ino) 2024-04-26T20:59:55.396 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:55.590 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628437 2024-04-26T20:59:55.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:55.605 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:55.606 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:55.606 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:55.606 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:55.606 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/rgw_pool_type").st_ino) 2024-04-26T20:59:55.606 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:55.793 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628463 2024-04-26T20:59:55.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:55.809 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:55.809 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:55.809 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:55.809 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:55.809 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/rgw_pool_type/ec-profile.yaml").st_ino) 2024-04-26T20:59:55.809 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:55.995 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628465 2024-04-26T20:59:55.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:56.009 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:56.009 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:56.009 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:56.009 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:56.009 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/rgw_pool_type/ec.yaml").st_ino) 2024-04-26T20:59:56.009 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:56.207 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628466 2024-04-26T20:59:56.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:56.222 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:56.222 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:56.222 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:56.222 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:56.222 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/rgw_pool_type/ec-cache.yaml").st_ino) 2024-04-26T20:59:56.222 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:56.423 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628464 2024-04-26T20:59:56.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:56.437 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:56.438 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:56.438 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:56.438 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:56.438 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/rgw_pool_type/replicated.yaml").st_ino) 2024-04-26T20:59:56.438 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:56.622 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628467 2024-04-26T20:59:56.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:56.637 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:56.637 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:56.637 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:56.637 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:56.638 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git").st_ino) 2024-04-26T20:59:56.638 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:56.835 INFO:teuthology.orchestra.run.smithi028.stdout:1099511627777 2024-04-26T20:59:56.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:56.851 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:56.851 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:56.851 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:56.851 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:56.851 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/HEAD").st_ino) 2024-04-26T20:59:56.851 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:57.034 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628328 2024-04-26T20:59:57.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:57.048 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:57.048 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:57.048 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:57.048 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:57.048 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/description").st_ino) 2024-04-26T20:59:57.048 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:57.242 INFO:teuthology.orchestra.run.smithi028.stdout:1099511627778 2024-04-26T20:59:57.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:57.257 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:57.257 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:57.257 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:57.258 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:57.258 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/index").st_ino) 2024-04-26T20:59:57.258 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:57.453 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628337 2024-04-26T20:59:57.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:57.468 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:57.468 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:57.468 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:57.468 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:57.468 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/objects").st_ino) 2024-04-26T20:59:57.468 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:57.648 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628307 2024-04-26T20:59:57.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:57.663 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:57.663 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:57.663 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:57.663 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:57.663 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/objects/info").st_ino) 2024-04-26T20:59:57.663 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:57.859 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628309 2024-04-26T20:59:57.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:57.873 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:57.873 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:57.873 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:57.874 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:57.874 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/objects/pack").st_ino) 2024-04-26T20:59:57.874 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:58.069 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628308 2024-04-26T20:59:58.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:58.084 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:58.085 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:58.085 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:58.085 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:58.085 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/objects/pack/pack-ac547ca4f300164f23b0bf1b8b1e66d5a7352309.idx").st_ino) 2024-04-26T20:59:58.085 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:58.276 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628315 2024-04-26T20:59:58.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:58.292 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:58.292 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:58.292 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:58.292 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:58.292 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/objects/pack/pack-ac547ca4f300164f23b0bf1b8b1e66d5a7352309.rev").st_ino) 2024-04-26T20:59:58.292 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:58.481 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628316 2024-04-26T20:59:58.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:58.496 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:58.496 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:58.496 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:58.496 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:58.496 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/objects/pack/pack-ac547ca4f300164f23b0bf1b8b1e66d5a7352309.pack").st_ino) 2024-04-26T20:59:58.496 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:58.694 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628314 2024-04-26T20:59:58.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:58.708 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:58.708 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:58.708 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:58.708 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:58.709 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/hooks").st_ino) 2024-04-26T20:59:58.709 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:58.903 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628282 2024-04-26T20:59:58.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:58.918 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:58.918 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:58.918 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:58.918 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:58.918 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/hooks/pre-merge-commit.sample").st_ino) 2024-04-26T20:59:58.918 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:59.117 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628287 2024-04-26T20:59:59.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:59.131 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:59.131 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:59.131 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:59.131 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:59.132 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/hooks/pre-commit.sample").st_ino) 2024-04-26T20:59:59.132 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:59.332 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628286 2024-04-26T20:59:59.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:59.347 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:59.347 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:59.347 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:59.347 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:59.347 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/hooks/fsmonitor-watchman.sample").st_ino) 2024-04-26T20:59:59.347 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:59.544 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628291 2024-04-26T20:59:59.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:59.559 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:59.559 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:59.559 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:59.559 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:59.559 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/hooks/commit-msg.sample").st_ino) 2024-04-26T20:59:59.559 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:59.754 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628294 2024-04-26T20:59:59.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:59.769 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:59.769 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:59.769 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:59.769 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:59.769 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/hooks/pre-rebase.sample").st_ino) 2024-04-26T20:59:59.769 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T20:59:59.962 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628296 2024-04-26T20:59:59.966 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T20:59:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T20:59:59.976 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T20:59:59.976 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T20:59:59.976 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T20:59:59.976 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T20:59:59.976 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/hooks/pre-receive.sample").st_ino) 2024-04-26T20:59:59.976 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:00.171 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628290 2024-04-26T21:00:00.175 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:00.185 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:00.186 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:00.186 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:00.186 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:00.186 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/hooks/prepare-commit-msg.sample").st_ino) 2024-04-26T21:00:00.186 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:00.380 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628289 2024-04-26T21:00:00.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:00.395 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:00.395 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:00.395 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:00.396 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:00.396 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/hooks/update.sample").st_ino) 2024-04-26T21:00:00.396 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:00.592 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628288 2024-04-26T21:00:00.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:00.607 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:00.607 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:00.607 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:00.607 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:00.607 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/hooks/pre-push.sample").st_ino) 2024-04-26T21:00:00.607 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:00.804 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628292 2024-04-26T21:00:00.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:00.819 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:00.819 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:00.820 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:00.820 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:00.820 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/hooks/sendemail-validate.sample").st_ino) 2024-04-26T21:00:00.820 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:01.013 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628285 2024-04-26T21:00:01.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:01.027 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:01.028 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:01.028 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:01.028 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:01.028 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/hooks/pre-applypatch.sample").st_ino) 2024-04-26T21:00:01.028 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:01.225 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628293 2024-04-26T21:00:01.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:01.240 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:01.240 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:01.240 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:01.240 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:01.240 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/hooks/push-to-checkout.sample").st_ino) 2024-04-26T21:00:01.240 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:01.439 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628283 2024-04-26T21:00:01.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:01.454 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:01.458 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:01.458 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:01.458 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:01.458 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/hooks/post-update.sample").st_ino) 2024-04-26T21:00:01.458 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:01.653 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628284 2024-04-26T21:00:01.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:01.669 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:01.669 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:01.669 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:01.670 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:01.670 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/hooks/applypatch-msg.sample").st_ino) 2024-04-26T21:00:01.670 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:01.866 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628295 2024-04-26T21:00:01.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:01.880 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:01.880 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:01.880 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:01.880 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:01.880 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/logs").st_ino) 2024-04-26T21:00:01.880 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:02.074 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628323 2024-04-26T21:00:02.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:02.088 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:02.088 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:02.088 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:02.088 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:02.088 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/logs/HEAD").st_ino) 2024-04-26T21:00:02.089 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:02.283 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628331 2024-04-26T21:00:02.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:02.298 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:02.298 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:02.298 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:02.298 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:02.298 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/logs/refs").st_ino) 2024-04-26T21:00:02.298 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:02.479 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628324 2024-04-26T21:00:02.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:02.493 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:02.493 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:02.493 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:02.493 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:02.493 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/logs/refs/heads").st_ino) 2024-04-26T21:00:02.493 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:02.684 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628332 2024-04-26T21:00:02.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:02.699 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:02.699 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:02.700 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:02.700 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:02.700 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/logs/refs/heads/master").st_ino) 2024-04-26T21:00:02.700 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:02.886 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628333 2024-04-26T21:00:02.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:02.896 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:02.897 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:02.897 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:02.897 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:02.897 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/logs/refs/remotes").st_ino) 2024-04-26T21:00:02.897 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:03.078 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628325 2024-04-26T21:00:03.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:03.092 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:03.092 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:03.092 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:03.092 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:03.093 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/logs/refs/remotes/origin").st_ino) 2024-04-26T21:00:03.093 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:03.289 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628326 2024-04-26T21:00:03.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:03.305 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:03.305 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:03.305 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:03.305 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:03.305 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/logs/refs/remotes/origin/HEAD").st_ino) 2024-04-26T21:00:03.305 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:03.505 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628327 2024-04-26T21:00:03.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:03.520 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:03.520 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:03.520 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:03.520 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:03.520 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/packed-refs").st_ino) 2024-04-26T21:00:03.520 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:03.717 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628319 2024-04-26T21:00:03.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:03.732 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:03.732 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:03.732 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:03.733 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:03.733 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/config").st_ino) 2024-04-26T21:00:03.733 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:03.927 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628336 2024-04-26T21:00:03.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:03.942 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:03.942 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:03.942 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:03.942 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:03.942 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/info").st_ino) 2024-04-26T21:00:03.942 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:04.139 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628279 2024-04-26T21:00:04.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:04.154 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:04.154 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:04.154 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:04.154 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:04.154 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/info/exclude").st_ino) 2024-04-26T21:00:04.154 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:04.350 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628280 2024-04-26T21:00:04.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:04.365 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:04.365 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:04.365 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:04.365 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:04.365 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/refs").st_ino) 2024-04-26T21:00:04.365 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:04.560 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628297 2024-04-26T21:00:04.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:04.575 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:04.575 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:04.575 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:04.575 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:04.575 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/refs/tags").st_ino) 2024-04-26T21:00:04.575 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:04.770 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628299 2024-04-26T21:00:04.775 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:04.786 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:04.786 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:04.786 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:04.786 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:04.786 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/refs/heads").st_ino) 2024-04-26T21:00:04.786 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:04.991 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628298 2024-04-26T21:00:04.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:05.006 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:05.006 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:05.006 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:05.006 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:05.006 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/refs/heads/master").st_ino) 2024-04-26T21:00:05.006 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:05.207 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628330 2024-04-26T21:00:05.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:05.222 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:05.222 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:05.222 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:05.222 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:05.222 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/refs/remotes").st_ino) 2024-04-26T21:00:05.222 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:05.416 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628320 2024-04-26T21:00:05.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:05.432 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:05.432 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:05.432 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:05.432 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:05.432 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/refs/remotes/origin").st_ino) 2024-04-26T21:00:05.432 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:05.628 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628321 2024-04-26T21:00:05.633 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:05.643 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:05.643 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:05.644 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:05.644 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:05.644 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/refs/remotes/origin/HEAD").st_ino) 2024-04-26T21:00:05.644 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:05.841 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628322 2024-04-26T21:00:05.845 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:05.855 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:05.855 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:05.855 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:05.855 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:05.855 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.git/branches").st_ino) 2024-04-26T21:00:05.855 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:06.049 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628281 2024-04-26T21:00:06.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:06.063 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:06.064 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:06.064 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:06.064 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:06.064 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/config_options").st_ino) 2024-04-26T21:00:06.064 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:06.251 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628384 2024-04-26T21:00:06.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:06.266 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:06.266 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:06.266 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:06.266 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:06.266 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/config_options/cephdeploy_conf.yaml").st_ino) 2024-04-26T21:00:06.266 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:06.464 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628385 2024-04-26T21:00:06.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:06.480 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:06.480 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:06.480 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:06.480 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:06.480 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros").st_ino) 2024-04-26T21:00:06.480 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:06.677 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628391 2024-04-26T21:00:06.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:06.691 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:06.692 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:06.692 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:06.692 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:06.692 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/a-supported-distro.yaml").st_ino) 2024-04-26T21:00:06.692 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:06.888 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628392 2024-04-26T21:00:06.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:06.903 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:06.903 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:06.903 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:06.903 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:06.903 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/supported").st_ino) 2024-04-26T21:00:06.903 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:07.099 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628420 2024-04-26T21:00:07.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:07.114 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:07.114 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:07.114 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:07.114 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:07.115 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/supported/ubuntu_16.04.yaml").st_ino) 2024-04-26T21:00:07.115 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:07.311 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628423 2024-04-26T21:00:07.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:07.326 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:07.327 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:07.327 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:07.327 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:07.327 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/supported/centos_7.2.yaml").st_ino) 2024-04-26T21:00:07.327 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:07.521 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628421 2024-04-26T21:00:07.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:07.537 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:07.537 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:07.537 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:07.537 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:07.537 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/supported/ubuntu_14.04.yaml").st_ino) 2024-04-26T21:00:07.537 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:07.734 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628422 2024-04-26T21:00:07.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:07.749 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:07.749 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:07.750 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:07.750 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:07.750 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all").st_ino) 2024-04-26T21:00:07.750 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:07.945 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628393 2024-04-26T21:00:07.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:07.960 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:07.960 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:07.960 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:07.960 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:07.960 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/fedora_17.yaml").st_ino) 2024-04-26T21:00:07.960 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:08.155 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628403 2024-04-26T21:00:08.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:08.171 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:08.171 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:08.171 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:08.171 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:08.171 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/centos_6.4.yaml").st_ino) 2024-04-26T21:00:08.171 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:08.366 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628395 2024-04-26T21:00:08.371 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:08.381 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:08.382 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:08.382 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:08.382 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:08.382 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/debian_7.0.yaml").st_ino) 2024-04-26T21:00:08.382 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:08.578 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628401 2024-04-26T21:00:08.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:08.594 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:08.594 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:08.594 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:08.595 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:08.595 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/debian_6.0.yaml").st_ino) 2024-04-26T21:00:08.595 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:08.791 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628400 2024-04-26T21:00:08.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:08.807 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:08.807 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:08.807 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:08.807 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:08.807 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/fedora_19.yaml").st_ino) 2024-04-26T21:00:08.807 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:09.003 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628405 2024-04-26T21:00:09.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:09.018 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:09.018 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:09.018 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:09.018 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:09.018 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/ubuntu_16.04.yaml").st_ino) 2024-04-26T21:00:09.018 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:09.216 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628419 2024-04-26T21:00:09.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:09.231 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:09.231 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:09.231 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:09.231 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:09.231 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/opensuse_13.2.yaml").st_ino) 2024-04-26T21:00:09.231 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:09.425 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628407 2024-04-26T21:00:09.430 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:09.440 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:09.440 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:09.440 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:09.441 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:09.441 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/opensuse_12.2.yaml").st_ino) 2024-04-26T21:00:09.441 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:09.627 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628406 2024-04-26T21:00:09.632 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:09.641 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:09.642 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:09.642 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:09.642 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:09.642 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/rhel_6.3.yaml").st_ino) 2024-04-26T21:00:09.642 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:09.826 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628409 2024-04-26T21:00:09.830 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:09.839 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:09.840 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:09.840 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:09.840 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:09.840 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/sles_11-sp2.yaml").st_ino) 2024-04-26T21:00:09.840 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:10.021 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628413 2024-04-26T21:00:10.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:10.037 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:10.037 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:10.037 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:10.037 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:10.037 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/centos_7.2.yaml").st_ino) 2024-04-26T21:00:10.037 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:10.237 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628399 2024-04-26T21:00:10.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:10.250 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:10.250 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:10.250 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:10.250 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:10.250 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/rhel_6.4.yaml").st_ino) 2024-04-26T21:00:10.250 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:10.445 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628410 2024-04-26T21:00:10.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:10.461 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:10.461 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:10.461 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:10.461 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:10.461 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/centos_7.0.yaml").st_ino) 2024-04-26T21:00:10.461 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:10.657 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628397 2024-04-26T21:00:10.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:10.673 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:10.673 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:10.673 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:10.673 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:10.673 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/ubuntu_14.04_i686.yaml").st_ino) 2024-04-26T21:00:10.673 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:10.868 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628418 2024-04-26T21:00:10.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:10.883 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:10.883 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:10.883 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:10.883 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:10.883 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/opensuse_42.1.yaml").st_ino) 2024-04-26T21:00:10.883 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:11.082 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628408 2024-04-26T21:00:11.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:11.098 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:11.098 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:11.098 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:11.098 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:11.098 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/debian_8.0.yaml").st_ino) 2024-04-26T21:00:11.098 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:11.293 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628402 2024-04-26T21:00:11.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:11.308 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:11.308 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:11.308 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:11.308 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:11.308 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/fedora_18.yaml").st_ino) 2024-04-26T21:00:11.308 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:11.504 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628404 2024-04-26T21:00:11.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:11.519 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:11.519 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:11.519 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:11.519 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:11.519 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/rhel_7.0.yaml").st_ino) 2024-04-26T21:00:11.519 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:11.713 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628412 2024-04-26T21:00:11.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:11.727 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:11.727 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:11.727 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:11.727 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:11.727 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/ubuntu_14.04.yaml").st_ino) 2024-04-26T21:00:11.727 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:11.922 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628416 2024-04-26T21:00:11.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:11.937 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:11.937 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:11.937 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:11.937 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:11.937 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/centos_7.1.yaml").st_ino) 2024-04-26T21:00:11.937 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:12.135 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628398 2024-04-26T21:00:12.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:12.149 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:12.149 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:12.149 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:12.149 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:12.149 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/ubuntu_12.10.yaml").st_ino) 2024-04-26T21:00:12.150 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:12.348 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628415 2024-04-26T21:00:12.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:12.363 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:12.363 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:12.363 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:12.364 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:12.364 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/ubuntu_12.04.yaml").st_ino) 2024-04-26T21:00:12.364 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:12.561 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628414 2024-04-26T21:00:12.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:12.576 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:12.576 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:12.576 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:12.576 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:12.576 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/centos_6.3.yaml").st_ino) 2024-04-26T21:00:12.576 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:12.772 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628394 2024-04-26T21:00:12.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:12.787 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:12.787 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:12.787 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:12.787 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:12.787 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/rhel_6.5.yaml").st_ino) 2024-04-26T21:00:12.787 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:12.986 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628411 2024-04-26T21:00:12.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:13.001 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:13.001 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:13.001 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:13.002 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:13.002 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/ubuntu_14.04_aarch64.yaml").st_ino) 2024-04-26T21:00:13.002 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:13.197 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628417 2024-04-26T21:00:13.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:13.213 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:13.213 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:13.213 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:13.213 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:13.213 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/distros/all/centos_6.5.yaml").st_ino) 2024-04-26T21:00:13.213 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:13.413 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628396 2024-04-26T21:00:13.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:13.428 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:13.428 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:13.428 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:13.429 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:13.429 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites").st_ino) 2024-04-26T21:00:13.429 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:13.623 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628468 2024-04-26T21:00:13.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:13.638 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:13.638 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:13.638 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:13.638 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:13.639 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress").st_ino) 2024-04-26T21:00:13.639 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:13.837 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629781 2024-04-26T21:00:13.842 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:13.851 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:13.851 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:13.852 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:13.852 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:13.852 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/bench").st_ino) 2024-04-26T21:00:13.852 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:14.047 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629782 2024-04-26T21:00:14.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:14.064 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:14.065 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:14.065 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:14.065 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:14.065 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/bench/clusters").st_ino) 2024-04-26T21:00:14.065 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:14.262 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629784 2024-04-26T21:00:14.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:14.276 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:14.277 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:14.277 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:14.277 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:14.277 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/bench/clusters/fixed-3-cephfs.yaml").st_ino) 2024-04-26T21:00:14.277 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:14.468 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629785 2024-04-26T21:00:14.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:14.482 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:14.483 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:14.483 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:14.483 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:14.483 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/bench/%").st_ino) 2024-04-26T21:00:14.483 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:14.679 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629783 2024-04-26T21:00:14.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:14.694 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:14.695 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:14.695 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:14.695 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:14.695 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/bench/tasks").st_ino) 2024-04-26T21:00:14.695 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:14.889 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629786 2024-04-26T21:00:14.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:14.905 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:14.905 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:14.905 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:14.905 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:14.905 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/bench/tasks/kclient_workunit_suites_fsx.yaml").st_ino) 2024-04-26T21:00:14.905 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:15.101 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629788 2024-04-26T21:00:15.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:15.116 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:15.116 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:15.116 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:15.116 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:15.116 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/bench/tasks/cfuse_workunit_snaps.yaml").st_ino) 2024-04-26T21:00:15.116 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:15.312 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629787 2024-04-26T21:00:15.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:15.327 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:15.327 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:15.328 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:15.328 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:15.328 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/thrash").st_ino) 2024-04-26T21:00:15.328 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:15.526 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629789 2024-04-26T21:00:15.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:15.541 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:15.541 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:15.541 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:15.542 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:15.542 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/thrash/clusters").st_ino) 2024-04-26T21:00:15.542 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:15.738 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629791 2024-04-26T21:00:15.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:15.753 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:15.753 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:15.753 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:15.754 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:15.754 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/thrash/clusters/16-osd.yaml").st_ino) 2024-04-26T21:00:15.754 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:15.947 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629792 2024-04-26T21:00:15.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:15.962 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:15.962 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:15.962 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:15.962 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:15.962 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/thrash/clusters/3-osd-1-machine.yaml").st_ino) 2024-04-26T21:00:15.962 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:16.159 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629793 2024-04-26T21:00:16.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:16.174 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:16.174 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:16.174 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:16.174 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:16.174 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/thrash/clusters/8-osd.yaml").st_ino) 2024-04-26T21:00:16.174 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:16.374 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629794 2024-04-26T21:00:16.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:16.384 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:00:16.392 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:00:16.393 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:16.394 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:16.394 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:16.394 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:16.394 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/thrash/fs").st_ino) 2024-04-26T21:00:16.394 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:16.568 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629795 2024-04-26T21:00:16.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:16.581 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:16.581 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:16.582 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:16.582 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:16.582 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/thrash/fs/none.yaml").st_ino) 2024-04-26T21:00:16.582 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:16.708 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629797 2024-04-26T21:00:16.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:16.722 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:16.722 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:16.722 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:16.722 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:16.722 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/thrash/fs/btrfs.yaml").st_ino) 2024-04-26T21:00:16.722 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:16.889 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629796 2024-04-26T21:00:16.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:16.904 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:16.904 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:16.904 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:16.904 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:16.904 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/thrash/fs/xfs.yaml").st_ino) 2024-04-26T21:00:16.904 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:17.066 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629798 2024-04-26T21:00:17.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:17.080 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:17.080 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:17.080 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:17.080 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:17.080 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/thrash/workloads").st_ino) 2024-04-26T21:00:17.080 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:17.256 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629803 2024-04-26T21:00:17.260 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:17.270 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:17.270 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:17.270 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:17.270 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:17.270 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/thrash/workloads/readwrite.yaml").st_ino) 2024-04-26T21:00:17.270 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:17.440 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629807 2024-04-26T21:00:17.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:17.453 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:17.453 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:17.453 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:17.454 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:17.454 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/thrash/workloads/radosbench.yaml").st_ino) 2024-04-26T21:00:17.454 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:17.619 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629806 2024-04-26T21:00:17.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:17.632 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:17.632 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:17.632 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:17.632 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:17.632 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/thrash/workloads/iozone_cfuse.yaml").st_ino) 2024-04-26T21:00:17.632 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:17.809 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629805 2024-04-26T21:00:17.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:17.821 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:17.821 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:17.821 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:17.821 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:17.822 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/thrash/workloads/bonnie_cfuse.yaml").st_ino) 2024-04-26T21:00:17.822 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:18.000 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629804 2024-04-26T21:00:18.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:18.013 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:18.014 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:18.014 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:18.014 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:18.014 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/thrash/%").st_ino) 2024-04-26T21:00:18.014 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:18.190 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629790 2024-04-26T21:00:18.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:18.204 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:18.204 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:18.204 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:18.204 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:18.204 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/thrash/thrashers").st_ino) 2024-04-26T21:00:18.204 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:18.391 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629799 2024-04-26T21:00:18.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:18.404 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:18.404 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:18.404 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:18.404 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:18.404 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/thrash/thrashers/default.yaml").st_ino) 2024-04-26T21:00:18.404 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:18.577 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629800 2024-04-26T21:00:18.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:18.592 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:18.592 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:18.592 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:18.592 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:18.592 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/thrash/thrashers/more-down.yaml").st_ino) 2024-04-26T21:00:18.593 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:18.778 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629802 2024-04-26T21:00:18.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:18.793 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:18.793 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:18.793 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:18.793 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:18.793 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/stress/thrash/thrashers/fast.yaml").st_ino) 2024-04-26T21:00:18.793 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:18.987 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629801 2024-04-26T21:00:18.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:19.001 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:19.001 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:19.001 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:19.001 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:19.001 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/mgr").st_ino) 2024-04-26T21:00:19.001 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:19.197 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629041 2024-04-26T21:00:19.201 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:19.211 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:19.211 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:19.211 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:19.211 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:19.211 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/mgr/basic").st_ino) 2024-04-26T21:00:19.211 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:19.407 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629042 2024-04-26T21:00:19.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:19.421 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:19.421 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:19.422 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:19.422 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:19.422 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/mgr/basic/clusters").st_ino) 2024-04-26T21:00:19.422 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:19.618 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629044 2024-04-26T21:00:19.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:19.633 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:19.633 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:19.633 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:19.633 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:19.633 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/mgr/basic/clusters/2-node-mgr.yaml").st_ino) 2024-04-26T21:00:19.633 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:19.831 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629045 2024-04-26T21:00:19.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:19.844 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:19.845 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:19.845 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:19.845 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:19.845 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/mgr/basic/fs").st_ino) 2024-04-26T21:00:19.845 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:20.040 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629048 2024-04-26T21:00:20.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:20.054 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:20.054 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:20.054 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:20.054 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:20.054 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/mgr/basic/fs/xfs.yaml").st_ino) 2024-04-26T21:00:20.054 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:20.246 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629049 2024-04-26T21:00:20.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:20.262 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:20.262 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:20.262 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:20.262 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:20.262 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/mgr/basic/%").st_ino) 2024-04-26T21:00:20.262 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:20.457 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629043 2024-04-26T21:00:20.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:20.472 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:20.472 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:20.472 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:20.472 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:20.472 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/mgr/basic/tasks").st_ino) 2024-04-26T21:00:20.472 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:20.668 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629050 2024-04-26T21:00:20.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:20.683 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:20.683 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:20.683 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:20.684 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:20.684 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/mgr/basic/tasks/failover.yaml").st_ino) 2024-04-26T21:00:20.684 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:20.868 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629051 2024-04-26T21:00:20.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:20.882 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:20.882 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:20.882 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:20.882 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:20.882 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/mgr/basic/debug").st_ino) 2024-04-26T21:00:20.882 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:21.078 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629046 2024-04-26T21:00:21.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:21.094 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:21.094 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:21.094 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:21.094 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:21.094 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/mgr/basic/debug/mgr.yaml").st_ino) 2024-04-26T21:00:21.094 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:21.289 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629047 2024-04-26T21:00:21.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:21.304 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:21.304 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:21.304 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:21.304 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:21.304 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle").st_ino) 2024-04-26T21:00:21.304 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:21.498 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629094 2024-04-26T21:00:21.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:21.512 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:21.512 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:21.512 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:21.512 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:21.513 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd").st_ino) 2024-04-26T21:00:21.513 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:21.709 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629095 2024-04-26T21:00:21.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:21.723 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:21.723 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:21.723 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:21.723 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:21.723 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/powercycle").st_ino) 2024-04-26T21:00:21.723 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:21.924 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629103 2024-04-26T21:00:21.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:21.939 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:21.939 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:21.939 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:21.939 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:21.939 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/powercycle/default.yaml").st_ino) 2024-04-26T21:00:21.939 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:22.134 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629104 2024-04-26T21:00:22.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:22.149 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:22.149 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:22.149 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:22.149 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:22.149 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/clusters").st_ino) 2024-04-26T21:00:22.149 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:22.352 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629097 2024-04-26T21:00:22.357 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:22.367 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:22.367 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:22.367 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:22.367 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:22.367 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/clusters/3osd-1per-target.yaml").st_ino) 2024-04-26T21:00:22.367 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:22.565 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629098 2024-04-26T21:00:22.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:22.579 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:22.579 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:22.579 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:22.579 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:22.579 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/objectstore").st_ino) 2024-04-26T21:00:22.579 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:22.778 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629102 2024-04-26T21:00:22.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:22.793 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:22.793 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:22.793 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:22.793 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:22.793 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/fs").st_ino) 2024-04-26T21:00:22.793 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:22.975 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629099 2024-04-26T21:00:22.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:22.989 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:22.989 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:22.989 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:22.989 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:22.989 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/fs/btrfs.yaml").st_ino) 2024-04-26T21:00:22.989 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:23.186 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629100 2024-04-26T21:00:23.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:23.200 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:23.200 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:23.200 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:23.200 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:23.200 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/fs/xfs.yaml").st_ino) 2024-04-26T21:00:23.200 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:23.395 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629101 2024-04-26T21:00:23.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:23.410 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:23.410 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:23.410 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:23.410 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:23.410 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/%").st_ino) 2024-04-26T21:00:23.410 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:23.607 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629096 2024-04-26T21:00:23.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:23.622 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:23.622 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:23.622 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:23.622 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:23.622 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/tasks").st_ino) 2024-04-26T21:00:23.622 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:23.816 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629105 2024-04-26T21:00:23.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:23.831 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:23.832 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:23.832 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:23.832 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:23.832 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/tasks/readwrite.yaml").st_ino) 2024-04-26T21:00:23.832 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:24.028 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629117 2024-04-26T21:00:24.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:24.042 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:24.042 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:24.042 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:24.042 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:24.042 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/tasks/cfuse_workunit_suites_fsync.yaml").st_ino) 2024-04-26T21:00:24.043 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:24.238 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629112 2024-04-26T21:00:24.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:24.252 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:24.252 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:24.252 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:24.253 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:24.253 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/tasks/radosbench.yaml").st_ino) 2024-04-26T21:00:24.253 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:24.437 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629116 2024-04-26T21:00:24.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:24.452 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:24.452 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:24.452 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:24.452 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:24.452 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/tasks/cfuse_workunit_suites_pjd.yaml").st_ino) 2024-04-26T21:00:24.452 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:24.650 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629113 2024-04-26T21:00:24.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:24.665 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:24.665 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:24.665 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:24.665 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:24.665 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/tasks/cfuse_workunit_suites_ffsb.yaml").st_ino) 2024-04-26T21:00:24.665 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:24.863 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629109 2024-04-26T21:00:24.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:24.878 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:24.878 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:24.878 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:24.878 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:24.878 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/tasks/cfuse_workunit_suites_fsx.yaml").st_ino) 2024-04-26T21:00:24.878 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:25.076 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629111 2024-04-26T21:00:25.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:25.091 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:25.091 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:25.091 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:25.091 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:25.091 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/tasks/snaps-few-objects.yaml").st_ino) 2024-04-26T21:00:25.091 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:25.287 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629118 2024-04-26T21:00:25.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:25.302 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:25.302 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:25.302 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:25.302 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:25.302 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/tasks/cfuse_workunit_misc.yaml").st_ino) 2024-04-26T21:00:25.302 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:25.497 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629108 2024-04-26T21:00:25.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:25.511 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:25.511 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:25.511 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:25.511 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:25.511 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/tasks/admin_socket_objecter_requests.yaml").st_ino) 2024-04-26T21:00:25.511 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:25.696 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629106 2024-04-26T21:00:25.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:25.711 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:25.711 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:25.711 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:25.711 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:25.711 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/tasks/cfuse_workunit_suites_truncate_delay.yaml").st_ino) 2024-04-26T21:00:25.711 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:25.906 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629114 2024-04-26T21:00:25.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:25.921 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:25.921 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:25.922 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:25.922 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:25.922 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/tasks/snaps-many-objects.yaml").st_ino) 2024-04-26T21:00:25.922 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:26.118 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629119 2024-04-26T21:00:26.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:26.133 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:26.133 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:26.133 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:26.133 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:26.134 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/tasks/cfuse_workunit_kernel_untar_build.yaml").st_ino) 2024-04-26T21:00:26.134 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:26.330 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629107 2024-04-26T21:00:26.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:26.345 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:26.345 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:26.345 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:26.345 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:26.345 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/tasks/rados_api_tests.yaml").st_ino) 2024-04-26T21:00:26.345 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:26.543 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629115 2024-04-26T21:00:26.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:26.558 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:26.558 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:26.558 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:26.558 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:26.558 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/powercycle/osd/tasks/cfuse_workunit_suites_fsstress.yaml").st_ino) 2024-04-26T21:00:26.558 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:26.757 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629110 2024-04-26T21:00:26.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:26.773 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:26.773 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:26.773 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:26.773 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:26.773 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd").st_ino) 2024-04-26T21:00:26.773 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:26.968 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629436 2024-04-26T21:00:26.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:26.982 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:26.982 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:26.982 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:26.983 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:26.983 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli").st_ino) 2024-04-26T21:00:26.983 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:27.165 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629460 2024-04-26T21:00:27.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:27.180 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:27.180 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:27.180 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:27.180 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:27.180 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/features").st_ino) 2024-04-26T21:00:27.180 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:27.376 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629465 2024-04-26T21:00:27.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:27.391 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:27.391 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:27.391 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:27.391 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:27.391 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/features/defaults.yaml").st_ino) 2024-04-26T21:00:27.391 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:27.590 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629466 2024-04-26T21:00:27.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:27.605 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:27.605 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:27.605 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:27.605 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:27.605 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/features/journaling.yaml").st_ino) 2024-04-26T21:00:27.605 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:27.808 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629468 2024-04-26T21:00:27.813 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:27.823 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:27.823 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:27.823 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:27.823 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:27.823 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/features/layering.yaml").st_ino) 2024-04-26T21:00:27.823 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:28.009 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629469 2024-04-26T21:00:28.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:28.023 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:28.023 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:28.023 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:28.023 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:28.023 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/features/format-1.yaml").st_ino) 2024-04-26T21:00:28.023 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:28.223 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629467 2024-04-26T21:00:28.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:28.238 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:28.238 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:28.238 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:28.238 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:28.238 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/base").st_ino) 2024-04-26T21:00:28.238 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:28.430 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629462 2024-04-26T21:00:28.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:28.445 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:28.445 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:28.445 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:28.445 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:28.445 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/base/install.yaml").st_ino) 2024-04-26T21:00:28.445 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:28.642 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629463 2024-04-26T21:00:28.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:28.658 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:28.658 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:28.658 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:28.658 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:28.659 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/pool").st_ino) 2024-04-26T21:00:28.659 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:28.852 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629475 2024-04-26T21:00:28.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:28.866 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:28.866 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:28.866 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:28.867 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:28.867 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/pool/small-cache-pool.yaml").st_ino) 2024-04-26T21:00:28.867 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:29.067 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629479 2024-04-26T21:00:29.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:29.082 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:29.082 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:29.082 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:29.082 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:29.082 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/pool/replicated-data-pool.yaml").st_ino) 2024-04-26T21:00:29.082 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:29.282 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629478 2024-04-26T21:00:29.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:29.296 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:29.296 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:29.296 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:29.296 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:29.297 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/pool/ec-data-pool.yaml").st_ino) 2024-04-26T21:00:29.297 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:29.497 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629476 2024-04-26T21:00:29.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:29.512 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:29.512 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:29.512 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:29.512 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:29.512 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/pool/none.yaml").st_ino) 2024-04-26T21:00:29.512 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:29.709 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629477 2024-04-26T21:00:29.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:29.723 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:29.724 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:29.724 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:29.724 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:29.724 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/clusters").st_ino) 2024-04-26T21:00:29.724 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:29.908 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629464 2024-04-26T21:00:29.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:29.923 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:29.923 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:29.923 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:29.923 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:29.923 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/msgr-failures").st_ino) 2024-04-26T21:00:29.923 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:30.121 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629471 2024-04-26T21:00:30.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:30.136 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:30.136 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:30.136 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:30.136 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:30.136 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/msgr-failures/many.yaml").st_ino) 2024-04-26T21:00:30.136 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:30.334 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629473 2024-04-26T21:00:30.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:30.350 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:30.350 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:30.350 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:30.350 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:30.350 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/msgr-failures/few.yaml").st_ino) 2024-04-26T21:00:30.350 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:30.538 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629472 2024-04-26T21:00:30.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:30.552 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:30.553 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:30.553 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:30.553 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:30.553 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/objectstore").st_ino) 2024-04-26T21:00:30.553 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:30.738 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629474 2024-04-26T21:00:30.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:30.752 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:30.752 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:30.753 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:30.753 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:30.753 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/fs").st_ino) 2024-04-26T21:00:30.753 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:30.949 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629470 2024-04-26T21:00:30.953 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:30.963 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:30.963 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:30.963 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:30.963 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:30.963 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/workloads").st_ino) 2024-04-26T21:00:30.963 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:31.162 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629480 2024-04-26T21:00:31.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:31.176 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:31.176 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:31.176 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:31.176 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:31.176 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/workloads/rbd_cli_import_export.yaml").st_ino) 2024-04-26T21:00:31.176 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:31.374 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629482 2024-04-26T21:00:31.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:31.390 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:31.390 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:31.390 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:31.390 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:31.390 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/workloads/rbd_cli_copy.yaml").st_ino) 2024-04-26T21:00:31.390 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:31.588 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629481 2024-04-26T21:00:31.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:31.603 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:31.603 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:31.604 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:31.604 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:31.604 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/cli/%").st_ino) 2024-04-26T21:00:31.604 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:31.801 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629461 2024-04-26T21:00:31.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:31.815 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:31.816 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:31.816 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:31.816 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:31.816 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/maintenance").st_ino) 2024-04-26T21:00:31.816 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:32.014 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629515 2024-04-26T21:00:32.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:32.030 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:32.031 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:32.031 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:32.031 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:32.031 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/maintenance/base").st_ino) 2024-04-26T21:00:32.031 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:32.226 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629517 2024-04-26T21:00:32.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:32.240 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:32.240 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:32.240 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:32.240 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:32.240 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/maintenance/base/install.yaml").st_ino) 2024-04-26T21:00:32.241 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:32.437 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629518 2024-04-26T21:00:32.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:32.453 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:32.453 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:32.453 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:32.453 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:32.453 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/maintenance/clusters").st_ino) 2024-04-26T21:00:32.453 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:32.640 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629519 2024-04-26T21:00:32.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:32.655 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:32.656 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:32.656 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:32.656 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:32.656 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/maintenance/clusters/openstack.yaml").st_ino) 2024-04-26T21:00:32.656 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:32.854 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629522 2024-04-26T21:00:32.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:32.869 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:32.869 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:32.869 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:32.870 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:32.870 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/maintenance/clusters/+").st_ino) 2024-04-26T21:00:32.870 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:33.066 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629520 2024-04-26T21:00:33.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:33.080 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:33.080 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:33.080 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:33.080 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:33.080 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/maintenance/clusters/fixed-3.yaml").st_ino) 2024-04-26T21:00:33.080 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:33.276 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629521 2024-04-26T21:00:33.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:33.291 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:33.291 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:33.291 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:33.291 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:33.291 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/maintenance/objectstore").st_ino) 2024-04-26T21:00:33.291 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:33.484 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629523 2024-04-26T21:00:33.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:33.499 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:33.500 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:33.500 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:33.500 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:33.500 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/maintenance/workloads").st_ino) 2024-04-26T21:00:33.500 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:33.697 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629526 2024-04-26T21:00:33.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:33.712 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:33.712 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:33.712 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:33.712 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:33.712 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/maintenance/workloads/dynamic_features.yaml").st_ino) 2024-04-26T21:00:33.712 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:33.909 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629527 2024-04-26T21:00:33.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:33.924 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:33.924 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:33.924 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:33.924 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:33.924 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/maintenance/workloads/rebuild_object_map.yaml").st_ino) 2024-04-26T21:00:33.924 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:34.122 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629528 2024-04-26T21:00:34.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:34.137 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:34.137 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:34.137 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:34.137 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:34.137 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/maintenance/%").st_ino) 2024-04-26T21:00:34.137 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:34.329 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629516 2024-04-26T21:00:34.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:34.345 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:34.345 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:34.345 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:34.345 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:34.345 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/maintenance/xfs.yaml").st_ino) 2024-04-26T21:00:34.345 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:34.544 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629529 2024-04-26T21:00:34.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:34.558 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:34.559 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:34.559 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:34.559 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:34.559 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/maintenance/qemu").st_ino) 2024-04-26T21:00:34.559 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:34.755 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629524 2024-04-26T21:00:34.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:34.771 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:34.771 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:34.771 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:34.771 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:34.771 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/maintenance/qemu/xfstests.yaml").st_ino) 2024-04-26T21:00:34.771 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:34.966 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629525 2024-04-26T21:00:34.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:34.981 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:34.981 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:34.981 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:34.981 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:34.982 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/singleton").st_ino) 2024-04-26T21:00:34.982 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:35.167 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629589 2024-04-26T21:00:35.172 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:35.182 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:35.182 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:35.183 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:35.183 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:35.183 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/singleton/openstack.yaml").st_ino) 2024-04-26T21:00:35.183 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:35.380 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629607 2024-04-26T21:00:35.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:35.395 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:35.395 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:35.395 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:35.395 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:35.396 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/singleton/objectstore").st_ino) 2024-04-26T21:00:35.396 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:35.594 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629606 2024-04-26T21:00:35.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:35.609 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:35.609 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:35.609 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:35.609 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:35.609 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/singleton/%").st_ino) 2024-04-26T21:00:35.609 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:35.809 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629590 2024-04-26T21:00:35.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:35.824 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:35.824 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:35.825 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:35.825 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:35.825 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/singleton/all").st_ino) 2024-04-26T21:00:35.825 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:36.019 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629591 2024-04-26T21:00:36.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:36.033 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:36.033 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:36.033 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:36.034 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:36.034 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/singleton/all/rbd-vs-unmanaged-snaps.yaml").st_ino) 2024-04-26T21:00:36.034 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:36.228 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629599 2024-04-26T21:00:36.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:36.243 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:36.243 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:36.243 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:36.243 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:36.243 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/singleton/all/rbd_mirror.yaml").st_ino) 2024-04-26T21:00:36.243 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:36.443 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629600 2024-04-26T21:00:36.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:36.457 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:36.457 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:36.457 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:36.458 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:36.458 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/singleton/all/verify_pool.yaml").st_ino) 2024-04-26T21:00:36.458 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:36.652 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629605 2024-04-26T21:00:36.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:36.667 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:36.667 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:36.667 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:36.667 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:36.667 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/singleton/all/formatted-output.yaml").st_ino) 2024-04-26T21:00:36.667 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:36.872 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629593 2024-04-26T21:00:36.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:36.886 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:36.886 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:36.886 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:36.886 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:36.886 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/singleton/all/rbdmap_RBDMAPFILE.yaml").st_ino) 2024-04-26T21:00:36.886 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:37.081 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629601 2024-04-26T21:00:37.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:37.096 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:37.096 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:37.096 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:37.096 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:37.096 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/singleton/all/qemu-iotests-writeback.yaml").st_ino) 2024-04-26T21:00:37.096 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:37.292 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629597 2024-04-26T21:00:37.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:37.307 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:37.307 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:37.307 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:37.307 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:37.307 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/singleton/all/read-flags-writeback.yaml").st_ino) 2024-04-26T21:00:37.307 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:37.502 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629603 2024-04-26T21:00:37.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:37.516 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:37.516 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:37.516 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:37.517 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:37.517 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/singleton/all/permissions.yaml").st_ino) 2024-04-26T21:00:37.517 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:37.713 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629595 2024-04-26T21:00:37.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:37.729 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:37.729 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:37.729 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:37.729 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:37.729 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/singleton/all/read-flags-no-cache.yaml").st_ino) 2024-04-26T21:00:37.729 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:37.925 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629602 2024-04-26T21:00:37.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:37.941 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:37.941 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:37.941 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:37.941 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:37.941 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/singleton/all/qemu-iotests-no-cache.yaml").st_ino) 2024-04-26T21:00:37.941 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:38.136 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629596 2024-04-26T21:00:38.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:38.150 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:38.150 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:38.151 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:38.151 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:38.151 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/singleton/all/qemu-iotests-writethrough.yaml").st_ino) 2024-04-26T21:00:38.151 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:38.347 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629598 2024-04-26T21:00:38.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:38.362 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:38.362 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:38.362 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:38.362 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:38.362 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/singleton/all/read-flags-writethrough.yaml").st_ino) 2024-04-26T21:00:38.362 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:38.562 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629604 2024-04-26T21:00:38.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:38.577 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:38.577 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:38.577 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:38.577 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:38.577 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/singleton/all/admin_socket.yaml").st_ino) 2024-04-26T21:00:38.577 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:38.775 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629592 2024-04-26T21:00:38.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:38.790 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:38.790 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:38.790 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:38.790 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:38.790 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/singleton/all/merge_diff.yaml").st_ino) 2024-04-26T21:00:38.791 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:38.984 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629594 2024-04-26T21:00:38.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:38.999 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:38.999 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:38.999 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:39.000 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:39.000 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic").st_ino) 2024-04-26T21:00:39.000 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:39.196 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629437 2024-04-26T21:00:39.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:39.211 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:39.211 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:39.211 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:39.211 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:39.211 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/base").st_ino) 2024-04-26T21:00:39.211 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:39.408 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629439 2024-04-26T21:00:39.412 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:39.422 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:39.422 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:39.422 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:39.422 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:39.422 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/base/install.yaml").st_ino) 2024-04-26T21:00:39.422 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:39.605 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629440 2024-04-26T21:00:39.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:39.619 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:39.619 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:39.620 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:39.620 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:39.620 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/cachepool").st_ino) 2024-04-26T21:00:39.620 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:39.816 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629441 2024-04-26T21:00:39.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:39.831 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:39.831 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:39.831 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:39.831 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:39.831 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/cachepool/none.yaml").st_ino) 2024-04-26T21:00:39.831 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:40.025 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629442 2024-04-26T21:00:40.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:40.040 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:40.040 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:40.040 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:40.040 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:40.040 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/cachepool/small.yaml").st_ino) 2024-04-26T21:00:40.040 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:40.239 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629443 2024-04-26T21:00:40.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:40.253 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:40.253 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:40.253 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:40.253 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:40.253 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/clusters").st_ino) 2024-04-26T21:00:40.253 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:40.449 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629444 2024-04-26T21:00:40.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:40.464 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:40.464 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:40.464 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:40.464 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:40.464 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/clusters/openstack.yaml").st_ino) 2024-04-26T21:00:40.464 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:40.654 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629447 2024-04-26T21:00:40.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:40.668 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:40.668 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:40.669 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:40.669 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:40.669 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/clusters/+").st_ino) 2024-04-26T21:00:40.669 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:40.863 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629445 2024-04-26T21:00:40.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:40.877 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:40.877 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:40.877 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:40.877 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:40.877 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/clusters/fixed-1.yaml").st_ino) 2024-04-26T21:00:40.877 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:41.074 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629446 2024-04-26T21:00:41.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:41.089 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:41.089 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:41.089 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:41.089 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:41.089 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/msgr-failures").st_ino) 2024-04-26T21:00:41.090 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:41.286 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629450 2024-04-26T21:00:41.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:41.301 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:41.301 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:41.301 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:41.301 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:41.301 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/msgr-failures/many.yaml").st_ino) 2024-04-26T21:00:41.302 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:41.498 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629452 2024-04-26T21:00:41.503 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:41.513 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:41.513 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:41.514 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:41.514 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:41.514 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/msgr-failures/few.yaml").st_ino) 2024-04-26T21:00:41.514 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:41.711 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629451 2024-04-26T21:00:41.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:41.726 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:41.726 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:41.726 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:41.726 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:41.726 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/objectstore").st_ino) 2024-04-26T21:00:41.726 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:41.929 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629453 2024-04-26T21:00:41.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:41.944 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:41.944 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:41.944 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:41.944 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:41.944 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/fs").st_ino) 2024-04-26T21:00:41.945 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:42.140 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629448 2024-04-26T21:00:42.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:42.155 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:42.155 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:42.155 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:42.155 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:42.155 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/fs/xfs.yaml").st_ino) 2024-04-26T21:00:42.156 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:42.356 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629449 2024-04-26T21:00:42.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:42.371 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:42.371 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:42.371 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:42.371 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:42.371 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/%").st_ino) 2024-04-26T21:00:42.372 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:42.576 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629438 2024-04-26T21:00:42.580 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:42.591 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:42.591 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:42.591 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:42.592 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:42.592 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/tasks").st_ino) 2024-04-26T21:00:42.592 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:42.798 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629454 2024-04-26T21:00:42.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:42.813 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:42.813 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:42.813 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:42.813 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:42.813 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/tasks/rbd_lock_and_fence.yaml").st_ino) 2024-04-26T21:00:42.813 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:43.009 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629458 2024-04-26T21:00:43.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:43.023 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:43.023 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:43.023 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:43.023 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:43.024 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/tasks/rbd_cli_tests.yaml").st_ino) 2024-04-26T21:00:43.024 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:43.216 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629456 2024-04-26T21:00:43.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:43.230 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:43.230 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:43.230 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:43.230 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:43.231 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/tasks/rbd_api_tests_old_format.yaml").st_ino) 2024-04-26T21:00:43.231 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:43.432 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629455 2024-04-26T21:00:43.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:43.447 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:43.448 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:43.448 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:43.448 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:43.448 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/tasks/rbd_cls_tests.yaml").st_ino) 2024-04-26T21:00:43.448 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:43.647 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629457 2024-04-26T21:00:43.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:43.662 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:43.663 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:43.663 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:43.663 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:43.663 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/basic/tasks/rbd_python_api_tests_old_format.yaml").st_ino) 2024-04-26T21:00:43.663 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:43.859 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629459 2024-04-26T21:00:43.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:43.873 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:43.873 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:43.873 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:43.873 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:43.873 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd").st_ino) 2024-04-26T21:00:43.873 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:44.067 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629483 2024-04-26T21:00:44.072 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:44.083 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:44.083 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:44.083 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:44.083 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:44.084 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/pool").st_ino) 2024-04-26T21:00:44.084 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:44.277 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629501 2024-04-26T21:00:44.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:44.293 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:44.293 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:44.293 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:44.293 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:44.293 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/pool/small-cache-pool.yaml").st_ino) 2024-04-26T21:00:44.293 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:44.489 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629505 2024-04-26T21:00:44.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:44.505 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:44.505 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:44.505 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:44.505 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:44.505 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/pool/replicated-data-pool.yaml").st_ino) 2024-04-26T21:00:44.506 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:44.704 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629504 2024-04-26T21:00:44.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:44.718 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:44.718 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:44.718 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:44.718 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:44.718 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/pool/ec-data-pool.yaml").st_ino) 2024-04-26T21:00:44.718 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:44.911 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629502 2024-04-26T21:00:44.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:44.925 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:44.926 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:44.926 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:44.926 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:44.926 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/pool/none.yaml").st_ino) 2024-04-26T21:00:44.926 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:45.121 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629503 2024-04-26T21:00:45.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:45.135 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:45.135 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:45.135 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:45.135 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:45.135 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/clusters").st_ino) 2024-04-26T21:00:45.135 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:45.330 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629489 2024-04-26T21:00:45.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:45.345 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:45.345 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:45.345 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:45.345 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:45.345 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/clusters/openstack.yaml").st_ino) 2024-04-26T21:00:45.345 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:45.542 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629492 2024-04-26T21:00:45.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:45.556 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:45.556 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:45.556 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:45.556 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:45.557 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/clusters/+").st_ino) 2024-04-26T21:00:45.557 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:45.753 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629490 2024-04-26T21:00:45.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:45.768 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:45.769 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:45.769 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:45.769 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:45.769 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/clusters/fixed-3.yaml").st_ino) 2024-04-26T21:00:45.769 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:45.970 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629491 2024-04-26T21:00:45.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:45.985 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:45.985 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:45.985 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:45.985 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:45.985 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/msgr-failures").st_ino) 2024-04-26T21:00:45.985 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:46.181 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629498 2024-04-26T21:00:46.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:46.197 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:46.197 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:46.197 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:46.197 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:46.197 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/msgr-failures/few.yaml").st_ino) 2024-04-26T21:00:46.197 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:46.397 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629499 2024-04-26T21:00:46.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:46.412 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:46.412 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:46.412 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:46.412 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:46.412 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/objectstore").st_ino) 2024-04-26T21:00:46.412 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:46.515 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:00:46.523 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:00:46.609 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629500 2024-04-26T21:00:46.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:46.626 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:46.626 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:46.626 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:46.626 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:46.626 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/config").st_ino) 2024-04-26T21:00:46.627 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:46.778 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629493 2024-04-26T21:00:46.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:46.793 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:46.793 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:46.793 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:46.793 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:46.793 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/config/skip-partial-discard.yaml").st_ino) 2024-04-26T21:00:46.794 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:46.946 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629496 2024-04-26T21:00:46.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:46.961 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:46.961 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:46.961 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:46.961 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:46.961 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/config/none.yaml").st_ino) 2024-04-26T21:00:46.961 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:47.162 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629495 2024-04-26T21:00:47.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:47.177 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:47.177 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:47.177 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:47.177 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:47.177 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/config/copy-on-read.yaml").st_ino) 2024-04-26T21:00:47.177 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:47.380 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629494 2024-04-26T21:00:47.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:47.394 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:47.395 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:47.395 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:47.395 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:47.395 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/fs").st_ino) 2024-04-26T21:00:47.395 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:47.593 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629497 2024-04-26T21:00:47.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:47.609 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:47.609 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:47.609 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:47.609 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:47.609 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/workloads").st_ino) 2024-04-26T21:00:47.609 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:47.793 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629506 2024-04-26T21:00:47.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:47.807 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:47.807 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:47.807 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:47.807 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:47.807 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/workloads/c_api_tests.yaml").st_ino) 2024-04-26T21:00:47.808 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:48.006 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629507 2024-04-26T21:00:48.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:48.020 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:48.020 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:48.021 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:48.021 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:48.021 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/workloads/rbd_fio.yaml").st_ino) 2024-04-26T21:00:48.021 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:48.216 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629514 2024-04-26T21:00:48.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:48.231 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:48.231 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:48.232 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:48.232 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:48.232 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/workloads/c_api_tests_with_defaults.yaml").st_ino) 2024-04-26T21:00:48.232 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:48.428 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629508 2024-04-26T21:00:48.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:48.443 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:48.444 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:48.444 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:48.444 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:48.444 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/workloads/python_api_tests.yaml").st_ino) 2024-04-26T21:00:48.444 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:48.643 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629511 2024-04-26T21:00:48.647 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:48.658 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:48.658 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:48.658 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:48.658 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:48.658 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/workloads/fsx.yaml").st_ino) 2024-04-26T21:00:48.658 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:48.852 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629510 2024-04-26T21:00:48.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:48.867 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:48.867 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:48.867 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:48.867 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:48.867 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/workloads/python_api_tests_with_journaling.yaml").st_ino) 2024-04-26T21:00:48.867 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:49.068 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629513 2024-04-26T21:00:49.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:49.083 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:49.083 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:49.083 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:49.084 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:49.084 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/workloads/c_api_tests_with_journaling.yaml").st_ino) 2024-04-26T21:00:49.084 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:49.279 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629509 2024-04-26T21:00:49.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:49.294 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:49.295 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:49.295 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:49.295 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:49.295 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/workloads/python_api_tests_with_defaults.yaml").st_ino) 2024-04-26T21:00:49.295 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:49.490 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629512 2024-04-26T21:00:49.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:49.504 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:49.505 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:49.505 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:49.505 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:49.505 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/cache").st_ino) 2024-04-26T21:00:49.505 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:49.705 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629485 2024-04-26T21:00:49.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:49.721 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:49.721 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:49.721 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:49.721 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:49.721 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/cache/writeback.yaml").st_ino) 2024-04-26T21:00:49.721 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:49.917 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629487 2024-04-26T21:00:49.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:49.931 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:49.931 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:49.931 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:49.931 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:49.931 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/cache/writethrough.yaml").st_ino) 2024-04-26T21:00:49.931 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:50.128 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629488 2024-04-26T21:00:50.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:50.143 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:50.143 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:50.144 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:50.144 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:50.144 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/cache/none.yaml").st_ino) 2024-04-26T21:00:50.144 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:50.338 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629486 2024-04-26T21:00:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:50.353 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:50.353 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:50.353 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:50.354 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:50.354 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/librbd/%").st_ino) 2024-04-26T21:00:50.354 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:50.549 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629484 2024-04-26T21:00:50.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:50.567 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:50.568 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:50.568 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:50.568 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:50.568 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/valgrind").st_ino) 2024-04-26T21:00:50.568 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:50.764 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629635 2024-04-26T21:00:50.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:50.779 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:50.779 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:50.779 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:50.779 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:50.779 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/valgrind/base").st_ino) 2024-04-26T21:00:50.779 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:50.978 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629637 2024-04-26T21:00:50.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:50.992 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:50.992 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:50.992 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:50.992 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:50.992 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/valgrind/base/install.yaml").st_ino) 2024-04-26T21:00:50.992 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:51.178 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629638 2024-04-26T21:00:51.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:51.194 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:51.194 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:51.194 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:51.194 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:51.194 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/valgrind/clusters").st_ino) 2024-04-26T21:00:51.195 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:51.394 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629639 2024-04-26T21:00:51.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:51.409 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:51.409 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:51.409 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:51.409 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:51.409 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/valgrind/objectstore").st_ino) 2024-04-26T21:00:51.409 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:51.607 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629641 2024-04-26T21:00:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:51.622 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:51.623 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:51.623 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:51.623 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:51.623 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/valgrind/fs").st_ino) 2024-04-26T21:00:51.623 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:51.820 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629640 2024-04-26T21:00:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:51.835 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:51.835 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:51.836 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:51.836 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:51.836 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/valgrind/workloads").st_ino) 2024-04-26T21:00:51.836 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:52.021 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629644 2024-04-26T21:00:52.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:52.037 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:52.037 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:52.037 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:52.037 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:52.037 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/valgrind/workloads/rbd_mirror.yaml").st_ino) 2024-04-26T21:00:52.037 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:52.234 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629652 2024-04-26T21:00:52.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:52.248 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:52.248 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:52.248 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:52.249 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:52.249 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/valgrind/workloads/c_api_tests.yaml").st_ino) 2024-04-26T21:00:52.249 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:52.433 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629645 2024-04-26T21:00:52.437 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:52.447 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:52.447 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:52.448 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:52.448 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:52.448 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/valgrind/workloads/c_api_tests_with_defaults.yaml").st_ino) 2024-04-26T21:00:52.448 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:52.640 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629646 2024-04-26T21:00:52.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:52.655 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:52.655 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:52.655 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:52.655 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:52.656 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/valgrind/workloads/python_api_tests.yaml").st_ino) 2024-04-26T21:00:52.656 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:52.852 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629649 2024-04-26T21:00:52.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:52.868 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:52.868 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:52.868 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:52.868 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:52.868 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/valgrind/workloads/fsx.yaml").st_ino) 2024-04-26T21:00:52.868 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:53.065 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629648 2024-04-26T21:00:53.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:53.080 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:53.080 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:53.080 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:53.081 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:53.081 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/valgrind/workloads/python_api_tests_with_journaling.yaml").st_ino) 2024-04-26T21:00:53.081 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:53.277 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629651 2024-04-26T21:00:53.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:53.291 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:53.291 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:53.292 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:53.292 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:53.292 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/valgrind/workloads/c_api_tests_with_journaling.yaml").st_ino) 2024-04-26T21:00:53.292 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:53.488 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629647 2024-04-26T21:00:53.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:53.503 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:53.503 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:53.503 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:53.504 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:53.504 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/valgrind/workloads/python_api_tests_with_defaults.yaml").st_ino) 2024-04-26T21:00:53.504 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:53.701 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629650 2024-04-26T21:00:53.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:53.716 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:53.716 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:53.716 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:53.716 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:53.716 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/valgrind/validator").st_ino) 2024-04-26T21:00:53.716 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:53.911 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629642 2024-04-26T21:00:53.915 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:53.925 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:53.925 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:53.925 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:53.925 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:53.925 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/valgrind/validator/memcheck.yaml").st_ino) 2024-04-26T21:00:53.925 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:54.123 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629643 2024-04-26T21:00:54.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:54.138 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:54.138 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:54.139 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:54.139 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:54.139 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/valgrind/%").st_ino) 2024-04-26T21:00:54.139 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:54.334 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629636 2024-04-26T21:00:54.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:54.348 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:54.348 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:54.348 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:54.348 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:54.348 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/nbd").st_ino) 2024-04-26T21:00:54.348 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:54.546 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629546 2024-04-26T21:00:54.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:54.561 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:54.561 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:54.561 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:54.561 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:54.561 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/nbd/base").st_ino) 2024-04-26T21:00:54.561 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:54.765 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629548 2024-04-26T21:00:54.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:54.781 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:54.781 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:54.781 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:54.781 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:54.781 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/nbd/cluster").st_ino) 2024-04-26T21:00:54.781 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:54.976 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629549 2024-04-26T21:00:54.981 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:54.991 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:54.991 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:54.991 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:54.991 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:54.991 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/nbd/cluster/openstack.yaml").st_ino) 2024-04-26T21:00:54.991 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:55.188 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629552 2024-04-26T21:00:55.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:55.203 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:55.203 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:55.203 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:55.203 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:55.203 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/nbd/cluster/+").st_ino) 2024-04-26T21:00:55.203 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:55.385 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629550 2024-04-26T21:00:55.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:55.400 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:55.400 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:55.400 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:55.400 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:55.400 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/nbd/cluster/fixed-3.yaml").st_ino) 2024-04-26T21:00:55.400 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:55.598 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629551 2024-04-26T21:00:55.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:55.613 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:55.613 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:55.613 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:55.613 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:55.613 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/nbd/msgr-failures").st_ino) 2024-04-26T21:00:55.613 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:55.808 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629554 2024-04-26T21:00:55.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:55.824 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:55.824 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:55.824 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:55.824 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:55.824 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/nbd/objectstore").st_ino) 2024-04-26T21:00:55.824 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:56.018 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629555 2024-04-26T21:00:56.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:56.034 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:56.034 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:56.034 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:56.034 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:56.035 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/nbd/fs").st_ino) 2024-04-26T21:00:56.035 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:56.231 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629553 2024-04-26T21:00:56.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:56.246 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:56.246 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:56.246 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:56.246 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:56.246 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/nbd/workloads").st_ino) 2024-04-26T21:00:56.246 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:56.443 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629557 2024-04-26T21:00:56.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:56.458 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:56.458 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:56.458 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:56.458 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:56.458 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/nbd/workloads/rbd_nbd.yaml").st_ino) 2024-04-26T21:00:56.458 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:56.655 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629559 2024-04-26T21:00:56.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:56.670 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:56.670 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:56.670 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:56.670 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:56.670 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/nbd/workloads/rbd_fsx_nbd.yaml").st_ino) 2024-04-26T21:00:56.670 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:56.866 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629558 2024-04-26T21:00:56.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:56.881 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:56.881 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:56.881 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:56.881 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:56.881 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/nbd/%").st_ino) 2024-04-26T21:00:56.881 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:57.077 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629547 2024-04-26T21:00:57.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:57.093 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:57.093 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:57.093 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:57.093 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:57.094 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/nbd/thrashers").st_ino) 2024-04-26T21:00:57.094 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:57.294 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629556 2024-04-26T21:00:57.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:57.309 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:57.309 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:57.309 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:57.309 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:57.309 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash").st_ino) 2024-04-26T21:00:57.309 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:57.505 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629608 2024-04-26T21:00:57.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:57.520 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:57.520 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:57.520 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:57.520 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:57.520 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/base").st_ino) 2024-04-26T21:00:57.520 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:57.715 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629610 2024-04-26T21:00:57.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:57.730 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:57.731 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:57.731 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:57.731 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:57.731 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/base/install.yaml").st_ino) 2024-04-26T21:00:57.731 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:57.926 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629611 2024-04-26T21:00:57.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:57.941 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:57.941 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:57.941 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:57.941 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:57.941 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/clusters").st_ino) 2024-04-26T21:00:57.941 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:58.143 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629612 2024-04-26T21:00:58.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:58.158 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:58.158 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:58.159 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:58.159 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:58.159 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/clusters/openstack.yaml").st_ino) 2024-04-26T21:00:58.159 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:58.356 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629615 2024-04-26T21:00:58.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:58.372 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:58.372 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:58.372 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:58.372 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:58.372 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/clusters/fixed-2.yaml").st_ino) 2024-04-26T21:00:58.372 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:58.570 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629614 2024-04-26T21:00:58.574 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:58.585 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:58.585 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:58.585 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:58.585 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:58.585 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/clusters/+").st_ino) 2024-04-26T21:00:58.585 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:58.775 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629613 2024-04-26T21:00:58.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:58.791 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:58.791 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:58.791 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:58.791 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:58.791 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/msgr-failures").st_ino) 2024-04-26T21:00:58.791 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:58.987 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629618 2024-04-26T21:00:58.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:59.003 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:59.003 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:59.003 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:59.003 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:59.003 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/msgr-failures/few.yaml").st_ino) 2024-04-26T21:00:59.003 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:59.203 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629619 2024-04-26T21:00:59.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:59.217 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:59.218 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:59.218 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:59.218 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:59.218 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/objectstore").st_ino) 2024-04-26T21:00:59.218 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:59.412 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629620 2024-04-26T21:00:59.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:59.427 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:59.428 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:59.428 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:59.428 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:59.428 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/fs").st_ino) 2024-04-26T21:00:59.428 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:59.623 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629616 2024-04-26T21:00:59.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:59.638 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:59.638 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:59.638 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:59.638 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:59.638 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/fs/xfs.yaml").st_ino) 2024-04-26T21:00:59.638 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:00:59.831 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629617 2024-04-26T21:00:59.835 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:00:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:00:59.846 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:00:59.846 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:00:59.846 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:00:59.846 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:00:59.846 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/workloads").st_ino) 2024-04-26T21:00:59.846 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:00.043 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629624 2024-04-26T21:01:00.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:00.058 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:00.058 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:00.058 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:00.058 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:00.058 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/workloads/rbd_fsx_cache_writeback.yaml").st_ino) 2024-04-26T21:01:00.058 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:00.251 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629630 2024-04-26T21:01:00.256 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:00.266 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:00.266 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:00.266 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:00.266 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:00.266 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/workloads/rbd_fsx_copy_on_read.yaml").st_ino) 2024-04-26T21:01:00.266 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:00.466 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629632 2024-04-26T21:01:00.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:00.482 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:00.482 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:00.482 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:00.482 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:00.482 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/workloads/rbd_fsx_journal.yaml").st_ino) 2024-04-26T21:01:00.482 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:00.678 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629633 2024-04-26T21:01:00.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:00.693 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:00.693 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:00.693 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:00.693 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:00.693 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/workloads/rbd_api_tests_journaling.yaml").st_ino) 2024-04-26T21:01:00.693 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:00.890 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629628 2024-04-26T21:01:00.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:00.905 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:00.905 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:00.905 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:00.905 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:00.905 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/workloads/rbd_fsx_nocache.yaml").st_ino) 2024-04-26T21:01:00.905 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:01.101 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629634 2024-04-26T21:01:01.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:01.115 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:01.115 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:01.115 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:01.116 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:01.116 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/workloads/rbd_api_tests.yaml").st_ino) 2024-04-26T21:01:01.116 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:01.313 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629626 2024-04-26T21:01:01.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:01.328 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:01.328 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:01.328 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:01.328 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:01.328 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/workloads/journal.yaml").st_ino) 2024-04-26T21:01:01.328 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:01.526 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629625 2024-04-26T21:01:01.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:01.541 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:01.541 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:01.542 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:01.542 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:01.542 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/workloads/rbd_api_tests_copy_on_read.yaml").st_ino) 2024-04-26T21:01:01.542 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:01.728 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629627 2024-04-26T21:01:01.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:01.743 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:01.743 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:01.743 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:01.743 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:01.743 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/workloads/rbd_api_tests_no_locking.yaml").st_ino) 2024-04-26T21:01:01.743 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:01.916 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629629 2024-04-26T21:01:01.920 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:01.929 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:01.929 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:01.929 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:01.930 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:01.930 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/workloads/rbd_fsx_cache_writethrough.yaml").st_ino) 2024-04-26T21:01:01.930 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:02.122 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629631 2024-04-26T21:01:02.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:02.137 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:02.137 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:02.137 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:02.137 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:02.137 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/%").st_ino) 2024-04-26T21:01:02.137 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:02.333 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629609 2024-04-26T21:01:02.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:02.348 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:02.348 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:02.348 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:02.348 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:02.348 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/thrashers").st_ino) 2024-04-26T21:01:02.349 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:02.546 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629621 2024-04-26T21:01:02.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:02.562 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:02.562 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:02.562 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:02.562 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:02.562 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/thrashers/default.yaml").st_ino) 2024-04-26T21:01:02.562 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:02.764 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629623 2024-04-26T21:01:02.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:02.779 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:02.779 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:02.779 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:02.779 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:02.780 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/thrash/thrashers/cache.yaml").st_ino) 2024-04-26T21:01:02.780 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:02.976 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629622 2024-04-26T21:01:02.980 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:02.991 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:02.991 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:02.991 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:02.991 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:02.991 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/mirror").st_ino) 2024-04-26T21:01:02.991 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:03.183 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629530 2024-04-26T21:01:03.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:03.198 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:03.198 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:03.198 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:03.198 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:03.198 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/mirror/base").st_ino) 2024-04-26T21:01:03.198 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:03.393 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629532 2024-04-26T21:01:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:03.408 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:03.408 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:03.408 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:03.409 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:03.409 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/mirror/base/install.yaml").st_ino) 2024-04-26T21:01:03.409 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:03.608 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629533 2024-04-26T21:01:03.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:03.624 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:03.624 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:03.624 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:03.624 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:03.624 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/mirror/cluster").st_ino) 2024-04-26T21:01:03.624 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:03.819 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629534 2024-04-26T21:01:03.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:03.834 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:03.834 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:03.835 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:03.835 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:03.835 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/mirror/cluster/openstack.yaml").st_ino) 2024-04-26T21:01:03.835 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:04.033 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629537 2024-04-26T21:01:04.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:04.048 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:04.048 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:04.048 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:04.048 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:04.048 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/mirror/cluster/2-node.yaml").st_ino) 2024-04-26T21:01:04.048 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:04.244 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629536 2024-04-26T21:01:04.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:04.260 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:04.260 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:04.260 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:04.260 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:04.260 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/mirror/cluster/+").st_ino) 2024-04-26T21:01:04.260 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:04.455 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629535 2024-04-26T21:01:04.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:04.470 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:04.470 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:04.470 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:04.470 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:04.470 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/mirror/msgr-failures").st_ino) 2024-04-26T21:01:04.470 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:04.666 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629539 2024-04-26T21:01:04.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:04.681 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:04.696 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:04.696 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:04.696 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:04.696 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/mirror/objectstore").st_ino) 2024-04-26T21:01:04.696 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:04.876 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629540 2024-04-26T21:01:04.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:04.891 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:04.891 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:04.891 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:04.891 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:04.891 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/mirror/fs").st_ino) 2024-04-26T21:01:04.892 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:05.075 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629538 2024-04-26T21:01:05.079 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:05.089 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:05.090 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:05.090 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:05.090 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:05.090 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/mirror/workloads").st_ino) 2024-04-26T21:01:05.090 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:05.289 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629543 2024-04-26T21:01:05.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:05.305 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:05.305 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:05.305 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:05.305 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:05.305 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/mirror/workloads/rbd-mirror-stress-workunit.yaml").st_ino) 2024-04-26T21:01:05.305 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:05.503 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629544 2024-04-26T21:01:05.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:05.520 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:05.520 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:05.520 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:05.521 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:05.521 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/mirror/workloads/rbd-mirror-workunit.yaml").st_ino) 2024-04-26T21:01:05.521 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:05.715 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629545 2024-04-26T21:01:05.720 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:05.731 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:05.731 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:05.731 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:05.731 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:05.731 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/mirror/%").st_ino) 2024-04-26T21:01:05.731 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:05.928 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629531 2024-04-26T21:01:05.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:05.943 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:05.943 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:05.943 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:05.943 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:05.943 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/mirror/rbd-mirror").st_ino) 2024-04-26T21:01:05.943 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:06.140 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629541 2024-04-26T21:01:06.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:06.155 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:06.156 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:06.156 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:06.156 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:06.156 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/mirror/rbd-mirror/one-per-cluster.yaml").st_ino) 2024-04-26T21:01:06.156 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:06.350 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629542 2024-04-26T21:01:06.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:06.365 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:06.365 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:06.365 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:06.365 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:06.366 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu").st_ino) 2024-04-26T21:01:06.366 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:06.561 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629560 2024-04-26T21:01:06.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:06.576 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:06.576 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:06.576 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:06.576 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:06.576 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/features").st_ino) 2024-04-26T21:01:06.576 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:06.772 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629570 2024-04-26T21:01:06.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:06.786 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:06.786 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:06.787 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:06.787 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:06.787 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/features/defaults.yaml").st_ino) 2024-04-26T21:01:06.787 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:06.981 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629571 2024-04-26T21:01:06.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:06.997 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:06.997 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:06.997 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:06.997 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:06.997 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/features/journaling.yaml").st_ino) 2024-04-26T21:01:06.997 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:07.192 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629572 2024-04-26T21:01:07.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:07.207 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:07.207 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:07.207 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:07.207 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:07.207 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/pool").st_ino) 2024-04-26T21:01:07.208 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:07.401 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629577 2024-04-26T21:01:07.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:07.416 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:07.416 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:07.416 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:07.416 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:07.416 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/pool/small-cache-pool.yaml").st_ino) 2024-04-26T21:01:07.416 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:07.613 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629582 2024-04-26T21:01:07.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:07.630 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:07.630 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:07.630 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:07.630 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:07.630 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/pool/replicated-data-pool.yaml").st_ino) 2024-04-26T21:01:07.630 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:07.828 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629581 2024-04-26T21:01:07.832 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:07.842 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:07.843 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:07.843 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:07.843 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:07.843 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/pool/ec-cache-pool.yaml").st_ino) 2024-04-26T21:01:07.843 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:08.039 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629578 2024-04-26T21:01:08.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:08.054 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:08.054 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:08.054 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:08.054 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:08.054 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/pool/ec-data-pool.yaml").st_ino) 2024-04-26T21:01:08.054 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:08.244 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629579 2024-04-26T21:01:08.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:08.259 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:08.260 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:08.260 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:08.260 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:08.260 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/pool/none.yaml").st_ino) 2024-04-26T21:01:08.260 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:08.456 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629580 2024-04-26T21:01:08.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:08.471 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:08.471 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:08.471 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:08.471 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:08.471 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/clusters").st_ino) 2024-04-26T21:01:08.471 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:08.664 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629566 2024-04-26T21:01:08.668 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:08.678 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:08.678 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:08.678 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:08.678 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:08.678 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/clusters/openstack.yaml").st_ino) 2024-04-26T21:01:08.678 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:08.883 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629569 2024-04-26T21:01:08.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:08.898 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:08.898 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:08.899 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:08.899 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:08.899 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/clusters/+").st_ino) 2024-04-26T21:01:08.899 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:09.095 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629567 2024-04-26T21:01:09.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:09.111 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:09.111 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:09.111 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:09.111 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:09.111 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/clusters/fixed-3.yaml").st_ino) 2024-04-26T21:01:09.111 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:09.307 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629568 2024-04-26T21:01:09.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:09.321 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:09.321 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:09.321 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:09.321 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:09.321 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/msgr-failures").st_ino) 2024-04-26T21:01:09.321 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:09.518 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629574 2024-04-26T21:01:09.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:09.532 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:09.533 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:09.533 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:09.533 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:09.533 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/msgr-failures/few.yaml").st_ino) 2024-04-26T21:01:09.533 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:09.721 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629575 2024-04-26T21:01:09.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:09.736 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:09.736 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:09.736 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:09.736 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:09.736 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/objectstore").st_ino) 2024-04-26T21:01:09.736 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:09.937 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629576 2024-04-26T21:01:09.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:09.953 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:09.953 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:09.953 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:09.953 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:09.953 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/fs").st_ino) 2024-04-26T21:01:09.953 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:10.144 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629573 2024-04-26T21:01:10.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:10.159 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:10.159 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:10.159 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:10.160 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:10.160 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/workloads").st_ino) 2024-04-26T21:01:10.160 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:10.357 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629583 2024-04-26T21:01:10.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:10.373 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:10.373 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:10.373 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:10.373 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:10.373 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/workloads/qemu_iozone.yaml.disabled").st_ino) 2024-04-26T21:01:10.373 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:10.570 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629587 2024-04-26T21:01:10.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:10.586 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:10.586 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:10.586 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:10.586 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:10.586 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/workloads/#qemu_xfstests.yaml#").st_ino) 2024-04-26T21:01:10.586 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:10.787 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629584 2024-04-26T21:01:10.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:10.802 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:10.802 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:10.802 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:10.802 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:10.802 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/workloads/qemu_xfstests.yaml").st_ino) 2024-04-26T21:01:10.803 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:10.997 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629588 2024-04-26T21:01:11.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:11.012 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:11.012 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:11.013 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:11.013 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:11.013 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/workloads/qemu_fsstress.yaml").st_ino) 2024-04-26T21:01:11.013 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:11.215 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629586 2024-04-26T21:01:11.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:11.229 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:11.229 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:11.229 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:11.230 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:11.230 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/workloads/qemu_bonnie.yaml").st_ino) 2024-04-26T21:01:11.230 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:11.423 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629585 2024-04-26T21:01:11.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:11.439 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:11.439 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:11.439 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:11.439 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:11.439 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/cache").st_ino) 2024-04-26T21:01:11.439 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:11.634 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629562 2024-04-26T21:01:11.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:11.650 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:11.650 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:11.650 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:11.650 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:11.651 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/cache/writeback.yaml").st_ino) 2024-04-26T21:01:11.651 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:11.847 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629564 2024-04-26T21:01:11.852 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:11.864 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:11.864 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:11.864 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:11.864 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:11.864 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/cache/writethrough.yaml").st_ino) 2024-04-26T21:01:11.864 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:12.058 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629565 2024-04-26T21:01:12.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:12.073 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:12.074 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:12.074 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:12.074 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:12.074 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/cache/none.yaml").st_ino) 2024-04-26T21:01:12.074 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:12.270 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629563 2024-04-26T21:01:12.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:12.287 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:12.287 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:12.288 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:12.288 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:12.288 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rbd/qemu/%").st_ino) 2024-04-26T21:01:12.288 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:12.489 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629561 2024-04-26T21:01:12.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:12.504 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:12.504 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:12.504 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:12.504 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:12.504 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rest").st_ino) 2024-04-26T21:01:12.505 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:12.700 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629653 2024-04-26T21:01:12.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:12.715 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:12.715 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:12.715 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:12.715 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:12.715 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rest/basic").st_ino) 2024-04-26T21:01:12.715 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:12.899 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629654 2024-04-26T21:01:12.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:12.914 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:12.914 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:12.914 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:12.914 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:12.914 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rest/basic/tasks").st_ino) 2024-04-26T21:01:12.914 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:13.110 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629655 2024-04-26T21:01:13.114 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:13.125 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:13.125 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:13.125 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:13.125 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:13.125 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rest/basic/tasks/rest_test.yaml").st_ino) 2024-04-26T21:01:13.125 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:13.322 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629656 2024-04-26T21:01:13.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:13.337 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:13.337 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:13.337 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:13.337 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:13.337 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw").st_ino) 2024-04-26T21:01:13.337 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:13.533 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629657 2024-04-26T21:01:13.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:13.548 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:13.548 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:13.548 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:13.548 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:13.548 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/multifs").st_ino) 2024-04-26T21:01:13.548 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:13.746 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629658 2024-04-26T21:01:13.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:13.762 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:13.762 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:13.762 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:13.762 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:13.762 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/multifs/clusters").st_ino) 2024-04-26T21:01:13.762 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:13.950 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629660 2024-04-26T21:01:13.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:13.964 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:13.965 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:13.965 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:13.965 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:13.965 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/multifs/clusters/fixed-2.yaml").st_ino) 2024-04-26T21:01:13.965 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:14.163 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629661 2024-04-26T21:01:14.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:14.179 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:14.179 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:14.179 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:14.179 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:14.179 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/multifs/objectstore").st_ino) 2024-04-26T21:01:14.179 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:14.380 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629667 2024-04-26T21:01:14.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:14.395 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:14.395 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:14.395 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:14.395 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:14.395 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/multifs/fs").st_ino) 2024-04-26T21:01:14.395 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:14.590 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629665 2024-04-26T21:01:14.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:14.605 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:14.605 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:14.605 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:14.605 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:14.605 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/multifs/fs/xfs.yaml").st_ino) 2024-04-26T21:01:14.605 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:14.809 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629666 2024-04-26T21:01:14.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:14.826 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:14.826 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:14.826 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:14.826 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:14.826 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/multifs/rgw_pool_type").st_ino) 2024-04-26T21:01:14.826 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:15.022 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629669 2024-04-26T21:01:15.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:15.037 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:15.037 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:15.038 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:15.038 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:15.038 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/multifs/frontend").st_ino) 2024-04-26T21:01:15.038 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:15.230 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629662 2024-04-26T21:01:15.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:15.246 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:15.246 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:15.246 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:15.246 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:15.246 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/multifs/frontend/apache.yaml").st_ino) 2024-04-26T21:01:15.246 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:15.442 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629663 2024-04-26T21:01:15.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:15.457 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:15.457 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:15.457 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:15.458 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:15.458 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/multifs/frontend/civetweb.yaml").st_ino) 2024-04-26T21:01:15.458 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:15.652 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629664 2024-04-26T21:01:15.657 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:15.667 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:15.667 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:15.668 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:15.668 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:15.668 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/multifs/overrides.yaml").st_ino) 2024-04-26T21:01:15.668 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:15.865 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629668 2024-04-26T21:01:15.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:15.881 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:15.881 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:15.881 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:15.881 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:15.881 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/multifs/%").st_ino) 2024-04-26T21:01:15.881 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:16.080 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629659 2024-04-26T21:01:16.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:16.097 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:16.097 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:16.097 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:16.097 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:16.097 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/multifs/tasks").st_ino) 2024-04-26T21:01:16.097 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:16.290 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629670 2024-04-26T21:01:16.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:16.305 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:16.305 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:16.305 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:16.306 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:16.306 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/multifs/tasks/rgw_roundtrip.yaml").st_ino) 2024-04-26T21:01:16.306 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:16.502 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629674 2024-04-26T21:01:16.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:16.517 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:16.517 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:16.517 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:16.517 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:16.517 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/multifs/tasks/rgw_multipart_upload.yaml").st_ino) 2024-04-26T21:01:16.517 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:16.648 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:01:16.656 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:01:16.705 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629672 2024-04-26T21:01:16.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:16.719 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:16.719 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:16.719 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:16.719 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:16.719 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/multifs/tasks/rgw_user_quota.yaml").st_ino) 2024-04-26T21:01:16.719 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:16.852 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629677 2024-04-26T21:01:16.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:16.867 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:16.867 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:16.867 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:16.867 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:16.867 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/multifs/tasks/rgw_readwrite.yaml").st_ino) 2024-04-26T21:01:16.867 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:17.016 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629673 2024-04-26T21:01:17.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:17.032 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:17.032 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:17.032 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:17.032 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:17.032 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/multifs/tasks/rgw_swift.yaml").st_ino) 2024-04-26T21:01:17.032 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:17.230 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629676 2024-04-26T21:01:17.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:17.246 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:17.246 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:17.246 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:17.246 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:17.246 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/multifs/tasks/rgw_s3tests.yaml").st_ino) 2024-04-26T21:01:17.246 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:17.451 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629675 2024-04-26T21:01:17.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:17.466 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:17.466 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:17.466 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:17.466 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:17.467 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/multifs/tasks/rgw_bucket_quota.yaml").st_ino) 2024-04-26T21:01:17.467 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:17.668 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629671 2024-04-26T21:01:17.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:17.684 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:17.684 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:17.684 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:17.684 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:17.684 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/singleton").st_ino) 2024-04-26T21:01:17.684 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:17.877 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629678 2024-04-26T21:01:17.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:17.892 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:17.892 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:17.892 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:17.892 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:17.892 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/singleton/objectstore").st_ino) 2024-04-26T21:01:17.892 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:18.087 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629690 2024-04-26T21:01:18.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:18.102 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:18.102 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:18.102 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:18.102 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:18.102 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/singleton/fs").st_ino) 2024-04-26T21:01:18.103 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:18.298 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629688 2024-04-26T21:01:18.302 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:18.313 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:18.313 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:18.313 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:18.313 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:18.313 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/singleton/fs/xfs.yaml").st_ino) 2024-04-26T21:01:18.313 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:18.496 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629689 2024-04-26T21:01:18.501 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:18.511 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:18.511 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:18.511 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:18.511 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:18.511 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/singleton/rgw_pool_type").st_ino) 2024-04-26T21:01:18.511 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:18.709 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629692 2024-04-26T21:01:18.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:18.724 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:18.724 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:18.724 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:18.724 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:18.724 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/singleton/frontend").st_ino) 2024-04-26T21:01:18.724 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:18.921 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629685 2024-04-26T21:01:18.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:18.937 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:18.937 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:18.937 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:18.937 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:18.937 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/singleton/frontend/apache.yaml").st_ino) 2024-04-26T21:01:18.937 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:19.134 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629686 2024-04-26T21:01:19.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:19.149 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:19.149 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:19.149 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:19.149 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:19.149 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/singleton/frontend/civetweb.yaml").st_ino) 2024-04-26T21:01:19.149 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:19.349 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629687 2024-04-26T21:01:19.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:19.364 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:19.364 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:19.364 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:19.364 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:19.364 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/singleton/overrides.yaml").st_ino) 2024-04-26T21:01:19.364 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:19.548 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629691 2024-04-26T21:01:19.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:19.564 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:19.564 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:19.564 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:19.565 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:19.565 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/singleton/%").st_ino) 2024-04-26T21:01:19.565 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:19.762 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629679 2024-04-26T21:01:19.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:19.777 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:19.777 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:19.777 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:19.777 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:19.777 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/singleton/xfs.yaml").st_ino) 2024-04-26T21:01:19.777 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:19.974 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629693 2024-04-26T21:01:19.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:19.989 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:19.989 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:19.989 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:19.989 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:19.989 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/singleton/all").st_ino) 2024-04-26T21:01:19.989 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:20.193 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629680 2024-04-26T21:01:20.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:20.209 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:20.209 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:20.209 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:20.209 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:20.209 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/singleton/all/radosgw-admin-multi-region.yaml").st_ino) 2024-04-26T21:01:20.209 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:20.405 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629682 2024-04-26T21:01:20.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:20.420 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:20.421 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:20.421 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:20.421 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:20.421 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/singleton/all/radosgw-admin.yaml").st_ino) 2024-04-26T21:01:20.421 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:20.617 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629683 2024-04-26T21:01:20.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:20.632 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:20.632 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:20.632 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:20.632 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:20.632 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/singleton/all/radosgw-admin-data-sync.yaml").st_ino) 2024-04-26T21:01:20.632 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:20.836 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629681 2024-04-26T21:01:20.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:20.852 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:20.852 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:20.852 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:20.852 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:20.852 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/singleton/all/radosgw-convert-to-region.yaml").st_ino) 2024-04-26T21:01:20.852 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:21.051 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629684 2024-04-26T21:01:21.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:21.066 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:21.066 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:21.066 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:21.067 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:21.067 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify").st_ino) 2024-04-26T21:01:21.067 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:21.268 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629694 2024-04-26T21:01:21.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:21.283 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:21.283 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:21.283 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:21.283 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:21.284 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify/validater").st_ino) 2024-04-26T21:01:21.284 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:21.481 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629712 2024-04-26T21:01:21.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:21.496 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:21.496 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:21.496 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:21.496 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:21.496 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify/validater/valgrind.yaml").st_ino) 2024-04-26T21:01:21.496 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:21.693 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629714 2024-04-26T21:01:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:21.708 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:21.708 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:21.709 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:21.709 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:21.709 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify/validater/lockdep.yaml").st_ino) 2024-04-26T21:01:21.709 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:21.907 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629713 2024-04-26T21:01:21.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:21.923 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:21.923 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:21.924 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:21.924 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:21.924 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify/clusters").st_ino) 2024-04-26T21:01:21.924 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:22.122 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629696 2024-04-26T21:01:22.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:22.137 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:22.137 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:22.137 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:22.137 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:22.138 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify/clusters/fixed-2.yaml").st_ino) 2024-04-26T21:01:22.138 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:22.335 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629697 2024-04-26T21:01:22.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:22.349 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:22.349 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:22.349 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:22.349 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:22.349 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify/msgr-failures").st_ino) 2024-04-26T21:01:22.349 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:22.546 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629703 2024-04-26T21:01:22.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:22.562 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:22.563 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:22.563 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:22.563 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:22.563 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify/msgr-failures/few.yaml").st_ino) 2024-04-26T21:01:22.563 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:22.760 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629704 2024-04-26T21:01:22.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:22.775 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:22.775 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:22.775 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:22.775 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:22.775 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify/objectstore").st_ino) 2024-04-26T21:01:22.775 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:22.963 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629705 2024-04-26T21:01:22.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:22.977 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:22.977 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:22.977 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:22.977 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:22.977 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify/fs").st_ino) 2024-04-26T21:01:22.978 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:23.184 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629701 2024-04-26T21:01:23.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:23.199 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:23.199 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:23.200 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:23.200 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:23.200 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify/fs/xfs.yaml").st_ino) 2024-04-26T21:01:23.200 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:23.397 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629702 2024-04-26T21:01:23.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:23.412 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:23.412 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:23.412 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:23.412 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:23.412 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify/rgw_pool_type").st_ino) 2024-04-26T21:01:23.412 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:23.609 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629707 2024-04-26T21:01:23.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:23.624 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:23.624 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:23.624 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:23.624 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:23.624 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify/frontend").st_ino) 2024-04-26T21:01:23.624 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:23.822 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629698 2024-04-26T21:01:23.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:23.837 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:23.837 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:23.837 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:23.837 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:23.837 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify/frontend/apache.yaml").st_ino) 2024-04-26T21:01:23.837 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:24.030 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629699 2024-04-26T21:01:24.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:24.045 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:24.045 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:24.045 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:24.045 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:24.045 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify/frontend/civetweb.yaml").st_ino) 2024-04-26T21:01:24.045 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:24.244 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629700 2024-04-26T21:01:24.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:24.260 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:24.260 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:24.260 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:24.260 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:24.260 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify/overrides.yaml").st_ino) 2024-04-26T21:01:24.260 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:24.455 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629706 2024-04-26T21:01:24.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:24.471 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:24.471 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:24.471 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:24.471 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:24.471 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify/%").st_ino) 2024-04-26T21:01:24.471 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:24.669 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629695 2024-04-26T21:01:24.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:24.683 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:24.683 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:24.683 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:24.684 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:24.684 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify/tasks").st_ino) 2024-04-26T21:01:24.684 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:24.878 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629708 2024-04-26T21:01:24.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:24.893 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:24.893 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:24.893 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:24.893 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:24.893 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify/tasks/rgw_s3tests_multiregion.yaml").st_ino) 2024-04-26T21:01:24.893 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:25.092 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629710 2024-04-26T21:01:25.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:25.107 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:25.107 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:25.107 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:25.107 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:25.107 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify/tasks/rgw_swift.yaml").st_ino) 2024-04-26T21:01:25.108 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:25.305 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629711 2024-04-26T21:01:25.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:25.321 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:25.321 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:25.321 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:25.321 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:25.321 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rgw/verify/tasks/rgw_s3tests.yaml").st_ino) 2024-04-26T21:01:25.321 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:25.517 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629709 2024-04-26T21:01:25.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:25.532 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:25.532 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:25.532 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:25.532 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:25.532 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-deploy").st_ino) 2024-04-26T21:01:25.532 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:25.727 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628523 2024-04-26T21:01:25.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:25.741 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:25.741 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:25.742 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:25.742 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:25.742 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-deploy/basic").st_ino) 2024-04-26T21:01:25.742 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:25.935 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628524 2024-04-26T21:01:25.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:25.950 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:25.950 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:25.950 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:25.950 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:25.950 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-deploy/basic/ceph-deploy-overrides").st_ino) 2024-04-26T21:01:25.950 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:26.147 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628526 2024-04-26T21:01:26.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:26.162 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:26.162 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:26.162 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:26.162 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:26.162 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-deploy/basic/python_versions").st_ino) 2024-04-26T21:01:26.162 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:26.357 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628529 2024-04-26T21:01:26.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:26.371 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:26.372 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:26.372 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:26.372 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:26.372 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-deploy/basic/python_versions/python_3.yaml").st_ino) 2024-04-26T21:01:26.372 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:26.568 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628531 2024-04-26T21:01:26.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:26.583 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:26.583 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:26.583 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:26.583 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:26.584 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-deploy/basic/python_versions/python_2.yaml").st_ino) 2024-04-26T21:01:26.584 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:26.781 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628530 2024-04-26T21:01:26.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:26.796 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:26.796 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:26.796 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:26.796 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:26.796 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-deploy/basic/config_options").st_ino) 2024-04-26T21:01:26.796 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:26.993 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628527 2024-04-26T21:01:26.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:27.009 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:27.009 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:27.009 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:27.010 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:27.010 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-deploy/basic/distros").st_ino) 2024-04-26T21:01:27.010 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:27.206 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628528 2024-04-26T21:01:27.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:27.224 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:27.224 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:27.224 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:27.224 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:27.224 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-deploy/basic/%").st_ino) 2024-04-26T21:01:27.224 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:27.420 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628525 2024-04-26T21:01:27.425 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:27.435 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:27.435 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:27.435 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:27.435 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:27.435 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-deploy/basic/tasks").st_ino) 2024-04-26T21:01:27.435 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:27.624 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628532 2024-04-26T21:01:27.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:27.639 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:27.639 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:27.640 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:27.640 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:27.640 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-deploy/basic/tasks/ceph-admin-commands.yaml").st_ino) 2024-04-26T21:01:27.640 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:27.839 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628533 2024-04-26T21:01:27.843 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:27.854 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:27.854 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:27.854 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:27.854 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:27.854 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt").st_ino) 2024-04-26T21:01:27.854 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:28.047 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629852 2024-04-26T21:01:28.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:28.062 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:28.062 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:28.062 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:28.062 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:28.062 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic").st_ino) 2024-04-26T21:01:28.062 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:28.257 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629853 2024-04-26T21:01:28.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:28.272 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:28.272 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:28.272 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:28.272 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:28.272 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/clusters").st_ino) 2024-04-26T21:01:28.272 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:28.474 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629855 2024-04-26T21:01:28.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:28.487 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:28.488 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:28.488 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:28.488 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:28.488 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/clusters/fixed-3.yaml").st_ino) 2024-04-26T21:01:28.488 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:28.685 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629856 2024-04-26T21:01:28.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:28.701 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:28.701 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:28.701 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:28.701 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:28.701 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/msgr-failures").st_ino) 2024-04-26T21:01:28.701 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:28.904 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629859 2024-04-26T21:01:28.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:28.919 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:28.919 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:28.919 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:28.919 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:28.919 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/msgr-failures/many.yaml").st_ino) 2024-04-26T21:01:28.920 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:29.115 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629861 2024-04-26T21:01:29.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:29.131 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:29.131 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:29.131 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:29.131 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:29.131 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/msgr-failures/few.yaml").st_ino) 2024-04-26T21:01:29.131 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:29.327 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629860 2024-04-26T21:01:29.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:29.342 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:29.343 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:29.343 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:29.343 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:29.343 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/fs").st_ino) 2024-04-26T21:01:29.343 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:29.538 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629857 2024-04-26T21:01:29.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:29.553 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:29.554 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:29.554 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:29.554 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:29.554 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/fs/btrfs.yaml").st_ino) 2024-04-26T21:01:29.554 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:29.751 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629858 2024-04-26T21:01:29.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:29.766 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:29.766 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:29.766 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:29.766 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:29.766 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/%").st_ino) 2024-04-26T21:01:29.766 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:29.963 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629854 2024-04-26T21:01:29.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:29.978 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:29.978 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:29.978 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:29.979 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:29.979 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/tasks").st_ino) 2024-04-26T21:01:29.979 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:30.175 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629862 2024-04-26T21:01:30.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:30.190 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:30.191 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:30.191 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:30.191 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:30.191 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/tasks/iozone-sync.yaml").st_ino) 2024-04-26T21:01:30.191 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:30.389 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629873 2024-04-26T21:01:30.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:30.404 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:30.404 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:30.404 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:30.404 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:30.404 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/tasks/dbench-short.yaml").st_ino) 2024-04-26T21:01:30.404 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:30.599 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629865 2024-04-26T21:01:30.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:30.615 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:30.615 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:30.615 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:30.615 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:30.615 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/tasks/blogbench.yaml").st_ino) 2024-04-26T21:01:30.615 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:30.813 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629863 2024-04-26T21:01:30.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:30.828 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:30.828 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:30.828 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:30.829 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:30.829 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/tasks/ffsb.yaml").st_ino) 2024-04-26T21:01:30.829 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:31.025 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629867 2024-04-26T21:01:31.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:31.040 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:31.040 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:31.040 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:31.040 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:31.040 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/tasks/pjd.yaml").st_ino) 2024-04-26T21:01:31.040 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:31.233 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629875 2024-04-26T21:01:31.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:31.248 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:31.249 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:31.249 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:31.249 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:31.249 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/tasks/fsync-tester.yaml").st_ino) 2024-04-26T21:01:31.249 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:31.446 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629871 2024-04-26T21:01:31.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:31.461 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:31.461 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:31.461 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:31.461 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:31.461 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/tasks/iogen.yaml").st_ino) 2024-04-26T21:01:31.461 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:31.666 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629872 2024-04-26T21:01:31.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:31.680 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:31.680 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:31.680 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:31.680 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:31.681 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/tasks/fsx.yaml").st_ino) 2024-04-26T21:01:31.681 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:31.877 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629870 2024-04-26T21:01:31.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:31.892 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:31.892 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:31.892 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:31.892 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:31.892 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/tasks/bonnie.yaml").st_ino) 2024-04-26T21:01:31.892 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:32.080 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629864 2024-04-26T21:01:32.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:32.097 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:32.097 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:32.097 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:32.098 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:32.098 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/tasks/fio.yaml").st_ino) 2024-04-26T21:01:32.098 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:32.292 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629868 2024-04-26T21:01:32.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:32.307 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:32.307 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:32.307 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:32.307 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:32.307 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/tasks/iozone.yaml").st_ino) 2024-04-26T21:01:32.308 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:32.506 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629874 2024-04-26T21:01:32.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:32.520 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:32.520 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:32.520 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:32.520 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:32.520 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/tasks/dbench.yaml").st_ino) 2024-04-26T21:01:32.520 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:32.709 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629866 2024-04-26T21:01:32.713 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:32.724 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:32.724 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:32.724 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:32.724 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:32.724 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/tgt/basic/tasks/fsstress.yaml").st_ino) 2024-04-26T21:01:32.724 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:32.922 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629869 2024-04-26T21:01:32.927 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:32.937 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:32.937 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:32.937 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:32.937 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:32.937 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/samba").st_ino) 2024-04-26T21:01:32.937 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:33.133 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629715 2024-04-26T21:01:33.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:33.148 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:33.148 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:33.148 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:33.148 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:33.148 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/samba/install").st_ino) 2024-04-26T21:01:33.149 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:33.345 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629721 2024-04-26T21:01:33.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:33.360 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:33.360 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:33.360 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:33.360 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:33.360 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/samba/install/install.yaml").st_ino) 2024-04-26T21:01:33.360 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:33.556 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629722 2024-04-26T21:01:33.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:33.571 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:33.571 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:33.571 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:33.571 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:33.571 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/samba/clusters").st_ino) 2024-04-26T21:01:33.571 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:33.757 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629717 2024-04-26T21:01:33.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:33.772 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:33.772 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:33.772 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:33.772 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:33.772 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/samba/clusters/samba-basic.yaml").st_ino) 2024-04-26T21:01:33.772 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:33.968 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629718 2024-04-26T21:01:33.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:33.982 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:33.982 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:33.982 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:33.982 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:33.982 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/samba/workload").st_ino) 2024-04-26T21:01:33.983 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:34.178 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629728 2024-04-26T21:01:34.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:34.194 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:34.194 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:34.194 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:34.194 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:34.194 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/samba/workload/cifs-kernel-build.yaml.disabled").st_ino) 2024-04-26T21:01:34.194 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:34.396 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629731 2024-04-26T21:01:34.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:34.410 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:34.410 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:34.411 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:34.411 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:34.411 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/samba/workload/cifs-fsstress.yaml").st_ino) 2024-04-26T21:01:34.411 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:34.606 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629730 2024-04-26T21:01:34.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:34.620 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:34.620 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:34.621 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:34.621 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:34.621 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/samba/workload/smbtorture.yaml").st_ino) 2024-04-26T21:01:34.621 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:34.817 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629732 2024-04-26T21:01:34.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:34.832 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:34.832 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:34.832 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:34.832 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:34.832 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/samba/workload/cifs-dbench.yaml").st_ino) 2024-04-26T21:01:34.832 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:35.037 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629729 2024-04-26T21:01:35.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:35.052 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:35.052 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:35.052 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:35.052 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:35.052 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/samba/fs").st_ino) 2024-04-26T21:01:35.052 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:35.247 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629719 2024-04-26T21:01:35.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:35.262 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:35.262 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:35.262 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:35.262 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:35.262 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/samba/fs/xfs.yaml").st_ino) 2024-04-26T21:01:35.262 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:35.459 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629720 2024-04-26T21:01:35.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:35.473 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:35.473 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:35.473 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:35.473 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:35.474 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/samba/mount").st_ino) 2024-04-26T21:01:35.474 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:35.670 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629723 2024-04-26T21:01:35.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:35.687 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:35.687 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:35.687 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:35.687 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:35.687 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/samba/mount/noceph.yaml").st_ino) 2024-04-26T21:01:35.687 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:35.882 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629727 2024-04-26T21:01:35.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:35.895 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:35.895 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:35.895 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:35.895 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:35.895 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/samba/mount/kclient.yaml").st_ino) 2024-04-26T21:01:35.895 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:36.091 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629725 2024-04-26T21:01:36.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:36.106 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:36.106 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:36.106 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:36.106 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:36.106 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/samba/mount/fuse.yaml").st_ino) 2024-04-26T21:01:36.106 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:36.302 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629724 2024-04-26T21:01:36.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:36.318 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:36.318 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:36.318 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:36.318 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:36.318 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/samba/mount/native.yaml").st_ino) 2024-04-26T21:01:36.318 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:36.505 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629726 2024-04-26T21:01:36.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:36.519 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:36.519 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:36.519 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:36.519 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:36.519 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/samba/%").st_ino) 2024-04-26T21:01:36.520 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:36.716 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629716 2024-04-26T21:01:36.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:36.732 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:36.732 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:36.732 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:36.732 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:36.732 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible").st_ino) 2024-04-26T21:01:36.732 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:36.914 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628496 2024-04-26T21:01:36.919 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:36.929 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:36.930 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:36.930 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:36.930 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:36.930 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke").st_ino) 2024-04-26T21:01:36.930 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:37.124 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628497 2024-04-26T21:01:37.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:37.139 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:37.139 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:37.139 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:37.139 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:37.139 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/5-tests").st_ino) 2024-04-26T21:01:37.139 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:37.337 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628518 2024-04-26T21:01:37.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:37.352 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:37.352 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:37.352 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:37.352 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:37.352 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/5-tests/cls.yaml").st_ino) 2024-04-26T21:01:37.352 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:37.546 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628520 2024-04-26T21:01:37.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:37.561 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:37.561 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:37.561 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:37.561 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:37.561 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/5-tests/rbd_cli_tests.yaml.disabled").st_ino) 2024-04-26T21:01:37.561 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:37.756 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628521 2024-04-26T21:01:37.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:37.771 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:37.771 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:37.771 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:37.771 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:37.772 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/5-tests/rbd_import_export.yaml").st_ino) 2024-04-26T21:01:37.772 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:37.969 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628522 2024-04-26T21:01:37.974 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:37.984 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:37.984 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:37.984 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:37.984 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:37.984 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/5-tests/ceph-admin-commands.yaml").st_ino) 2024-04-26T21:01:37.984 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:38.178 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628519 2024-04-26T21:01:38.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:38.194 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:38.194 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:38.194 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:38.194 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:38.194 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/1-distros").st_ino) 2024-04-26T21:01:38.194 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:38.388 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628502 2024-04-26T21:01:38.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:38.404 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:38.404 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:38.404 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:38.404 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:38.404 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/0-clusters").st_ino) 2024-04-26T21:01:38.404 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:38.604 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628499 2024-04-26T21:01:38.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:38.620 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:38.621 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:38.621 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:38.621 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:38.621 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/0-clusters/single_mon_osd.yaml").st_ino) 2024-04-26T21:01:38.621 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:38.820 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628501 2024-04-26T21:01:38.824 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:38.835 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:38.835 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:38.835 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:38.835 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:38.835 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/0-clusters/3-node.yaml").st_ino) 2024-04-26T21:01:38.835 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:39.032 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628500 2024-04-26T21:01:39.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:39.048 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:39.048 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:39.048 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:39.048 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:39.048 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/3-common").st_ino) 2024-04-26T21:01:39.048 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:39.243 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628505 2024-04-26T21:01:39.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:39.257 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:39.257 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:39.257 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:39.257 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:39.257 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/3-common/os_tuning").st_ino) 2024-04-26T21:01:39.257 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:39.454 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628507 2024-04-26T21:01:39.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:39.469 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:39.469 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:39.469 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:39.469 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:39.469 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/3-common/os_tuning/vm_friendly.yaml").st_ino) 2024-04-26T21:01:39.469 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:39.672 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628508 2024-04-26T21:01:39.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:39.686 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:39.686 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:39.686 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:39.686 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:39.686 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/3-common/source").st_ino) 2024-04-26T21:01:39.687 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:39.881 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628509 2024-04-26T21:01:39.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:39.896 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:39.896 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:39.896 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:39.897 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:39.897 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/3-common/source/upstream_stable.yaml").st_ino) 2024-04-26T21:01:39.897 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:40.098 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628510 2024-04-26T21:01:40.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:40.113 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:40.113 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:40.113 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:40.113 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:40.113 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/3-common/%").st_ino) 2024-04-26T21:01:40.113 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:40.296 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628506 2024-04-26T21:01:40.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:40.310 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:40.310 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:40.310 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:40.310 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:40.310 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/%").st_ino) 2024-04-26T21:01:40.310 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:40.501 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628498 2024-04-26T21:01:40.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:40.516 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:40.516 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:40.516 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:40.516 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:40.516 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/4-osd").st_ino) 2024-04-26T21:01:40.517 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:40.713 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628511 2024-04-26T21:01:40.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:40.729 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:40.729 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:40.729 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:40.729 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:40.729 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/4-osd/journal").st_ino) 2024-04-26T21:01:40.729 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:40.930 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628516 2024-04-26T21:01:40.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:40.946 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:40.946 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:40.946 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:40.946 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:40.946 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/4-osd/journal/collocated.yaml").st_ino) 2024-04-26T21:01:40.946 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:41.141 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628517 2024-04-26T21:01:41.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:41.156 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:41.156 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:41.156 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:41.156 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:41.157 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/4-osd/%").st_ino) 2024-04-26T21:01:41.157 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:41.354 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628512 2024-04-26T21:01:41.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:41.369 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:41.370 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:41.370 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:41.370 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:41.370 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/4-osd/devices").st_ino) 2024-04-26T21:01:41.370 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:41.567 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628513 2024-04-26T21:01:41.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:41.582 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:41.582 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:41.582 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:41.582 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:41.582 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/4-osd/devices/osd_auto_discovery.yaml").st_ino) 2024-04-26T21:01:41.582 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:41.777 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628515 2024-04-26T21:01:41.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:41.791 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:41.791 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:41.791 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:41.791 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:41.791 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/4-osd/devices/from_teuthology.yaml").st_ino) 2024-04-26T21:01:41.791 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:41.988 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628514 2024-04-26T21:01:41.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:42.003 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:42.004 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:42.004 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:42.004 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:42.004 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/2-setup").st_ino) 2024-04-26T21:01:42.004 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:42.200 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628503 2024-04-26T21:01:42.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:42.216 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:42.216 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:42.216 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:42.216 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:42.216 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-ansible/smoke/2-setup/ceph_ansible.yaml").st_ino) 2024-04-26T21:01:42.216 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:42.410 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628504 2024-04-26T21:01:42.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:42.425 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:42.425 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:42.425 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:42.425 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:42.425 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/big").st_ino) 2024-04-26T21:01:42.425 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:42.620 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628469 2024-04-26T21:01:42.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:42.635 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:42.635 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:42.635 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:42.635 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:42.635 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/big/rados-thrash").st_ino) 2024-04-26T21:01:42.636 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:42.831 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628470 2024-04-26T21:01:42.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:42.847 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:42.847 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:42.848 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:42.848 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:42.848 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/big/rados-thrash/clusters").st_ino) 2024-04-26T21:01:42.848 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:43.014 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628474 2024-04-26T21:01:43.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:43.028 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:43.028 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:43.028 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:43.028 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:43.028 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/big/rados-thrash/clusters/medium.yaml").st_ino) 2024-04-26T21:01:43.028 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:43.226 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628476 2024-04-26T21:01:43.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:43.240 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:43.240 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:43.240 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:43.240 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:43.240 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/big/rados-thrash/clusters/big.yaml").st_ino) 2024-04-26T21:01:43.240 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:43.440 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628475 2024-04-26T21:01:43.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:43.455 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:43.455 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:43.455 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:43.455 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:43.455 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/big/rados-thrash/clusters/small.yaml").st_ino) 2024-04-26T21:01:43.455 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:43.651 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628477 2024-04-26T21:01:43.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:43.666 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:43.666 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:43.666 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:43.666 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:43.666 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/big/rados-thrash/fs").st_ino) 2024-04-26T21:01:43.666 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:43.867 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628478 2024-04-26T21:01:43.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:43.881 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:43.881 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:43.882 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:43.882 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:43.882 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/big/rados-thrash/fs/btrfs.yaml").st_ino) 2024-04-26T21:01:43.882 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:44.081 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628479 2024-04-26T21:01:44.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:44.095 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:44.095 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:44.095 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:44.096 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:44.096 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/big/rados-thrash/fs/xfs.yaml").st_ino) 2024-04-26T21:01:44.096 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:44.293 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628480 2024-04-26T21:01:44.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:44.308 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:44.308 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:44.308 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:44.308 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:44.308 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/big/rados-thrash/workloads").st_ino) 2024-04-26T21:01:44.308 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:44.503 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628483 2024-04-26T21:01:44.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:44.519 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:44.519 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:44.519 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:44.519 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:44.519 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/big/rados-thrash/workloads/snaps-few-objects.yaml").st_ino) 2024-04-26T21:01:44.519 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:44.711 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628484 2024-04-26T21:01:44.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:44.726 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:44.726 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:44.726 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:44.726 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:44.726 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/big/rados-thrash/ceph").st_ino) 2024-04-26T21:01:44.726 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:44.922 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628472 2024-04-26T21:01:44.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:44.936 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:44.936 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:44.936 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:44.936 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:44.937 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/big/rados-thrash/ceph/ceph.yaml").st_ino) 2024-04-26T21:01:44.937 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:45.134 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628473 2024-04-26T21:01:45.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:45.149 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:45.149 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:45.149 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:45.149 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:45.149 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/big/rados-thrash/%").st_ino) 2024-04-26T21:01:45.149 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:45.343 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628471 2024-04-26T21:01:45.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:45.358 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:45.358 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:45.359 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:45.359 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:45.359 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/big/rados-thrash/thrashers").st_ino) 2024-04-26T21:01:45.359 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:45.557 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628481 2024-04-26T21:01:45.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:45.572 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:45.572 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:45.572 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:45.572 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:45.572 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/big/rados-thrash/thrashers/default.yaml").st_ino) 2024-04-26T21:01:45.572 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:45.767 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628482 2024-04-26T21:01:45.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:45.782 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:45.782 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:45.782 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:45.782 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:45.782 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/mixed-clients").st_ino) 2024-04-26T21:01:45.782 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:45.983 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629052 2024-04-26T21:01:45.988 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:45.998 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:45.998 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:45.998 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:45.998 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:45.998 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/mixed-clients/basic").st_ino) 2024-04-26T21:01:45.998 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:46.194 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629053 2024-04-26T21:01:46.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:46.209 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:46.209 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:46.209 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:46.209 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:46.209 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/mixed-clients/basic/clusters").st_ino) 2024-04-26T21:01:46.210 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:46.414 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629054 2024-04-26T21:01:46.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:46.429 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:46.429 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:46.429 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:46.429 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:46.429 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/mixed-clients/basic/clusters/fixed-3.yaml").st_ino) 2024-04-26T21:01:46.429 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:46.627 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629055 2024-04-26T21:01:46.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:46.642 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:46.643 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:46.643 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:46.643 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:46.643 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/mixed-clients/basic/fs").st_ino) 2024-04-26T21:01:46.643 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:46.794 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:01:46.801 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:01:46.839 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629056 2024-04-26T21:01:46.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:46.853 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:46.853 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:46.853 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:46.854 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:46.854 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/mixed-clients/basic/fs/btrfs.yaml").st_ino) 2024-04-26T21:01:46.854 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:47.048 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629057 2024-04-26T21:01:47.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:47.062 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:47.062 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:47.063 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:47.063 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:47.063 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/mixed-clients/basic/tasks").st_ino) 2024-04-26T21:01:47.063 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:47.211 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629058 2024-04-26T21:01:47.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:47.226 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:47.226 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:47.226 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:47.226 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:47.226 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/mixed-clients/basic/tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml").st_ino) 2024-04-26T21:01:47.226 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:47.423 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629060 2024-04-26T21:01:47.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:47.439 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:47.439 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:47.439 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:47.439 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:47.439 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/mixed-clients/basic/tasks/kernel_cfuse_workunits_dbench_iozone.yaml").st_ino) 2024-04-26T21:01:47.439 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:47.642 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629059 2024-04-26T21:01:47.646 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:47.656 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:47.656 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:47.656 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:47.656 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:47.656 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs").st_ino) 2024-04-26T21:01:47.656 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:47.853 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628551 2024-04-26T21:01:47.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:47.868 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:47.868 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:47.868 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:47.868 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:47.868 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless").st_ino) 2024-04-26T21:01:47.868 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:48.072 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628764 2024-04-26T21:01:48.077 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:48.090 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:48.090 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:48.090 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:48.090 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:48.090 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/traceless").st_ino) 2024-04-26T21:01:48.090 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:48.274 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628784 2024-04-26T21:01:48.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:48.288 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:48.288 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:48.289 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:48.289 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:48.289 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/traceless/50pc.yaml").st_ino) 2024-04-26T21:01:48.289 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:48.486 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628785 2024-04-26T21:01:48.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:48.501 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:48.501 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:48.501 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:48.501 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:48.501 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/overrides").st_ino) 2024-04-26T21:01:48.501 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:48.698 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628774 2024-04-26T21:01:48.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:48.712 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:48.712 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:48.712 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:48.712 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:48.712 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/overrides/whitelist_wrongly_marked_down.yaml").st_ino) 2024-04-26T21:01:48.712 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:48.909 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628778 2024-04-26T21:01:48.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:48.923 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:48.923 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:48.923 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:48.923 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:48.923 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/overrides/debug.yaml").st_ino) 2024-04-26T21:01:48.923 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:49.122 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628776 2024-04-26T21:01:49.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:49.137 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:49.137 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:49.137 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:49.137 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:49.137 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/overrides/+").st_ino) 2024-04-26T21:01:49.137 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:49.332 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628775 2024-04-26T21:01:49.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:49.347 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:49.348 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:49.348 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:49.348 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:49.348 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/overrides/frag_enable.yaml").st_ino) 2024-04-26T21:01:49.348 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:49.542 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628777 2024-04-26T21:01:49.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:49.557 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:49.557 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:49.557 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:49.557 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:49.557 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/clusters").st_ino) 2024-04-26T21:01:49.557 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:49.754 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628767 2024-04-26T21:01:49.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:49.770 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:49.770 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:49.770 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:49.770 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:49.770 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/clusters/fixed-2-ucephfs.yaml").st_ino) 2024-04-26T21:01:49.770 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:49.967 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628768 2024-04-26T21:01:49.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:49.982 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:49.982 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:49.983 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:49.983 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:49.983 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/objectstore").st_ino) 2024-04-26T21:01:49.983 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:50.178 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628773 2024-04-26T21:01:50.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:50.194 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:50.194 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:50.194 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:50.195 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:50.195 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/fs").st_ino) 2024-04-26T21:01:50.195 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:50.392 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628769 2024-04-26T21:01:50.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:50.412 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:50.412 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:50.412 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:50.412 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:50.412 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/fs/xfs.yaml").st_ino) 2024-04-26T21:01:50.412 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:50.598 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628770 2024-04-26T21:01:50.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:50.613 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:50.613 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:50.614 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:50.614 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:50.614 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/mount").st_ino) 2024-04-26T21:01:50.614 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:50.811 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628771 2024-04-26T21:01:50.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:50.825 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:50.825 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:50.825 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:50.825 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:50.825 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/mount/fuse.yaml").st_ino) 2024-04-26T21:01:50.825 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:51.021 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628772 2024-04-26T21:01:51.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:51.035 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:51.035 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:51.036 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:51.036 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:51.036 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/begin.yaml").st_ino) 2024-04-26T21:01:51.036 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:51.232 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628766 2024-04-26T21:01:51.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:51.248 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:51.248 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:51.248 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:51.248 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:51.248 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/%").st_ino) 2024-04-26T21:01:51.248 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:51.444 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628765 2024-04-26T21:01:51.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:51.458 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:51.458 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:51.458 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:51.458 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:51.458 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/tasks").st_ino) 2024-04-26T21:01:51.459 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:51.651 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628779 2024-04-26T21:01:51.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:51.666 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:51.666 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:51.667 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:51.667 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:51.667 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/tasks/cfuse_workunit_suites_ffsb.yaml").st_ino) 2024-04-26T21:01:51.667 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:51.868 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628782 2024-04-26T21:01:51.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:51.883 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:51.883 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:51.883 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:51.883 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:51.883 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/tasks/cfuse_workunit_suites_blogbench.yaml").st_ino) 2024-04-26T21:01:51.883 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:52.077 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628780 2024-04-26T21:01:52.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:52.093 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:52.093 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:52.093 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:52.093 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:52.093 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/tasks/cfuse_workunit_suites_fsstress.yaml").st_ino) 2024-04-26T21:01:52.093 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:52.292 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628783 2024-04-26T21:01:52.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:52.308 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:52.308 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:52.308 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:52.308 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:52.309 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/traceless/tasks/cfuse_workunit_suites_dbench.yaml").st_ino) 2024-04-26T21:01:52.309 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:52.507 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628781 2024-04-26T21:01:52.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:52.521 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:52.521 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:52.521 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:52.522 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:52.522 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multifs").st_ino) 2024-04-26T21:01:52.522 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:52.719 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628634 2024-04-26T21:01:52.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:52.735 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:52.735 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:52.735 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:52.735 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:52.736 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multifs/overrides").st_ino) 2024-04-26T21:01:52.736 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:52.930 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628644 2024-04-26T21:01:52.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:52.946 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:52.946 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:52.947 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:52.947 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:52.947 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multifs/overrides/whitelist_wrongly_marked_down.yaml").st_ino) 2024-04-26T21:01:52.947 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:53.144 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628649 2024-04-26T21:01:53.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:53.160 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:53.160 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:53.160 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:53.160 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:53.160 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multifs/overrides/mon-debug.yaml").st_ino) 2024-04-26T21:01:53.160 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:53.359 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628648 2024-04-26T21:01:53.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:53.373 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:53.373 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:53.373 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:53.374 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:53.374 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multifs/overrides/debug.yaml").st_ino) 2024-04-26T21:01:53.374 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:53.569 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628646 2024-04-26T21:01:53.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:53.584 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:53.584 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:53.584 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:53.584 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:53.584 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multifs/overrides/+").st_ino) 2024-04-26T21:01:53.584 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:53.777 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628645 2024-04-26T21:01:53.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:53.792 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:53.793 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:53.793 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:53.793 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:53.793 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multifs/overrides/frag_enable.yaml").st_ino) 2024-04-26T21:01:53.793 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:53.992 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628647 2024-04-26T21:01:53.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:54.008 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:54.008 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:54.008 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:54.008 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:54.008 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multifs/clusters").st_ino) 2024-04-26T21:01:54.008 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:54.204 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628637 2024-04-26T21:01:54.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:54.219 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:54.219 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:54.219 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:54.219 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:54.219 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multifs/clusters/2-remote-clients.yaml").st_ino) 2024-04-26T21:01:54.219 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:54.413 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628638 2024-04-26T21:01:54.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:54.428 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:54.428 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:54.428 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:54.428 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:54.428 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multifs/objectstore").st_ino) 2024-04-26T21:01:54.429 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:54.624 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628643 2024-04-26T21:01:54.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:54.639 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:54.639 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:54.639 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:54.639 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:54.639 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multifs/fs").st_ino) 2024-04-26T21:01:54.639 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:54.836 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628639 2024-04-26T21:01:54.840 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:54.850 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:54.851 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:54.851 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:54.851 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:54.851 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multifs/fs/xfs.yaml").st_ino) 2024-04-26T21:01:54.851 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:55.048 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628640 2024-04-26T21:01:55.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:55.063 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:55.063 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:55.063 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:55.063 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:55.064 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multifs/mount").st_ino) 2024-04-26T21:01:55.064 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:55.258 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628641 2024-04-26T21:01:55.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:55.272 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:55.272 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:55.272 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:55.272 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:55.272 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multifs/mount/fuse.yaml").st_ino) 2024-04-26T21:01:55.272 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:55.470 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628642 2024-04-26T21:01:55.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:55.485 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:55.485 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:55.485 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:55.485 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:55.485 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multifs/begin.yaml").st_ino) 2024-04-26T21:01:55.485 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:55.681 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628636 2024-04-26T21:01:55.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:55.696 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:55.696 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:55.696 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:55.696 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:55.696 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multifs/%").st_ino) 2024-04-26T21:01:55.696 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:55.894 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628635 2024-04-26T21:01:55.899 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:55.909 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:55.910 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:55.910 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:55.910 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:55.910 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multifs/tasks").st_ino) 2024-04-26T21:01:55.910 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:56.112 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628650 2024-04-26T21:01:56.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:56.126 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:56.127 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:56.127 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:56.127 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:56.127 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multifs/tasks/failover.yaml").st_ino) 2024-04-26T21:01:56.127 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:56.322 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628651 2024-04-26T21:01:56.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:56.337 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:56.337 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:56.337 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:56.337 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:56.337 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/snaps").st_ino) 2024-04-26T21:01:56.337 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:56.520 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628705 2024-04-26T21:01:56.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:56.535 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:56.535 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:56.535 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:56.535 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:56.535 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/snaps/overrides").st_ino) 2024-04-26T21:01:56.535 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:56.733 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628715 2024-04-26T21:01:56.738 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:56.748 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:56.748 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:56.748 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:56.749 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:56.749 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/snaps/overrides/whitelist_wrongly_marked_down.yaml").st_ino) 2024-04-26T21:01:56.749 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:56.943 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628719 2024-04-26T21:01:56.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:56.959 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:56.959 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:56.959 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:56.959 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:56.960 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/snaps/overrides/debug.yaml").st_ino) 2024-04-26T21:01:56.960 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:57.159 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628717 2024-04-26T21:01:57.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:57.174 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:57.174 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:57.174 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:57.174 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:57.174 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/snaps/overrides/+").st_ino) 2024-04-26T21:01:57.174 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:57.372 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628716 2024-04-26T21:01:57.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:57.387 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:57.387 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:57.387 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:57.387 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:57.387 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/snaps/overrides/frag_enable.yaml").st_ino) 2024-04-26T21:01:57.387 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:57.583 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628718 2024-04-26T21:01:57.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:57.598 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:57.598 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:57.598 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:57.598 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:57.598 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/snaps/clusters").st_ino) 2024-04-26T21:01:57.598 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:57.795 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628708 2024-04-26T21:01:57.800 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:57.810 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:57.810 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:57.810 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:57.810 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:57.810 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/snaps/clusters/fixed-2-ucephfs.yaml").st_ino) 2024-04-26T21:01:57.811 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:58.007 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628709 2024-04-26T21:01:58.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:58.022 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:58.022 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:58.022 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:58.022 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:58.022 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/snaps/objectstore").st_ino) 2024-04-26T21:01:58.022 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:58.222 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628714 2024-04-26T21:01:58.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:58.237 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:58.237 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:58.237 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:58.237 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:58.237 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/snaps/fs").st_ino) 2024-04-26T21:01:58.237 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:58.434 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628710 2024-04-26T21:01:58.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:58.449 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:58.450 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:58.450 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:58.450 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:58.450 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/snaps/fs/xfs.yaml").st_ino) 2024-04-26T21:01:58.450 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:58.647 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628711 2024-04-26T21:01:58.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:58.662 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:58.662 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:58.662 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:58.662 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:58.662 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/snaps/mount").st_ino) 2024-04-26T21:01:58.662 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:58.847 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628712 2024-04-26T21:01:58.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:58.860 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:58.861 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:58.861 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:58.861 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:58.861 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/snaps/mount/fuse.yaml").st_ino) 2024-04-26T21:01:58.861 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:59.058 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628713 2024-04-26T21:01:59.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:59.074 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:59.074 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:59.074 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:59.074 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:59.074 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/snaps/begin.yaml").st_ino) 2024-04-26T21:01:59.075 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:59.269 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628707 2024-04-26T21:01:59.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:59.284 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:59.284 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:59.284 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:59.284 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:59.284 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/snaps/%").st_ino) 2024-04-26T21:01:59.284 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:59.482 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628706 2024-04-26T21:01:59.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:59.497 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:59.497 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:59.497 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:59.497 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:59.497 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/snaps/tasks").st_ino) 2024-04-26T21:01:59.497 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:59.693 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628720 2024-04-26T21:01:59.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:59.708 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:59.708 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:59.708 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:59.708 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:59.708 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/snaps/tasks/snaptests.yaml").st_ino) 2024-04-26T21:01:59.708 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:01:59.905 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628721 2024-04-26T21:01:59.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:01:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:01:59.920 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:01:59.920 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:01:59.920 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:01:59.920 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:01:59.920 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic").st_ino) 2024-04-26T21:01:59.921 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:00.113 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628571 2024-04-26T21:02:00.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:00.129 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:00.129 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:00.129 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:00.130 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:00.130 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/overrides").st_ino) 2024-04-26T21:02:00.130 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:00.325 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628584 2024-04-26T21:02:00.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:00.340 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:00.340 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:00.340 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:00.340 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:00.340 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/overrides/whitelist_wrongly_marked_down.yaml").st_ino) 2024-04-26T21:02:00.340 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:00.540 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628588 2024-04-26T21:02:00.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:00.555 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:00.555 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:00.555 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:00.555 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:00.555 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/overrides/debug.yaml").st_ino) 2024-04-26T21:02:00.556 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:00.751 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628586 2024-04-26T21:02:00.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:00.766 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:00.766 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:00.766 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:00.766 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:00.766 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/overrides/+").st_ino) 2024-04-26T21:02:00.766 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:00.964 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628585 2024-04-26T21:02:00.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:00.979 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:00.979 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:00.979 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:00.979 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:00.979 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/overrides/frag_enable.yaml").st_ino) 2024-04-26T21:02:00.979 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:01.178 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628587 2024-04-26T21:02:01.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:01.193 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:01.193 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:01.193 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:01.193 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:01.193 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/clusters").st_ino) 2024-04-26T21:02:01.193 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:01.389 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628574 2024-04-26T21:02:01.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:01.405 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:01.405 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:01.405 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:01.405 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:01.405 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/clusters/fixed-2-ucephfs.yaml").st_ino) 2024-04-26T21:02:01.406 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:01.610 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628575 2024-04-26T21:02:01.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:01.625 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:01.626 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:01.626 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:01.626 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:01.626 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/objectstore").st_ino) 2024-04-26T21:02:01.626 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:01.821 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628583 2024-04-26T21:02:01.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:01.836 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:01.837 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:01.837 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:01.837 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:01.837 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/fs").st_ino) 2024-04-26T21:02:01.837 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:02.033 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628576 2024-04-26T21:02:02.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:02.049 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:02.049 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:02.049 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:02.049 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:02.049 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/fs/xfs.yaml").st_ino) 2024-04-26T21:02:02.049 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:02.244 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628577 2024-04-26T21:02:02.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:02.260 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:02.260 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:02.260 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:02.260 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:02.260 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/mount").st_ino) 2024-04-26T21:02:02.260 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:02.455 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628581 2024-04-26T21:02:02.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:02.471 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:02.471 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:02.471 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:02.471 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:02.471 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/mount/fuse.yaml").st_ino) 2024-04-26T21:02:02.471 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:02.668 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628582 2024-04-26T21:02:02.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:02.683 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:02.683 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:02.683 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:02.683 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:02.683 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/inline").st_ino) 2024-04-26T21:02:02.683 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:02.867 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628578 2024-04-26T21:02:02.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:02.883 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:02.883 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:02.883 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:02.883 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:02.883 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/inline/yes.yaml").st_ino) 2024-04-26T21:02:02.883 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:03.084 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628580 2024-04-26T21:02:03.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:03.099 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:03.099 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:03.099 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:03.099 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:03.100 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/inline/no.yaml").st_ino) 2024-04-26T21:02:03.100 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:03.300 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628579 2024-04-26T21:02:03.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:03.316 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:03.316 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:03.316 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:03.316 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:03.316 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/begin.yaml").st_ino) 2024-04-26T21:02:03.316 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:03.513 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628573 2024-04-26T21:02:03.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:03.528 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:03.528 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:03.528 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:03.529 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:03.529 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/%").st_ino) 2024-04-26T21:02:03.529 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:03.723 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628572 2024-04-26T21:02:03.728 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:03.738 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:03.739 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:03.739 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:03.739 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:03.739 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks").st_ino) 2024-04-26T21:02:03.739 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:03.936 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628589 2024-04-26T21:02:03.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:03.951 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:03.951 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:03.951 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:03.951 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:03.951 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_suites_fsync.yaml").st_ino) 2024-04-26T21:02:03.951 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:04.135 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628602 2024-04-26T21:02:04.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:04.149 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:04.149 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:04.150 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:04.150 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:04.150 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/libcephfs_interface_tests.yaml").st_ino) 2024-04-26T21:02:04.150 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:04.356 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628608 2024-04-26T21:02:04.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:04.371 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:04.371 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:04.371 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:04.371 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:04.372 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_suites_pjd.yaml").st_ino) 2024-04-26T21:02:04.373 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:04.554 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628605 2024-04-26T21:02:04.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:04.568 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:04.568 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:04.568 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:04.568 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:04.568 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_norstats.yaml").st_ino) 2024-04-26T21:02:04.568 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:04.764 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628595 2024-04-26T21:02:04.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:04.779 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:04.779 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:04.779 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:04.779 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:04.779 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_trivial_sync.yaml").st_ino) 2024-04-26T21:02:04.779 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:04.974 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628607 2024-04-26T21:02:04.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:04.989 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:04.989 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:04.989 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:04.989 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:04.989 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_suites_ffsb.yaml").st_ino) 2024-04-26T21:02:04.989 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:05.184 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628599 2024-04-26T21:02:05.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:05.199 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:05.199 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:05.199 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:05.200 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:05.200 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_suites_fsx.yaml").st_ino) 2024-04-26T21:02:05.200 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:05.398 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628601 2024-04-26T21:02:05.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:05.413 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:05.413 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:05.413 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:05.413 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:05.413 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/libcephfs_java.yaml").st_ino) 2024-04-26T21:02:05.414 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:05.608 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628609 2024-04-26T21:02:05.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:05.623 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:05.623 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:05.623 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:05.623 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:05.623 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_misc.yaml").st_ino) 2024-04-26T21:02:05.623 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:05.819 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628593 2024-04-26T21:02:05.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:05.833 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:05.833 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:05.833 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:05.834 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:05.834 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_misc_test_o_trunc.yaml").st_ino) 2024-04-26T21:02:05.834 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:06.028 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628594 2024-04-26T21:02:06.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:06.043 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:06.043 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:06.043 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:06.043 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:06.043 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/mds_creation_retry.yaml").st_ino) 2024-04-26T21:02:06.043 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:06.238 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628611 2024-04-26T21:02:06.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:06.253 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:06.253 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:06.253 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:06.253 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:06.253 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/asok_dump_tree.yaml").st_ino) 2024-04-26T21:02:06.253 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:06.438 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628590 2024-04-26T21:02:06.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:06.452 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:06.452 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:06.452 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:06.452 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:06.452 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_quota.yaml").st_ino) 2024-04-26T21:02:06.452 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:06.648 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628596 2024-04-26T21:02:06.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:06.663 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:06.664 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:06.664 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:06.664 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:06.664 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_suites_truncate_delay.yaml").st_ino) 2024-04-26T21:02:06.664 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:06.864 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628606 2024-04-26T21:02:06.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:06.879 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:06.879 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:06.879 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:06.879 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:06.879 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_kernel_untar_build.yaml").st_ino) 2024-04-26T21:02:06.880 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:07.060 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628592 2024-04-26T21:02:07.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:07.074 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:07.075 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:07.075 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:07.075 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:07.075 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_suites_blogbench.yaml").st_ino) 2024-04-26T21:02:07.075 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:07.270 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628597 2024-04-26T21:02:07.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:07.284 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:07.284 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:07.284 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:07.284 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:07.284 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/libcephfs_python.yaml").st_ino) 2024-04-26T21:02:07.284 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:07.469 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628610 2024-04-26T21:02:07.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:07.484 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:07.484 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:07.484 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:07.484 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:07.485 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_suites_fsstress.yaml").st_ino) 2024-04-26T21:02:07.485 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:07.681 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628600 2024-04-26T21:02:07.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:07.695 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:07.695 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:07.695 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:07.696 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:07.696 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/cephfs_scrub_tests.yaml").st_ino) 2024-04-26T21:02:07.696 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:07.893 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628591 2024-04-26T21:02:07.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:07.907 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:07.907 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:07.907 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:07.907 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:07.907 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_suites_iogen.yaml").st_ino) 2024-04-26T21:02:07.907 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:08.108 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628603 2024-04-26T21:02:08.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:08.123 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:08.123 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:08.123 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:08.123 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:08.123 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_suites_dbench.yaml").st_ino) 2024-04-26T21:02:08.123 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:08.321 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628598 2024-04-26T21:02:08.326 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:08.336 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:08.336 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:08.336 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:08.336 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:08.336 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/basic/tasks/cfuse_workunit_suites_iozone.yaml").st_ino) 2024-04-26T21:02:08.336 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:08.533 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628604 2024-04-26T21:02:08.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:08.548 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:08.548 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:08.548 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:08.548 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:08.548 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery").st_ino) 2024-04-26T21:02:08.549 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:08.744 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628670 2024-04-26T21:02:08.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:08.759 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:08.759 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:08.759 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:08.759 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:08.759 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/overrides").st_ino) 2024-04-26T21:02:08.759 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:08.960 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628680 2024-04-26T21:02:08.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:08.976 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:08.976 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:08.976 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:08.976 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:08.976 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/overrides/whitelist_wrongly_marked_down.yaml").st_ino) 2024-04-26T21:02:08.976 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:09.161 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628685 2024-04-26T21:02:09.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:09.176 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:09.176 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:09.176 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:09.176 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:09.176 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/overrides/fuse-unmounted.yaml").st_ino) 2024-04-26T21:02:09.176 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:09.373 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628684 2024-04-26T21:02:09.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:09.388 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:09.388 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:09.388 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:09.388 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:09.388 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/overrides/debug.yaml").st_ino) 2024-04-26T21:02:09.388 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:09.586 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628682 2024-04-26T21:02:09.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:09.603 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:09.603 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:09.603 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:09.603 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:09.603 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/overrides/+").st_ino) 2024-04-26T21:02:09.603 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:09.799 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628681 2024-04-26T21:02:09.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:09.814 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:09.814 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:09.814 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:09.814 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:09.814 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/overrides/frag_enable.yaml").st_ino) 2024-04-26T21:02:09.814 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:10.012 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628683 2024-04-26T21:02:10.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:10.028 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:10.028 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:10.028 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:10.028 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:10.028 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/clusters").st_ino) 2024-04-26T21:02:10.028 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:10.212 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628673 2024-04-26T21:02:10.217 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:10.227 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:10.227 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:10.227 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:10.228 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:10.228 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/clusters/4-remote-clients.yaml").st_ino) 2024-04-26T21:02:10.228 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:10.428 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628674 2024-04-26T21:02:10.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:10.445 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:10.445 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:10.445 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:10.445 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:10.445 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/objectstore").st_ino) 2024-04-26T21:02:10.445 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:10.640 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628679 2024-04-26T21:02:10.645 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:10.655 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:10.655 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:10.655 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:10.655 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:10.655 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/fs").st_ino) 2024-04-26T21:02:10.655 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:10.855 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628675 2024-04-26T21:02:10.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:10.870 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:10.870 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:10.870 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:10.870 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:10.870 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/fs/xfs.yaml").st_ino) 2024-04-26T21:02:10.871 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:11.072 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628676 2024-04-26T21:02:11.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:11.090 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:11.090 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:11.090 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:11.090 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:11.090 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/mount").st_ino) 2024-04-26T21:02:11.090 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:11.286 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628677 2024-04-26T21:02:11.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:11.301 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:11.301 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:11.301 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:11.301 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:11.301 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/mount/fuse.yaml").st_ino) 2024-04-26T21:02:11.301 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:11.484 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628678 2024-04-26T21:02:11.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:11.498 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:11.498 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:11.498 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:11.498 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:11.498 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/begin.yaml").st_ino) 2024-04-26T21:02:11.499 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:11.692 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628672 2024-04-26T21:02:11.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:11.707 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:11.707 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:11.707 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:11.707 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:11.708 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/%").st_ino) 2024-04-26T21:02:11.708 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:11.897 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628671 2024-04-26T21:02:11.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:11.912 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:11.912 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:11.912 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:11.913 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:11.913 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/tasks").st_ino) 2024-04-26T21:02:11.913 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:12.115 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628686 2024-04-26T21:02:12.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:12.129 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:12.130 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:12.130 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:12.130 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:12.130 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/tasks/backtrace.yaml").st_ino) 2024-04-26T21:02:12.130 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:12.325 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628688 2024-04-26T21:02:12.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:12.341 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:12.341 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:12.341 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:12.341 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:12.341 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/tasks/forward-scrub.yaml").st_ino) 2024-04-26T21:02:12.341 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:12.538 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628696 2024-04-26T21:02:12.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:12.553 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:12.553 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:12.553 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:12.553 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:12.553 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/tasks/client-readahad.yaml").st_ino) 2024-04-26T21:02:12.553 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:12.749 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628691 2024-04-26T21:02:12.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:12.763 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:12.764 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:12.764 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:12.764 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:12.764 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/tasks/volume-client.yaml").st_ino) 2024-04-26T21:02:12.764 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:12.957 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628704 2024-04-26T21:02:12.962 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:12.972 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:12.972 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:12.972 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:12.972 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:12.972 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/tasks/config-commands.yaml").st_ino) 2024-04-26T21:02:12.972 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:13.171 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628693 2024-04-26T21:02:13.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:13.186 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:13.187 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:13.187 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:13.187 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:13.187 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/tasks/mds-full.yaml").st_ino) 2024-04-26T21:02:13.187 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:13.382 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628700 2024-04-26T21:02:13.387 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:13.398 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:13.398 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:13.398 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:13.398 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:13.398 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/tasks/sessionmap.yaml").st_ino) 2024-04-26T21:02:13.398 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:13.597 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628702 2024-04-26T21:02:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:13.611 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:13.611 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:13.612 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:13.612 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:13.612 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/tasks/data-scan.yaml").st_ino) 2024-04-26T21:02:13.612 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:13.809 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628695 2024-04-26T21:02:13.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:13.824 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:13.824 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:13.824 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:13.824 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:13.824 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/tasks/cap-flush.yaml").st_ino) 2024-04-26T21:02:13.824 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:14.021 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628689 2024-04-26T21:02:14.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:14.036 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:14.036 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:14.036 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:14.036 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:14.037 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/tasks/mds-flush.yaml").st_ino) 2024-04-26T21:02:14.037 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:14.233 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628699 2024-04-26T21:02:14.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:14.248 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:14.248 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:14.248 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:14.248 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:14.248 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/tasks/auto-repair.yaml").st_ino) 2024-04-26T21:02:14.248 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:14.445 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628687 2024-04-26T21:02:14.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:14.459 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:14.460 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:14.460 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:14.460 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:14.460 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/tasks/damage.yaml").st_ino) 2024-04-26T21:02:14.460 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:14.655 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628694 2024-04-26T21:02:14.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:14.670 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:14.670 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:14.670 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:14.670 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:14.670 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/tasks/client-recovery.yaml").st_ino) 2024-04-26T21:02:14.670 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:14.863 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628692 2024-04-26T21:02:14.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:14.879 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:14.879 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:14.879 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:14.879 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:14.879 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/tasks/journal-repair.yaml").st_ino) 2024-04-26T21:02:14.879 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:15.079 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628698 2024-04-26T21:02:15.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:15.094 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:15.094 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:15.094 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:15.094 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:15.094 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/tasks/client-limits.yaml").st_ino) 2024-04-26T21:02:15.094 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:15.292 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628690 2024-04-26T21:02:15.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:15.306 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:15.306 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:15.307 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:15.307 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:15.307 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/tasks/strays.yaml").st_ino) 2024-04-26T21:02:15.307 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:15.503 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628703 2024-04-26T21:02:15.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:15.518 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:15.518 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:15.518 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:15.518 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:15.518 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/tasks/fragment.yaml").st_ino) 2024-04-26T21:02:15.518 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:15.713 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628697 2024-04-26T21:02:15.718 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:15.728 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:15.728 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:15.728 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:15.728 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:15.729 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/recovery/tasks/pool-perm.yaml").st_ino) 2024-04-26T21:02:15.729 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:15.923 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628701 2024-04-26T21:02:15.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:15.938 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:15.938 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:15.938 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:15.938 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:15.938 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/permission").st_ino) 2024-04-26T21:02:15.938 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:16.134 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628652 2024-04-26T21:02:16.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:16.149 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:16.149 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:16.149 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:16.149 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:16.149 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/permission/overrides").st_ino) 2024-04-26T21:02:16.149 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:16.342 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628662 2024-04-26T21:02:16.346 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:16.357 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:16.357 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:16.357 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:16.357 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:16.357 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/permission/overrides/whitelist_wrongly_marked_down.yaml").st_ino) 2024-04-26T21:02:16.357 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:16.542 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628666 2024-04-26T21:02:16.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:16.556 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:16.556 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:16.556 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:16.556 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:16.556 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/permission/overrides/debug.yaml").st_ino) 2024-04-26T21:02:16.556 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:16.752 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628664 2024-04-26T21:02:16.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:16.767 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:16.767 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:16.768 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:16.768 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:16.768 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/permission/overrides/+").st_ino) 2024-04-26T21:02:16.768 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:16.944 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:02:16.949 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:02:16.973 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628663 2024-04-26T21:02:16.978 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:16.989 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:16.989 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:16.989 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:16.989 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:16.989 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/permission/overrides/frag_enable.yaml").st_ino) 2024-04-26T21:02:16.989 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:17.165 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628665 2024-04-26T21:02:17.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:17.179 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:17.179 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:17.179 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:17.179 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:17.179 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/permission/clusters").st_ino) 2024-04-26T21:02:17.179 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:17.333 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628655 2024-04-26T21:02:17.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:17.348 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:17.348 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:17.348 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:17.348 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:17.348 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/permission/clusters/fixed-2-ucephfs.yaml").st_ino) 2024-04-26T21:02:17.348 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:17.544 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628656 2024-04-26T21:02:17.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:17.559 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:17.559 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:17.559 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:17.559 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:17.559 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/permission/objectstore").st_ino) 2024-04-26T21:02:17.559 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:17.759 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628661 2024-04-26T21:02:17.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:17.774 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:17.774 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:17.774 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:17.774 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:17.775 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/permission/fs").st_ino) 2024-04-26T21:02:17.775 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:17.966 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628657 2024-04-26T21:02:17.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:17.982 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:17.982 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:17.982 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:17.982 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:17.982 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/permission/fs/xfs.yaml").st_ino) 2024-04-26T21:02:17.982 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:18.163 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628658 2024-04-26T21:02:18.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:18.177 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:18.177 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:18.177 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:18.177 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:18.177 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/permission/mount").st_ino) 2024-04-26T21:02:18.177 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:18.373 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628659 2024-04-26T21:02:18.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:18.388 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:18.388 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:18.388 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:18.388 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:18.388 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/permission/mount/fuse.yaml").st_ino) 2024-04-26T21:02:18.388 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:18.584 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628660 2024-04-26T21:02:18.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:18.599 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:18.599 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:18.599 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:18.599 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:18.599 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/permission/begin.yaml").st_ino) 2024-04-26T21:02:18.599 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:18.797 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628654 2024-04-26T21:02:18.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:18.812 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:18.812 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:18.812 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:18.812 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:18.812 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/permission/%").st_ino) 2024-04-26T21:02:18.812 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:19.007 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628653 2024-04-26T21:02:19.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:19.022 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:19.023 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:19.023 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:19.023 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:19.023 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/permission/tasks").st_ino) 2024-04-26T21:02:19.023 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:19.218 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628667 2024-04-26T21:02:19.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:19.232 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:19.233 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:19.233 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:19.233 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:19.233 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/permission/tasks/cfuse_workunit_suites_pjd.yaml").st_ino) 2024-04-26T21:02:19.233 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:19.434 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628669 2024-04-26T21:02:19.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:19.449 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:19.449 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:19.449 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:19.449 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:19.449 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/permission/tasks/cfuse_workunit_misc.yaml").st_ino) 2024-04-26T21:02:19.449 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:19.649 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628668 2024-04-26T21:02:19.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:19.664 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:19.664 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:19.664 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:19.664 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:19.664 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/standbyreplay").st_ino) 2024-04-26T21:02:19.664 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:19.840 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628722 2024-04-26T21:02:19.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:19.854 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:19.854 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:19.854 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:19.854 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:19.854 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/standbyreplay/overrides").st_ino) 2024-04-26T21:02:19.854 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:20.048 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628732 2024-04-26T21:02:20.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:20.063 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:20.064 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:20.064 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:20.064 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:20.064 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/standbyreplay/overrides/whitelist_wrongly_marked_down.yaml").st_ino) 2024-04-26T21:02:20.064 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:20.244 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628736 2024-04-26T21:02:20.249 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:20.258 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:20.258 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:20.259 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:20.259 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:20.259 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/standbyreplay/overrides/debug.yaml").st_ino) 2024-04-26T21:02:20.259 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:20.455 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628734 2024-04-26T21:02:20.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:20.470 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:20.470 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:20.470 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:20.470 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:20.471 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/standbyreplay/overrides/+").st_ino) 2024-04-26T21:02:20.471 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:20.668 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628733 2024-04-26T21:02:20.672 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:20.682 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:20.683 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:20.683 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:20.683 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:20.683 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/standbyreplay/overrides/frag_enable.yaml").st_ino) 2024-04-26T21:02:20.683 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:20.880 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628735 2024-04-26T21:02:20.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:20.895 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:20.895 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:20.895 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:20.895 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:20.895 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/standbyreplay/clusters").st_ino) 2024-04-26T21:02:20.895 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:21.095 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628725 2024-04-26T21:02:21.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:21.109 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:21.109 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:21.109 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:21.109 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:21.110 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/standbyreplay/clusters/standby-replay.yaml").st_ino) 2024-04-26T21:02:21.110 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:21.306 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628726 2024-04-26T21:02:21.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:21.322 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:21.322 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:21.322 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:21.322 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:21.322 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/standbyreplay/objectstore").st_ino) 2024-04-26T21:02:21.322 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:21.518 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628731 2024-04-26T21:02:21.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:21.533 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:21.533 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:21.533 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:21.534 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:21.534 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/standbyreplay/fs").st_ino) 2024-04-26T21:02:21.534 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:21.736 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628727 2024-04-26T21:02:21.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:21.751 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:21.751 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:21.751 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:21.751 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:21.751 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/standbyreplay/fs/xfs.yaml").st_ino) 2024-04-26T21:02:21.751 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:21.949 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628728 2024-04-26T21:02:21.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:21.964 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:21.964 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:21.964 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:21.964 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:21.964 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/standbyreplay/mount").st_ino) 2024-04-26T21:02:21.965 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:22.167 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628729 2024-04-26T21:02:22.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:22.182 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:22.182 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:22.182 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:22.182 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:22.182 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/standbyreplay/mount/fuse.yaml").st_ino) 2024-04-26T21:02:22.182 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:22.378 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628730 2024-04-26T21:02:22.383 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:22.393 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:22.393 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:22.393 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:22.394 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:22.394 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/standbyreplay/begin.yaml").st_ino) 2024-04-26T21:02:22.394 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:22.599 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628724 2024-04-26T21:02:22.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:22.613 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:22.613 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:22.613 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:22.613 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:22.613 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/standbyreplay/%").st_ino) 2024-04-26T21:02:22.614 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:22.810 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628723 2024-04-26T21:02:22.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:22.825 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:22.825 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:22.825 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:22.825 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:22.825 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/standbyreplay/tasks").st_ino) 2024-04-26T21:02:22.825 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:23.019 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628737 2024-04-26T21:02:23.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:23.034 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:23.034 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:23.034 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:23.034 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:23.035 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/standbyreplay/tasks/migration.yaml").st_ino) 2024-04-26T21:02:23.035 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:23.231 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628738 2024-04-26T21:02:23.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:23.245 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:23.245 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:23.245 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:23.245 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:23.245 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/32bits").st_ino) 2024-04-26T21:02:23.245 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:23.441 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628552 2024-04-26T21:02:23.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:23.455 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:23.455 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:23.455 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:23.455 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:23.455 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/32bits/overrides").st_ino) 2024-04-26T21:02:23.455 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:23.656 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628562 2024-04-26T21:02:23.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:23.671 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:23.671 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:23.671 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:23.671 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:23.671 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/32bits/overrides/whitelist_wrongly_marked_down.yaml").st_ino) 2024-04-26T21:02:23.671 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:23.867 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628567 2024-04-26T21:02:23.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:23.883 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:23.883 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:23.883 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:23.883 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:23.883 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/32bits/overrides/debug.yaml").st_ino) 2024-04-26T21:02:23.883 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:24.068 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628564 2024-04-26T21:02:24.074 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:24.084 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:24.084 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:24.084 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:24.084 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:24.084 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/32bits/overrides/+").st_ino) 2024-04-26T21:02:24.084 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:24.281 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628563 2024-04-26T21:02:24.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:24.295 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:24.296 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:24.296 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:24.296 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:24.296 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/32bits/overrides/faked-ino.yaml").st_ino) 2024-04-26T21:02:24.296 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:24.492 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628565 2024-04-26T21:02:24.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:24.507 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:24.507 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:24.507 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:24.507 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:24.507 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/32bits/overrides/frag_enable.yaml").st_ino) 2024-04-26T21:02:24.507 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:24.701 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628566 2024-04-26T21:02:24.706 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:24.717 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:24.717 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:24.717 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:24.717 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:24.717 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/32bits/clusters").st_ino) 2024-04-26T21:02:24.717 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:24.914 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628555 2024-04-26T21:02:24.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:24.929 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:24.929 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:24.929 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:24.929 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:24.929 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/32bits/clusters/fixed-2-ucephfs.yaml").st_ino) 2024-04-26T21:02:24.929 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:25.115 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628556 2024-04-26T21:02:25.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:25.130 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:25.130 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:25.131 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:25.131 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:25.131 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/32bits/objectstore").st_ino) 2024-04-26T21:02:25.131 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:25.326 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628561 2024-04-26T21:02:25.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:25.341 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:25.341 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:25.341 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:25.341 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:25.341 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/32bits/fs").st_ino) 2024-04-26T21:02:25.341 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:25.537 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628557 2024-04-26T21:02:25.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:25.552 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:25.552 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:25.552 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:25.552 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:25.552 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/32bits/fs/xfs.yaml").st_ino) 2024-04-26T21:02:25.552 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:25.750 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628558 2024-04-26T21:02:25.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:25.765 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:25.765 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:25.765 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:25.765 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:25.765 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/32bits/mount").st_ino) 2024-04-26T21:02:25.766 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:25.966 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628559 2024-04-26T21:02:25.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:25.981 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:25.981 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:25.981 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:25.981 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:25.981 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/32bits/mount/fuse.yaml").st_ino) 2024-04-26T21:02:25.981 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:26.177 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628560 2024-04-26T21:02:26.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:26.192 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:26.193 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:26.193 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:26.193 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:26.193 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/32bits/begin.yaml").st_ino) 2024-04-26T21:02:26.193 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:26.392 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628554 2024-04-26T21:02:26.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:26.407 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:26.407 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:26.407 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:26.408 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:26.408 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/32bits/%").st_ino) 2024-04-26T21:02:26.408 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:26.603 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628553 2024-04-26T21:02:26.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:26.617 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:26.617 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:26.617 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:26.617 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:26.617 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/32bits/tasks").st_ino) 2024-04-26T21:02:26.617 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:26.811 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628568 2024-04-26T21:02:26.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:26.826 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:26.826 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:26.826 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:26.826 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:26.826 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/32bits/tasks/cfuse_workunit_suites_pjd.yaml").st_ino) 2024-04-26T21:02:26.826 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:27.025 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628570 2024-04-26T21:02:27.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:27.039 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:27.039 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:27.039 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:27.040 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:27.040 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/32bits/tasks/cfuse_workunit_suites_fsstress.yaml").st_ino) 2024-04-26T21:02:27.040 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:27.225 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628569 2024-04-26T21:02:27.230 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:27.240 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:27.240 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:27.240 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:27.241 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:27.241 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash").st_ino) 2024-04-26T21:02:27.241 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:27.436 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628739 2024-04-26T21:02:27.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:27.452 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:27.452 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:27.452 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:27.452 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:27.453 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/ceph-thrash").st_ino) 2024-04-26T21:02:27.453 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:27.648 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628742 2024-04-26T21:02:27.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:27.663 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:27.663 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:27.663 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:27.663 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:27.663 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/ceph-thrash/default.yaml").st_ino) 2024-04-26T21:02:27.663 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:27.857 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628743 2024-04-26T21:02:27.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:27.873 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:27.873 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:27.873 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:27.873 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:27.873 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/overrides").st_ino) 2024-04-26T21:02:27.873 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:28.068 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628754 2024-04-26T21:02:28.073 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:28.085 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:28.085 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:28.085 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:28.085 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:28.085 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/overrides/whitelist_wrongly_marked_down.yaml").st_ino) 2024-04-26T21:02:28.085 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:28.279 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628758 2024-04-26T21:02:28.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:28.294 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:28.295 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:28.295 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:28.295 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:28.295 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/overrides/debug.yaml").st_ino) 2024-04-26T21:02:28.295 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:28.491 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628756 2024-04-26T21:02:28.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:28.506 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:28.506 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:28.506 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:28.506 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:28.506 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/overrides/+").st_ino) 2024-04-26T21:02:28.506 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:28.706 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628755 2024-04-26T21:02:28.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:28.721 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:28.721 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:28.721 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:28.721 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:28.721 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/overrides/frag_enable.yaml").st_ino) 2024-04-26T21:02:28.722 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:28.917 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628757 2024-04-26T21:02:28.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:28.934 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:28.934 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:28.934 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:28.934 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:28.934 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/clusters").st_ino) 2024-04-26T21:02:28.934 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:29.129 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628744 2024-04-26T21:02:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:29.145 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:29.145 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:29.145 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:29.145 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:29.145 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/clusters/mds-1active-1standby.yaml").st_ino) 2024-04-26T21:02:29.145 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:29.343 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628745 2024-04-26T21:02:29.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:29.358 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:29.358 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:29.358 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:29.358 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:29.358 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/msgr-failures").st_ino) 2024-04-26T21:02:29.359 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:29.554 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628750 2024-04-26T21:02:29.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:29.570 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:29.570 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:29.570 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:29.570 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:29.570 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/msgr-failures/osd-mds-delay.yaml").st_ino) 2024-04-26T21:02:29.570 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:29.754 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628752 2024-04-26T21:02:29.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:29.768 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:29.768 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:29.768 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:29.768 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:29.768 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/msgr-failures/none.yaml").st_ino) 2024-04-26T21:02:29.768 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:29.960 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628751 2024-04-26T21:02:29.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:29.975 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:29.975 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:29.975 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:29.976 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:29.976 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/objectstore").st_ino) 2024-04-26T21:02:29.976 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:30.173 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628753 2024-04-26T21:02:30.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:30.189 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:30.189 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:30.189 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:30.189 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:30.189 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/fs").st_ino) 2024-04-26T21:02:30.189 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:30.375 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628746 2024-04-26T21:02:30.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:30.389 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:30.389 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:30.389 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:30.389 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:30.389 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/fs/xfs.yaml").st_ino) 2024-04-26T21:02:30.390 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:30.584 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628747 2024-04-26T21:02:30.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:30.600 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:30.600 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:30.600 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:30.600 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:30.600 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/mount").st_ino) 2024-04-26T21:02:30.600 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:30.782 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628748 2024-04-26T21:02:30.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:30.797 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:30.797 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:30.797 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:30.797 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:30.797 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/mount/fuse.yaml").st_ino) 2024-04-26T21:02:30.797 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:30.999 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628749 2024-04-26T21:02:31.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:31.014 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:31.014 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:31.014 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:31.014 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:31.014 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/begin.yaml").st_ino) 2024-04-26T21:02:31.014 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:31.201 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628741 2024-04-26T21:02:31.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:31.215 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:31.215 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:31.215 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:31.215 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:31.216 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/%").st_ino) 2024-04-26T21:02:31.216 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:31.412 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628740 2024-04-26T21:02:31.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:31.427 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:31.428 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:31.428 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:31.428 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:31.428 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/tasks").st_ino) 2024-04-26T21:02:31.428 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:31.624 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628759 2024-04-26T21:02:31.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:31.638 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:31.638 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:31.638 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:31.638 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:31.638 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/tasks/cfuse_workunit_suites_pjd.yaml").st_ino) 2024-04-26T21:02:31.638 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:31.825 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628762 2024-04-26T21:02:31.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:31.839 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:31.839 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:31.839 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:31.839 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:31.839 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/tasks/cfuse_workunit_trivial_sync.yaml").st_ino) 2024-04-26T21:02:31.839 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:32.043 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628763 2024-04-26T21:02:32.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:32.059 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:32.059 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:32.059 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:32.059 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:32.059 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/tasks/cfuse_workunit_snaptests.yaml").st_ino) 2024-04-26T21:02:32.059 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:32.256 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628760 2024-04-26T21:02:32.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:32.271 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:32.271 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:32.271 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:32.271 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:32.271 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/thrash/tasks/cfuse_workunit_suites_fsstress.yaml").st_ino) 2024-04-26T21:02:32.271 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:32.467 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628761 2024-04-26T21:02:32.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:32.481 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:32.482 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:32.482 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:32.482 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:32.482 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify").st_ino) 2024-04-26T21:02:32.482 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:32.678 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628786 2024-04-26T21:02:32.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:32.693 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:32.693 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:32.693 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:32.693 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:32.693 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/validater").st_ino) 2024-04-26T21:02:32.693 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:32.888 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628806 2024-04-26T21:02:32.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:32.902 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:32.902 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:32.902 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:32.902 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:32.902 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml").st_ino) 2024-04-26T21:02:32.902 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:33.100 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628808 2024-04-26T21:02:33.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:33.116 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:33.116 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:33.116 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:33.116 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:33.116 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/validater/lockdep.yaml").st_ino) 2024-04-26T21:02:33.116 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:33.311 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628807 2024-04-26T21:02:33.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:33.325 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:33.325 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:33.325 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:33.325 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:33.325 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/overrides").st_ino) 2024-04-26T21:02:33.325 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:33.522 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628796 2024-04-26T21:02:33.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:33.538 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:33.538 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:33.538 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:33.539 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:33.539 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/overrides/whitelist_wrongly_marked_down.yaml").st_ino) 2024-04-26T21:02:33.539 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:33.736 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628801 2024-04-26T21:02:33.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:33.752 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:33.752 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:33.752 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:33.752 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:33.752 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/overrides/mon-debug.yaml").st_ino) 2024-04-26T21:02:33.753 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:33.949 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628800 2024-04-26T21:02:33.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:33.965 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:33.965 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:33.965 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:33.965 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:33.965 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/overrides/debug.yaml").st_ino) 2024-04-26T21:02:33.965 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:34.165 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628798 2024-04-26T21:02:34.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:34.180 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:34.180 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:34.180 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:34.181 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:34.181 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/overrides/+").st_ino) 2024-04-26T21:02:34.181 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:34.379 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628797 2024-04-26T21:02:34.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:34.394 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:34.394 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:34.394 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:34.394 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:34.395 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/overrides/frag_enable.yaml").st_ino) 2024-04-26T21:02:34.395 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:34.590 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628799 2024-04-26T21:02:34.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:34.605 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:34.605 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:34.605 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:34.605 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:34.605 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/clusters").st_ino) 2024-04-26T21:02:34.605 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:34.800 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628789 2024-04-26T21:02:34.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:34.815 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:34.815 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:34.815 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:34.816 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:34.816 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/clusters/fixed-2-ucephfs.yaml").st_ino) 2024-04-26T21:02:34.816 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:35.013 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628790 2024-04-26T21:02:35.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:35.028 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:35.028 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:35.028 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:35.028 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:35.028 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/objectstore").st_ino) 2024-04-26T21:02:35.028 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:35.224 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628795 2024-04-26T21:02:35.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:35.239 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:35.239 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:35.239 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:35.239 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:35.239 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/fs").st_ino) 2024-04-26T21:02:35.239 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:35.436 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628791 2024-04-26T21:02:35.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:35.450 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:35.450 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:35.450 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:35.450 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:35.450 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/fs/xfs.yaml").st_ino) 2024-04-26T21:02:35.451 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:35.654 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628792 2024-04-26T21:02:35.659 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:35.669 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:35.669 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:35.669 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:35.669 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:35.669 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/mount").st_ino) 2024-04-26T21:02:35.669 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:35.866 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628793 2024-04-26T21:02:35.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:35.880 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:35.880 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:35.880 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:35.880 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:35.881 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/mount/fuse.yaml").st_ino) 2024-04-26T21:02:35.881 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:36.078 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628794 2024-04-26T21:02:36.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:36.095 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:36.095 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:36.095 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:36.095 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:36.095 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/begin.yaml").st_ino) 2024-04-26T21:02:36.095 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:36.273 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628788 2024-04-26T21:02:36.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:36.288 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:36.288 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:36.288 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:36.288 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:36.288 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/%").st_ino) 2024-04-26T21:02:36.289 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:36.483 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628787 2024-04-26T21:02:36.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:36.498 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:36.498 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:36.498 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:36.498 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:36.498 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/tasks").st_ino) 2024-04-26T21:02:36.499 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:36.696 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628802 2024-04-26T21:02:36.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:36.711 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:36.711 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:36.711 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:36.712 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:36.712 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/tasks/libcephfs_interface_tests.yaml").st_ino) 2024-04-26T21:02:36.712 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:36.909 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628805 2024-04-26T21:02:36.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:36.923 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:36.923 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:36.923 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:36.923 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:36.923 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/tasks/cfuse_workunit_suites_fsstress.yaml").st_ino) 2024-04-26T21:02:36.923 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:37.115 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628804 2024-04-26T21:02:37.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:37.130 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:37.130 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:37.130 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:37.130 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:37.130 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/verify/tasks/cfuse_workunit_suites_dbench.yaml").st_ino) 2024-04-26T21:02:37.130 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:37.327 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628803 2024-04-26T21:02:37.332 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:37.342 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:37.342 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:37.342 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:37.342 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:37.342 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient").st_ino) 2024-04-26T21:02:37.381 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:37.539 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628612 2024-04-26T21:02:37.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:37.553 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:37.553 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:37.553 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:37.553 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:37.553 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/overrides").st_ino) 2024-04-26T21:02:37.554 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:37.749 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628624 2024-04-26T21:02:37.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:37.764 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:37.764 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:37.764 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:37.764 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:37.764 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/overrides/whitelist_wrongly_marked_down.yaml").st_ino) 2024-04-26T21:02:37.764 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:37.961 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628628 2024-04-26T21:02:37.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:37.975 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:37.975 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:37.975 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:37.975 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:37.975 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/overrides/debug.yaml").st_ino) 2024-04-26T21:02:37.975 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:38.173 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628626 2024-04-26T21:02:38.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:38.188 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:38.188 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:38.188 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:38.188 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:38.189 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/overrides/+").st_ino) 2024-04-26T21:02:38.189 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:38.389 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628625 2024-04-26T21:02:38.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:38.404 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:38.404 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:38.404 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:38.404 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:38.404 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/overrides/frag_enable.yaml").st_ino) 2024-04-26T21:02:38.404 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:38.598 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628627 2024-04-26T21:02:38.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:38.613 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:38.614 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:38.614 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:38.614 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:38.614 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/clusters").st_ino) 2024-04-26T21:02:38.614 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:38.820 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628615 2024-04-26T21:02:38.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:38.835 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:38.835 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:38.835 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:38.835 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:38.835 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/clusters/three_clients.yaml").st_ino) 2024-04-26T21:02:38.835 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:39.032 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628616 2024-04-26T21:02:39.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:39.046 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:39.046 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:39.046 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:39.046 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:39.046 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/clusters/two_clients.yaml").st_ino) 2024-04-26T21:02:39.046 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:39.244 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628617 2024-04-26T21:02:39.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:39.258 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:39.258 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:39.258 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:39.258 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:39.258 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/objectstore").st_ino) 2024-04-26T21:02:39.258 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:39.455 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628623 2024-04-26T21:02:39.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:39.470 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:39.470 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:39.470 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:39.470 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:39.470 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/fs").st_ino) 2024-04-26T21:02:39.470 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:39.672 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628618 2024-04-26T21:02:39.677 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:39.687 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:39.687 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:39.687 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:39.688 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:39.688 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/fs/xfs.yaml").st_ino) 2024-04-26T21:02:39.688 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:39.881 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628619 2024-04-26T21:02:39.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:39.896 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:39.896 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:39.896 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:39.896 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:39.896 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/mount").st_ino) 2024-04-26T21:02:39.896 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:40.093 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628620 2024-04-26T21:02:40.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:40.109 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:40.109 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:40.109 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:40.109 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:40.109 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/mount/fuse.yaml").st_ino) 2024-04-26T21:02:40.109 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:40.308 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628621 2024-04-26T21:02:40.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:40.323 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:40.323 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:40.323 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:40.323 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:40.323 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/mount/kclient.yaml.disabled").st_ino) 2024-04-26T21:02:40.323 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:40.528 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628622 2024-04-26T21:02:40.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:40.543 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:40.543 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:40.543 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:40.543 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:40.543 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/begin.yaml").st_ino) 2024-04-26T21:02:40.544 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:40.739 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628614 2024-04-26T21:02:40.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:40.754 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:40.754 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:40.754 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:40.754 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:40.754 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/%").st_ino) 2024-04-26T21:02:40.755 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:40.950 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628613 2024-04-26T21:02:40.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:40.964 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:40.964 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:40.964 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:40.964 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:40.964 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/tasks").st_ino) 2024-04-26T21:02:40.964 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:41.164 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628629 2024-04-26T21:02:41.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:41.178 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:41.178 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:41.178 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:41.179 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:41.179 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/tasks/fsx-mpi.yaml.disabled").st_ino) 2024-04-26T21:02:41.179 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:41.379 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628631 2024-04-26T21:02:41.384 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:41.394 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:41.394 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:41.394 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:41.394 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:41.394 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/tasks/ior-shared-file.yaml").st_ino) 2024-04-26T21:02:41.394 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:41.592 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628632 2024-04-26T21:02:41.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:41.606 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:41.607 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:41.607 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:41.607 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:41.607 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/tasks/cephfs_misc_tests.yaml").st_ino) 2024-04-26T21:02:41.607 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:41.805 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628630 2024-04-26T21:02:41.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:41.820 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:41.821 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:41.821 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:41.821 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:41.821 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/fs/multiclient/tasks/mdtest.yaml").st_ino) 2024-04-26T21:02:41.821 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:42.019 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628633 2024-04-26T21:02:42.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:42.034 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:42.034 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:42.034 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:42.034 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:42.034 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-disk").st_ino) 2024-04-26T21:02:42.034 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:42.231 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628534 2024-04-26T21:02:42.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:42.246 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:42.246 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:42.246 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:42.246 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:42.246 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-disk/basic").st_ino) 2024-04-26T21:02:42.246 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:42.442 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628535 2024-04-26T21:02:42.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:42.459 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:42.460 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:42.460 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:42.460 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:42.460 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-disk/basic/distros").st_ino) 2024-04-26T21:02:42.460 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:42.656 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628537 2024-04-26T21:02:42.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:42.671 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:42.671 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:42.671 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:42.671 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:42.671 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-disk/basic/%").st_ino) 2024-04-26T21:02:42.671 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:42.868 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628536 2024-04-26T21:02:42.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:42.882 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:42.882 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:42.883 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:42.883 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:42.883 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-disk/basic/tasks").st_ino) 2024-04-26T21:02:42.883 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:43.084 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628538 2024-04-26T21:02:43.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:43.099 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:43.099 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:43.099 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:43.099 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:43.099 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/ceph-disk/basic/tasks/ceph-disk.yaml").st_ino) 2024-04-26T21:02:43.099 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:43.295 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628539 2024-04-26T21:02:43.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:43.310 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:43.310 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:43.310 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:43.311 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:43.311 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd").st_ino) 2024-04-26T21:02:43.311 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:43.506 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628915 2024-04-26T21:02:43.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:43.521 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:43.521 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:43.521 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:43.521 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:43.522 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/unmap").st_ino) 2024-04-26T21:02:43.522 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:43.719 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628982 2024-04-26T21:02:43.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:43.734 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:43.734 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:43.734 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:43.734 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:43.734 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/unmap/clusters").st_ino) 2024-04-26T21:02:43.735 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:43.930 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628986 2024-04-26T21:02:43.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:43.944 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:43.944 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:43.944 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:43.945 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:43.945 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/unmap/clusters/separate-client.yaml").st_ino) 2024-04-26T21:02:43.945 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:44.144 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628987 2024-04-26T21:02:44.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:44.159 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:44.159 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:44.159 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:44.159 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:44.159 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/unmap/ceph").st_ino) 2024-04-26T21:02:44.159 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:44.353 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628984 2024-04-26T21:02:44.358 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:44.368 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:44.368 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:44.368 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:44.368 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:44.368 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/unmap/ceph/ceph.yaml").st_ino) 2024-04-26T21:02:44.368 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:44.571 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628985 2024-04-26T21:02:44.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:44.585 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:44.585 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:44.586 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:44.586 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:44.586 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/unmap/kernels").st_ino) 2024-04-26T21:02:44.586 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:44.784 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628989 2024-04-26T21:02:44.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:44.799 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:44.799 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:44.799 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:44.799 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:44.799 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/unmap/kernels/pre-single-major.yaml").st_ino) 2024-04-26T21:02:44.799 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:44.995 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628990 2024-04-26T21:02:44.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:45.009 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:45.010 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:45.010 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:45.010 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:45.010 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/unmap/kernels/single-major-on.yaml").st_ino) 2024-04-26T21:02:45.010 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:45.207 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628992 2024-04-26T21:02:45.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:45.223 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:45.223 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:45.223 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:45.223 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:45.223 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/unmap/kernels/single-major-off.yaml").st_ino) 2024-04-26T21:02:45.223 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:45.426 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628991 2024-04-26T21:02:45.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:45.441 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:45.441 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:45.441 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:45.441 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:45.441 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/unmap/conf.yaml").st_ino) 2024-04-26T21:02:45.441 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:45.630 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628988 2024-04-26T21:02:45.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:45.644 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:45.644 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:45.644 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:45.644 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:45.644 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/unmap/%").st_ino) 2024-04-26T21:02:45.644 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:45.840 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628983 2024-04-26T21:02:45.844 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:45.855 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:45.855 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:45.855 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:45.855 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:45.855 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/unmap/xfs.yaml").st_ino) 2024-04-26T21:02:45.855 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:46.054 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628995 2024-04-26T21:02:46.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:46.069 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:46.069 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:46.069 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:46.069 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:46.069 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/unmap/tasks").st_ino) 2024-04-26T21:02:46.070 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:46.263 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628993 2024-04-26T21:02:46.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:46.278 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:46.278 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:46.278 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:46.278 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:46.278 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/unmap/tasks/unmap.yaml").st_ino) 2024-04-26T21:02:46.278 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:46.466 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628994 2024-04-26T21:02:46.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:46.481 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:46.481 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:46.481 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:46.481 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:46.482 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd").st_ino) 2024-04-26T21:02:46.482 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:46.678 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628937 2024-04-26T21:02:46.682 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:46.693 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:46.693 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:46.693 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:46.693 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:46.693 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd/clusters").st_ino) 2024-04-26T21:02:46.693 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:46.891 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628939 2024-04-26T21:02:46.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:46.906 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:46.906 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:46.907 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:46.907 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:46.907 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd/clusters/fixed-3.yaml").st_ino) 2024-04-26T21:02:46.907 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:47.089 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628940 2024-04-26T21:02:47.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:47.103 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:47.103 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:47.103 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:47.103 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:47.103 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd/msgr-failures").st_ino) 2024-04-26T21:02:47.103 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:47.108 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:02:47.157 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:02:47.277 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628944 2024-04-26T21:02:47.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:47.292 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:47.292 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:47.292 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:47.292 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:47.292 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd/msgr-failures/many.yaml").st_ino) 2024-04-26T21:02:47.292 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:47.489 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628946 2024-04-26T21:02:47.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:47.504 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:47.504 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:47.504 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:47.504 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:47.505 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd/msgr-failures/few.yaml").st_ino) 2024-04-26T21:02:47.505 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:47.700 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628945 2024-04-26T21:02:47.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:47.715 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:47.716 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:47.716 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:47.716 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:47.716 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd/fs").st_ino) 2024-04-26T21:02:47.716 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:47.910 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628942 2024-04-26T21:02:47.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:47.924 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:47.925 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:47.925 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:47.925 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:47.925 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd/fs/btrfs.yaml").st_ino) 2024-04-26T21:02:47.925 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:48.121 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628943 2024-04-26T21:02:48.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:48.135 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:48.136 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:48.136 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:48.136 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:48.136 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd/conf.yaml").st_ino) 2024-04-26T21:02:48.136 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:48.333 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628941 2024-04-26T21:02:48.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:48.348 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:48.348 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:48.348 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:48.348 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:48.348 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd/%").st_ino) 2024-04-26T21:02:48.348 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:48.549 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628938 2024-04-26T21:02:48.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:48.564 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:48.564 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:48.564 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:48.564 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:48.565 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd/tasks").st_ino) 2024-04-26T21:02:48.565 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:48.760 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628947 2024-04-26T21:02:48.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:48.775 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:48.775 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:48.775 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:48.775 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:48.775 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd/tasks/rbd_workunit_suites_fsstress.yaml").st_ino) 2024-04-26T21:02:48.775 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:48.972 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628952 2024-04-26T21:02:48.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:48.987 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:48.987 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:48.987 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:48.987 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:48.987 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd/tasks/rbd_fio.yaml").st_ino) 2024-04-26T21:02:48.987 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:49.183 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628948 2024-04-26T21:02:49.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:49.198 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:49.198 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:49.199 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:49.199 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:49.199 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd/tasks/rbd_workunit_suites_ffsb.yaml").st_ino) 2024-04-26T21:02:49.199 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:49.401 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628951 2024-04-26T21:02:49.405 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:49.416 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:49.416 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:49.416 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:49.416 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:49.416 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd/tasks/rbd_workunit_suites_fsstress_ext4.yaml").st_ino) 2024-04-26T21:02:49.416 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:49.610 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628954 2024-04-26T21:02:49.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:49.625 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:49.625 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:49.625 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:49.625 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:49.626 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd/tasks/rbd_workunit_suites_iozone.yaml").st_ino) 2024-04-26T21:02:49.626 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:49.825 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628956 2024-04-26T21:02:49.829 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:49.840 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:49.840 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:49.840 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:49.840 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:49.840 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd/tasks/rbd_workunit_suites_dbench.yaml").st_ino) 2024-04-26T21:02:49.840 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:50.038 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628950 2024-04-26T21:02:50.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:50.053 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:50.053 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:50.053 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:50.053 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:50.053 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd/tasks/rbd_workunit_kernel_untar_build.yaml").st_ino) 2024-04-26T21:02:50.054 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:50.250 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628949 2024-04-26T21:02:50.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:50.265 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:50.265 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:50.265 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:50.265 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:50.265 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd/tasks/rbd_workunit_trivial_sync.yaml").st_ino) 2024-04-26T21:02:50.265 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:50.461 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628957 2024-04-26T21:02:50.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:50.476 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:50.476 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:50.476 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:50.476 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:50.477 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd/tasks/rbd_workunit_suites_fsstress_btrfs.yaml").st_ino) 2024-04-26T21:02:50.477 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:50.675 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628953 2024-04-26T21:02:50.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:50.690 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:50.690 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:50.690 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:50.690 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:50.690 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd/tasks/rbd_workunit_suites_fsx.yaml").st_ino) 2024-04-26T21:02:50.690 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:50.885 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628955 2024-04-26T21:02:50.889 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:50.899 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:50.899 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:50.899 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:50.899 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:50.899 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/singleton").st_ino) 2024-04-26T21:02:50.900 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:51.096 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628958 2024-04-26T21:02:51.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:51.111 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:51.111 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:51.111 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:51.111 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:51.111 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/singleton/msgr-failures").st_ino) 2024-04-26T21:02:51.112 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:51.293 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628963 2024-04-26T21:02:51.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:51.307 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:51.307 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:51.307 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:51.307 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:51.308 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/singleton/msgr-failures/many.yaml").st_ino) 2024-04-26T21:02:51.308 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:51.505 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628965 2024-04-26T21:02:51.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:51.521 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:51.521 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:51.522 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:51.522 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:51.522 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/singleton/msgr-failures/few.yaml").st_ino) 2024-04-26T21:02:51.522 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:51.717 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628964 2024-04-26T21:02:51.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:51.733 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:51.733 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:51.733 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:51.733 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:51.733 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/singleton/fs").st_ino) 2024-04-26T21:02:51.733 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:51.929 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628961 2024-04-26T21:02:51.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:51.944 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:51.944 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:51.945 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:51.945 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:51.945 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/singleton/fs/btrfs.yaml").st_ino) 2024-04-26T21:02:51.945 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:52.148 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628962 2024-04-26T21:02:52.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:52.163 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:52.163 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:52.163 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:52.163 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:52.163 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/singleton/conf.yaml").st_ino) 2024-04-26T21:02:52.163 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:52.358 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628960 2024-04-26T21:02:52.363 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:52.374 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:52.374 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:52.374 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:52.374 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:52.374 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/singleton/%").st_ino) 2024-04-26T21:02:52.374 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:52.571 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628959 2024-04-26T21:02:52.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:52.585 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:52.586 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:52.586 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:52.586 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:52.586 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/singleton/tasks").st_ino) 2024-04-26T21:02:52.586 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:52.783 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628966 2024-04-26T21:02:52.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:52.798 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:52.798 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:52.798 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:52.798 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:52.798 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/singleton/tasks/rbd_xfstests.yaml").st_ino) 2024-04-26T21:02:52.798 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:52.994 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628967 2024-04-26T21:02:52.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:53.010 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:53.010 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:53.010 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:53.010 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:53.010 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount").st_ino) 2024-04-26T21:02:53.011 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:53.205 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628916 2024-04-26T21:02:53.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:53.219 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:53.219 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:53.219 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:53.219 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:53.219 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount/install").st_ino) 2024-04-26T21:02:53.219 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:53.413 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628923 2024-04-26T21:02:53.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:53.428 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:53.428 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:53.428 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:53.428 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:53.428 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount/install/ceph.yaml").st_ino) 2024-04-26T21:02:53.429 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:53.630 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628924 2024-04-26T21:02:53.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:53.645 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:53.645 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:53.645 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:53.645 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:53.646 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount/clusters").st_ino) 2024-04-26T21:02:53.646 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:53.842 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628918 2024-04-26T21:02:53.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:53.856 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:53.856 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:53.857 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:53.857 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:53.857 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount/clusters/fixed-3.yaml").st_ino) 2024-04-26T21:02:53.857 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:54.054 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628919 2024-04-26T21:02:54.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:54.068 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:54.068 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:54.068 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:54.068 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:54.068 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount/msgr-failures").st_ino) 2024-04-26T21:02:54.068 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:54.262 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628925 2024-04-26T21:02:54.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:54.277 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:54.278 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:54.278 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:54.278 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:54.278 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount/msgr-failures/many.yaml").st_ino) 2024-04-26T21:02:54.278 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:54.475 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628927 2024-04-26T21:02:54.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:54.489 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:54.489 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:54.489 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:54.490 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:54.490 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount/msgr-failures/few.yaml").st_ino) 2024-04-26T21:02:54.490 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:54.691 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628926 2024-04-26T21:02:54.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:54.705 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:54.705 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:54.706 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:54.706 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:54.706 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount/fs").st_ino) 2024-04-26T21:02:54.706 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:54.904 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628921 2024-04-26T21:02:54.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:54.919 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:54.919 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:54.919 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:54.919 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:54.919 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount/fs/btrfs.yaml").st_ino) 2024-04-26T21:02:54.919 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:55.116 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628922 2024-04-26T21:02:55.121 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:55.131 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:55.131 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:55.131 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:55.131 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:55.132 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount/conf.yaml").st_ino) 2024-04-26T21:02:55.132 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:55.325 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628920 2024-04-26T21:02:55.329 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:55.340 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:55.340 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:55.340 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:55.340 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:55.340 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount/%").st_ino) 2024-04-26T21:02:55.341 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:55.538 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628917 2024-04-26T21:02:55.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:55.554 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:55.554 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:55.554 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:55.554 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:55.554 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount/tasks").st_ino) 2024-04-26T21:02:55.554 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:55.750 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628928 2024-04-26T21:02:55.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:55.765 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:55.765 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:55.765 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:55.765 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:55.765 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount/tasks/rbd_map_unmap.yaml").st_ino) 2024-04-26T21:02:55.765 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:55.966 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628935 2024-04-26T21:02:55.970 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:55.980 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:55.980 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:55.980 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:55.980 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:55.980 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount/tasks/rbd_kfsx.yaml").st_ino) 2024-04-26T21:02:55.980 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:56.179 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628933 2024-04-26T21:02:56.184 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:56.194 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:56.194 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:56.194 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:56.194 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:56.194 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount/tasks/rbd_image_read.yaml").st_ino) 2024-04-26T21:02:56.194 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:56.390 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628931 2024-04-26T21:02:56.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:56.405 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:56.405 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:56.405 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:56.405 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:56.405 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount/tasks/rbd_simple_big.yaml").st_ino) 2024-04-26T21:02:56.405 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:56.600 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628936 2024-04-26T21:02:56.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:56.614 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:56.614 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:56.614 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:56.614 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:56.614 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount/tasks/rbd_concurrent.yaml").st_ino) 2024-04-26T21:02:56.615 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:56.816 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628929 2024-04-26T21:02:56.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:56.831 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:56.831 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:56.831 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:56.831 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:56.831 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount/tasks/rbd_map_snapshot_io.yaml").st_ino) 2024-04-26T21:02:56.831 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:57.026 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628934 2024-04-26T21:02:57.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:57.040 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:57.040 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:57.041 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:57.041 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:57.041 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount/tasks/rbd_huge_tickets.yaml").st_ino) 2024-04-26T21:02:57.041 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:57.238 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628930 2024-04-26T21:02:57.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:57.253 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:57.253 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:57.253 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:57.253 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:57.253 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/rbd-nomount/tasks/rbd_kernel.yaml").st_ino) 2024-04-26T21:02:57.253 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:57.448 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628932 2024-04-26T21:02:57.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:57.463 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:57.463 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:57.463 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:57.463 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:57.463 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/thrash").st_ino) 2024-04-26T21:02:57.464 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:57.655 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628968 2024-04-26T21:02:57.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:57.670 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:57.670 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:57.670 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:57.670 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:57.670 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/thrash/clusters").st_ino) 2024-04-26T21:02:57.670 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:57.853 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628970 2024-04-26T21:02:57.858 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:57.867 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:57.867 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:57.868 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:57.868 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:57.868 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/thrash/clusters/fixed-3.yaml").st_ino) 2024-04-26T21:02:57.868 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:58.063 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628971 2024-04-26T21:02:58.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:58.078 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:58.078 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:58.078 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:58.078 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:58.078 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/thrash/fs").st_ino) 2024-04-26T21:02:58.078 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:58.282 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628973 2024-04-26T21:02:58.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:58.298 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:58.298 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:58.298 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:58.298 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:58.298 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/thrash/fs/btrfs.yaml").st_ino) 2024-04-26T21:02:58.298 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:58.493 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628974 2024-04-26T21:02:58.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:58.508 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:58.508 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:58.508 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:58.508 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:58.508 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/thrash/workloads").st_ino) 2024-04-26T21:02:58.508 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:58.708 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628978 2024-04-26T21:02:58.712 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:58.722 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:58.722 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:58.722 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:58.722 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:58.722 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/thrash/workloads/rbd_fio.yaml").st_ino) 2024-04-26T21:02:58.722 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:58.918 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628979 2024-04-26T21:02:58.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:58.933 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:58.933 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:58.933 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:58.933 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:58.933 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/thrash/workloads/rbd_workunit_suites_ffsb.yaml").st_ino) 2024-04-26T21:02:58.933 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:59.130 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628980 2024-04-26T21:02:59.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:59.145 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:59.145 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:59.145 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:59.145 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:59.145 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/thrash/workloads/rbd_workunit_suites_iozone.yaml.disabled").st_ino) 2024-04-26T21:02:59.145 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:59.342 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628981 2024-04-26T21:02:59.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:59.357 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:59.357 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:59.357 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:59.357 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:59.357 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/thrash/conf.yaml").st_ino) 2024-04-26T21:02:59.357 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:59.554 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628972 2024-04-26T21:02:59.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:59.570 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:59.570 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:59.570 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:59.570 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:59.570 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/thrash/%").st_ino) 2024-04-26T21:02:59.570 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:59.768 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628969 2024-04-26T21:02:59.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:59.783 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:59.783 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:59.783 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:59.783 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:59.784 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/thrash/thrashers").st_ino) 2024-04-26T21:02:59.784 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:02:59.981 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628975 2024-04-26T21:02:59.986 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:02:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:02:59.997 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:02:59.997 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:02:59.997 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:02:59.997 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:02:59.997 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/thrash/thrashers/default.yaml").st_ino) 2024-04-26T21:02:59.997 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:00.198 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628976 2024-04-26T21:03:00.202 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:00.212 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:00.212 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:00.212 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:00.213 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:00.213 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/krbd/thrash/thrashers/mon-thrasher.yaml").st_ino) 2024-04-26T21:03:00.213 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:00.408 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628977 2024-04-26T21:03:00.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:00.423 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:00.423 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:00.423 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:00.423 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:00.423 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/buildpackages").st_ino) 2024-04-26T21:03:00.423 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:00.619 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628485 2024-04-26T21:03:00.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:00.635 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:00.635 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:00.635 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:00.635 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:00.635 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/buildpackages/tests").st_ino) 2024-04-26T21:03:00.635 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:00.831 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628491 2024-04-26T21:03:00.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:00.847 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:00.847 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:00.847 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:00.848 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:00.848 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/buildpackages/tests/distros").st_ino) 2024-04-26T21:03:00.848 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:01.045 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628493 2024-04-26T21:03:01.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:01.060 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:01.060 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:01.060 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:01.060 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:01.060 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/buildpackages/tests/%").st_ino) 2024-04-26T21:03:01.060 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:01.258 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628492 2024-04-26T21:03:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:01.274 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:01.274 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:01.274 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:01.274 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:01.274 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/buildpackages/tests/tasks").st_ino) 2024-04-26T21:03:01.274 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:01.470 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628494 2024-04-26T21:03:01.474 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:01.485 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:01.485 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:01.485 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:01.485 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:01.485 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/buildpackages/tests/tasks/release.yaml").st_ino) 2024-04-26T21:03:01.485 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:01.686 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628495 2024-04-26T21:03:01.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:01.700 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:01.700 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:01.700 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:01.700 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:01.700 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/buildpackages/any").st_ino) 2024-04-26T21:03:01.700 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:01.896 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628486 2024-04-26T21:03:01.900 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:01.911 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:01.911 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:01.911 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:01.911 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:01.911 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/buildpackages/any/distros").st_ino) 2024-04-26T21:03:01.911 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:02.108 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628488 2024-04-26T21:03:02.113 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:02.123 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:02.123 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:02.123 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:02.123 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:02.123 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/buildpackages/any/%").st_ino) 2024-04-26T21:03:02.123 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:02.307 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628487 2024-04-26T21:03:02.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:02.322 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:02.322 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:02.322 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:02.322 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:02.323 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/buildpackages/any/tasks").st_ino) 2024-04-26T21:03:02.323 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:02.517 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628489 2024-04-26T21:03:02.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:02.531 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:02.532 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:02.532 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:02.532 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:02.532 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/buildpackages/any/tasks/release.yaml").st_ino) 2024-04-26T21:03:02.532 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:02.729 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628490 2024-04-26T21:03:02.733 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:02.743 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:02.744 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:02.744 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:02.744 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:02.744 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados").st_ino) 2024-04-26T21:03:02.744 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:02.939 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629120 2024-04-26T21:03:02.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:02.953 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:02.953 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:02.953 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:02.953 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:02.954 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/objectstore").st_ino) 2024-04-26T21:03:02.954 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:03.149 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629201 2024-04-26T21:03:03.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:03.164 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:03.164 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:03.164 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:03.165 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:03.165 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/objectstore/filestore-idempotent-aio-journal.yaml").st_ino) 2024-04-26T21:03:03.165 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:03.355 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629205 2024-04-26T21:03:03.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:03.370 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:03.370 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:03.370 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:03.370 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:03.370 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/objectstore/ceph_objectstore_tool.yaml").st_ino) 2024-04-26T21:03:03.370 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:03.567 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629203 2024-04-26T21:03:03.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:03.582 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:03.582 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:03.582 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:03.582 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:03.582 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/objectstore/fusestore.yaml").st_ino) 2024-04-26T21:03:03.582 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:03.778 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629207 2024-04-26T21:03:03.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:03.793 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:03.793 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:03.793 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:03.793 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:03.793 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/objectstore/keyvaluedb.yaml").st_ino) 2024-04-26T21:03:03.793 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:03.990 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629208 2024-04-26T21:03:03.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:04.004 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:04.004 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:04.004 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:04.004 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:04.004 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/objectstore/objectstore.yaml").st_ino) 2024-04-26T21:03:04.005 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:04.206 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629210 2024-04-26T21:03:04.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:04.220 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:04.221 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:04.221 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:04.221 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:04.221 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/objectstore/alloc-hint.yaml").st_ino) 2024-04-26T21:03:04.221 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:04.420 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629202 2024-04-26T21:03:04.424 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:04.435 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:04.435 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:04.435 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:04.435 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:04.435 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/objectstore/filejournal.yaml").st_ino) 2024-04-26T21:03:04.435 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:04.629 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629204 2024-04-26T21:03:04.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:04.644 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:04.644 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:04.644 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:04.644 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:04.644 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/objectstore/objectcacher-stress.yaml").st_ino) 2024-04-26T21:03:04.644 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:04.848 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629209 2024-04-26T21:03:04.853 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:04.863 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:04.863 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:04.863 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:04.863 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:04.863 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/objectstore/filestore-idempotent.yaml").st_ino) 2024-04-26T21:03:04.863 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:05.058 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629206 2024-04-26T21:03:05.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:05.073 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:05.073 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:05.073 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:05.073 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:05.073 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton").st_ino) 2024-04-26T21:03:05.073 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:05.269 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629222 2024-04-26T21:03:05.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:05.284 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:05.284 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:05.284 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:05.284 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:05.284 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/msgr").st_ino) 2024-04-26T21:03:05.284 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:05.481 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629252 2024-04-26T21:03:05.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:05.495 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:05.496 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:05.496 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:05.496 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:05.496 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/msgr-failures").st_ino) 2024-04-26T21:03:05.496 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:05.694 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629253 2024-04-26T21:03:05.699 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:05.709 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:05.709 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:05.709 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:05.709 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:05.709 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/msgr-failures/many.yaml").st_ino) 2024-04-26T21:03:05.709 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:05.907 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629255 2024-04-26T21:03:05.912 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:05.922 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:05.923 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:05.923 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:05.923 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:05.923 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/msgr-failures/few.yaml").st_ino) 2024-04-26T21:03:05.923 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:06.121 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629254 2024-04-26T21:03:06.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:06.135 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:06.135 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:06.135 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:06.135 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:06.135 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/objectstore").st_ino) 2024-04-26T21:03:06.135 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:06.333 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629256 2024-04-26T21:03:06.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:06.348 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:06.348 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:06.348 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:06.348 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:06.348 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/fs").st_ino) 2024-04-26T21:03:06.349 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:06.546 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629250 2024-04-26T21:03:06.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:06.560 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:06.560 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:06.560 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:06.560 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:06.560 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/fs/xfs.yaml").st_ino) 2024-04-26T21:03:06.560 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:06.757 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629251 2024-04-26T21:03:06.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:06.772 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:06.773 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:06.773 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:06.773 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:06.773 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/%").st_ino) 2024-04-26T21:03:06.773 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:06.969 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629223 2024-04-26T21:03:06.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:06.984 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:06.984 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:06.984 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:06.984 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:06.984 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all").st_ino) 2024-04-26T21:03:06.984 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:07.182 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629224 2024-04-26T21:03:07.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:07.198 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:07.198 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:07.198 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:07.198 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:07.198 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/divergent_priors.yaml").st_ino) 2024-04-26T21:03:07.198 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:07.382 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629227 2024-04-26T21:03:07.386 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:07.396 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:07.397 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:07.397 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:07.397 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:07.397 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/watch-notify-same-primary.yaml").st_ino) 2024-04-26T21:03:07.397 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:07.594 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629249 2024-04-26T21:03:07.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:07.610 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:07.611 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:07.611 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:07.611 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:07.611 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/ec-lost-unfound.yaml").st_ino) 2024-04-26T21:03:07.611 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:07.806 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629230 2024-04-26T21:03:07.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:07.821 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:07.821 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:07.821 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:07.821 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:07.821 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/rest-api.yaml").st_ino) 2024-04-26T21:03:07.821 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:08.018 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629245 2024-04-26T21:03:08.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:08.032 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:08.032 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:08.032 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:08.033 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:08.033 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/mon-config-keys.yaml").st_ino) 2024-04-26T21:03:08.033 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:08.233 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629233 2024-04-26T21:03:08.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:08.248 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:08.248 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:08.248 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:08.248 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:08.248 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/mon-thrasher.yaml").st_ino) 2024-04-26T21:03:08.248 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:08.447 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629235 2024-04-26T21:03:08.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:08.462 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:08.462 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:08.462 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:08.462 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:08.462 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/osd-recovery-incomplete.yaml").st_ino) 2024-04-26T21:03:08.462 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:08.659 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629237 2024-04-26T21:03:08.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:08.675 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:08.675 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:08.675 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:08.675 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:08.675 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/osd-recovery.yaml").st_ino) 2024-04-26T21:03:08.675 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:08.875 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629238 2024-04-26T21:03:08.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:08.890 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:08.890 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:08.890 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:08.890 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:08.890 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/admin-socket.yaml").st_ino) 2024-04-26T21:03:08.891 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:09.095 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629225 2024-04-26T21:03:09.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:09.111 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:09.111 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:09.111 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:09.111 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:09.111 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/reg11184.yaml").st_ino) 2024-04-26T21:03:09.111 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:09.308 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629243 2024-04-26T21:03:09.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:09.323 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:09.324 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:09.324 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:09.324 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:09.324 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/mon-seesaw.yaml").st_ino) 2024-04-26T21:03:09.324 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:09.524 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629234 2024-04-26T21:03:09.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:09.540 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:09.540 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:09.540 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:09.540 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:09.540 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/radostool.yaml").st_ino) 2024-04-26T21:03:09.540 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:09.735 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629241 2024-04-26T21:03:09.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:09.751 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:09.751 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:09.751 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:09.751 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:09.751 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/thrash_cache_writeback_proxy_none.yaml").st_ino) 2024-04-26T21:03:09.751 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:09.948 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629248 2024-04-26T21:03:09.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:09.962 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:09.963 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:09.963 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:09.963 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:09.963 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/dump-stuck.yaml").st_ino) 2024-04-26T21:03:09.963 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:10.160 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629229 2024-04-26T21:03:10.165 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:10.177 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:10.177 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:10.177 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:10.177 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:10.177 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/resolve_stuck_peering.yaml").st_ino) 2024-04-26T21:03:10.178 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:10.374 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629244 2024-04-26T21:03:10.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:10.389 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:10.389 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:10.389 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:10.389 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:10.389 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/osd-backfill.yaml").st_ino) 2024-04-26T21:03:10.389 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:10.577 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629236 2024-04-26T21:03:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:10.591 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:10.591 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:10.591 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:10.591 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:10.592 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/divergent_priors2.yaml").st_ino) 2024-04-26T21:03:10.592 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:10.793 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629228 2024-04-26T21:03:10.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:10.808 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:10.808 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:10.808 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:10.808 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:10.809 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/peer.yaml").st_ino) 2024-04-26T21:03:10.809 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:11.006 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629239 2024-04-26T21:03:11.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:11.022 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:11.022 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:11.022 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:11.022 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:11.022 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/test_envlibrados_for_rocksdb.yaml").st_ino) 2024-04-26T21:03:11.022 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:11.222 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629246 2024-04-26T21:03:11.227 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:11.238 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:11.238 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:11.238 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:11.238 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:11.238 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/thrash-rados.yaml").st_ino) 2024-04-26T21:03:11.238 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:11.434 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629247 2024-04-26T21:03:11.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:11.449 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:11.449 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:11.449 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:11.450 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:11.450 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/lost-unfound.yaml").st_ino) 2024-04-26T21:03:11.450 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:11.647 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629232 2024-04-26T21:03:11.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:11.662 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:11.662 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:11.662 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:11.662 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:11.662 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/cephtool.yaml").st_ino) 2024-04-26T21:03:11.662 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:11.859 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629226 2024-04-26T21:03:11.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:11.873 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:11.873 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:11.874 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:11.874 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:11.874 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/pg-removal-interruption.yaml").st_ino) 2024-04-26T21:03:11.874 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:12.076 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629240 2024-04-26T21:03:12.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:12.090 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:12.091 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:12.091 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:12.091 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:12.091 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/rebuild-mondb.yaml").st_ino) 2024-04-26T21:03:12.091 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:12.286 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629242 2024-04-26T21:03:12.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:12.300 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:12.300 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:12.300 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:12.300 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:12.300 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/all/lost-unfound-delete.yaml").st_ino) 2024-04-26T21:03:12.301 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:12.498 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629231 2024-04-26T21:03:12.502 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:12.512 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:12.512 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:12.512 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:12.512 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:12.512 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton/rados.yaml").st_ino) 2024-04-26T21:03:12.512 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:12.712 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629257 2024-04-26T21:03:12.716 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:12.727 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:12.727 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:12.727 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:12.727 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:12.728 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic").st_ino) 2024-04-26T21:03:12.728 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:12.924 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629121 2024-04-26T21:03:12.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:12.939 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:12.939 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:12.939 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:12.939 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:12.939 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/msgr").st_ino) 2024-04-26T21:03:12.939 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:13.133 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629134 2024-04-26T21:03:13.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:13.149 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:13.149 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:13.149 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:13.149 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:13.149 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/msgr/async.yaml").st_ino) 2024-04-26T21:03:13.149 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:13.346 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629135 2024-04-26T21:03:13.351 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:13.361 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:13.362 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:13.362 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:13.362 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:13.362 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/msgr/simple.yaml").st_ino) 2024-04-26T21:03:13.362 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:13.558 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629137 2024-04-26T21:03:13.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:13.573 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:13.573 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:13.574 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:13.574 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:13.574 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/msgr/random.yaml").st_ino) 2024-04-26T21:03:13.574 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:13.775 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629136 2024-04-26T21:03:13.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:13.791 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:13.791 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:13.791 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:13.791 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:13.791 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/clusters").st_ino) 2024-04-26T21:03:13.791 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:13.989 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629123 2024-04-26T21:03:13.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:14.005 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:14.005 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:14.005 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:14.005 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:14.005 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/clusters/openstack.yaml").st_ino) 2024-04-26T21:03:14.005 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:14.201 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629126 2024-04-26T21:03:14.205 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:14.215 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:14.216 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:14.216 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:14.216 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:14.216 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/clusters/fixed-2.yaml").st_ino) 2024-04-26T21:03:14.216 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:14.411 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629125 2024-04-26T21:03:14.415 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:14.425 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:14.425 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:14.425 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:14.426 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:14.426 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/clusters/+").st_ino) 2024-04-26T21:03:14.426 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:14.621 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629124 2024-04-26T21:03:14.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:14.636 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:14.636 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:14.636 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:14.636 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:14.636 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/msgr-failures").st_ino) 2024-04-26T21:03:14.636 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:14.832 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629131 2024-04-26T21:03:14.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:14.847 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:14.847 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:14.847 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:14.847 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:14.847 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/msgr-failures/many.yaml").st_ino) 2024-04-26T21:03:14.847 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:15.044 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629133 2024-04-26T21:03:15.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:15.058 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:15.059 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:15.059 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:15.059 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:15.059 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/msgr-failures/few.yaml").st_ino) 2024-04-26T21:03:15.059 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:15.262 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629132 2024-04-26T21:03:15.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:15.278 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:15.278 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:15.278 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:15.278 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:15.278 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/objectstore").st_ino) 2024-04-26T21:03:15.278 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:15.476 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629138 2024-04-26T21:03:15.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:15.491 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:15.491 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:15.491 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:15.491 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:15.491 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/fs").st_ino) 2024-04-26T21:03:15.491 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:15.687 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629127 2024-04-26T21:03:15.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:15.702 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:15.702 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:15.703 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:15.703 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:15.703 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/fs/btrfs.yaml").st_ino) 2024-04-26T21:03:15.703 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:15.899 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629128 2024-04-26T21:03:15.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:15.914 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:15.914 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:15.914 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:15.914 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:15.914 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/fs/xfs.yaml").st_ino) 2024-04-26T21:03:15.914 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:16.114 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629129 2024-04-26T21:03:16.119 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:16.128 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:16.129 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:16.129 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:16.129 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:16.129 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/%").st_ino) 2024-04-26T21:03:16.129 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:16.326 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629122 2024-04-26T21:03:16.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:16.340 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:16.340 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:16.340 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:16.341 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:16.341 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/mon_kv_backend").st_ino) 2024-04-26T21:03:16.341 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:16.538 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629130 2024-04-26T21:03:16.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:16.552 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:16.552 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:16.552 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:16.552 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:16.552 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/tasks").st_ino) 2024-04-26T21:03:16.553 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:16.748 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629140 2024-04-26T21:03:16.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:16.763 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:16.763 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:16.763 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:16.763 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:16.763 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/tasks/readwrite.yaml").st_ino) 2024-04-26T21:03:16.764 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:16.964 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629149 2024-04-26T21:03:16.968 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:16.979 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:16.979 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:16.979 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:16.979 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:16.979 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/tasks/rados_cls_all.yaml").st_ino) 2024-04-26T21:03:16.979 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:17.183 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629142 2024-04-26T21:03:17.188 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:17.198 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:17.198 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:17.198 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:17.198 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:17.198 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/tasks/rados_workunit_loadgen_mostlyread.yaml").st_ino) 2024-04-26T21:03:17.198 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:17.296 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:03:17.304 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:03:17.406 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629148 2024-04-26T21:03:17.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:17.423 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:17.423 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:17.423 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:17.423 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:17.423 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/tasks/rados_stress_watch.yaml").st_ino) 2024-04-26T21:03:17.423 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:17.621 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629144 2024-04-26T21:03:17.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:17.636 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:17.636 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:17.636 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:17.636 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:17.636 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/tasks/rados_workunit_loadgen_big.yaml").st_ino) 2024-04-26T21:03:17.636 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:17.833 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629146 2024-04-26T21:03:17.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:17.848 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:17.848 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:17.848 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:17.849 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:17.849 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/tasks/scrub_test.yaml").st_ino) 2024-04-26T21:03:17.849 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:18.045 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629152 2024-04-26T21:03:18.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:18.061 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:18.061 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:18.061 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:18.061 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:18.061 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/tasks/rados_workunit_loadgen_mix.yaml").st_ino) 2024-04-26T21:03:18.061 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:18.259 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629147 2024-04-26T21:03:18.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:18.273 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:18.273 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:18.274 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:18.274 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:18.274 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/tasks/rados_python.yaml").st_ino) 2024-04-26T21:03:18.274 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:18.473 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629143 2024-04-26T21:03:18.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:18.488 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:18.488 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:18.489 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:18.489 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:18.489 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/tasks/repair_test.yaml").st_ino) 2024-04-26T21:03:18.489 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:18.691 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629150 2024-04-26T21:03:18.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:18.706 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:18.707 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:18.707 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:18.707 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:18.707 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/tasks/rados_api_tests.yaml").st_ino) 2024-04-26T21:03:18.707 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:18.906 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629141 2024-04-26T21:03:18.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:18.919 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:18.919 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:18.919 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:18.919 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:18.919 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/tasks/rgw_snaps.yaml").st_ino) 2024-04-26T21:03:18.920 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:19.113 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629151 2024-04-26T21:03:19.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:19.128 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:19.128 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:19.128 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:19.128 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:19.128 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/tasks/rados_striper.yaml").st_ino) 2024-04-26T21:03:19.128 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:19.329 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629145 2024-04-26T21:03:19.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:19.343 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:19.343 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:19.343 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:19.343 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:19.343 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/basic/rados.yaml").st_ino) 2024-04-26T21:03:19.343 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:19.527 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629139 2024-04-26T21:03:19.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:19.541 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:19.542 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:19.542 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:19.542 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:19.542 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-shec").st_ino) 2024-04-26T21:03:19.542 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:19.741 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629291 2024-04-26T21:03:19.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:19.755 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:19.756 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:19.756 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:19.756 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:19.756 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/leveldb.yaml").st_ino) 2024-04-26T21:03:19.756 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:19.952 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629299 2024-04-26T21:03:19.957 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:19.968 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:19.968 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:19.968 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:19.968 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:19.968 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/clusters").st_ino) 2024-04-26T21:03:19.968 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:20.166 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629293 2024-04-26T21:03:20.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:20.180 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:20.180 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:20.181 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:20.181 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:20.181 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/clusters/openstack.yaml").st_ino) 2024-04-26T21:03:20.181 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:20.378 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629296 2024-04-26T21:03:20.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:20.393 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:20.393 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:20.393 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:20.393 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:20.393 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/clusters/+").st_ino) 2024-04-26T21:03:20.393 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:20.591 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629294 2024-04-26T21:03:20.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:20.606 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:20.607 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:20.607 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:20.607 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:20.607 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/clusters/fixed-4.yaml").st_ino) 2024-04-26T21:03:20.607 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:20.803 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629295 2024-04-26T21:03:20.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:20.818 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:20.818 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:20.818 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:20.818 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:20.818 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/msgr-failures").st_ino) 2024-04-26T21:03:20.818 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:21.014 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629300 2024-04-26T21:03:21.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:21.029 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:21.029 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:21.029 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:21.029 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:21.029 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/objectstore").st_ino) 2024-04-26T21:03:21.029 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:21.235 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629301 2024-04-26T21:03:21.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:21.251 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:21.251 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:21.251 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:21.251 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:21.251 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/fs").st_ino) 2024-04-26T21:03:21.251 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:21.444 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629297 2024-04-26T21:03:21.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:21.459 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:21.459 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:21.459 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:21.459 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:21.459 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/fs/xfs.yaml").st_ino) 2024-04-26T21:03:21.460 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:21.656 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629298 2024-04-26T21:03:21.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:21.671 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:21.671 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:21.671 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:21.671 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:21.672 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/workloads").st_ino) 2024-04-26T21:03:21.672 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:21.874 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629305 2024-04-26T21:03:21.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:21.888 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:21.888 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:21.888 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:21.889 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:21.889 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/workloads/ec-rados-plugin=shec-k=4-m=3-c=2.yaml").st_ino) 2024-04-26T21:03:21.889 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:22.084 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629306 2024-04-26T21:03:22.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:22.099 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:22.099 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:22.099 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:22.099 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:22.099 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/%").st_ino) 2024-04-26T21:03:22.099 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:22.287 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629292 2024-04-26T21:03:22.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:22.301 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:22.301 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:22.301 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:22.302 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:22.302 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/thrashers").st_ino) 2024-04-26T21:03:22.302 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:22.502 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629303 2024-04-26T21:03:22.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:22.518 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:22.518 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:22.518 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:22.518 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:22.518 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/thrashers/default.yaml").st_ino) 2024-04-26T21:03:22.518 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:22.719 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629304 2024-04-26T21:03:22.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:22.735 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:22.735 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:22.735 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:22.735 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:22.735 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-shec/rados.yaml").st_ino) 2024-04-26T21:03:22.735 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:22.931 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629302 2024-04-26T21:03:22.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:22.945 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:22.945 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:22.945 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:22.945 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:22.946 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code").st_ino) 2024-04-26T21:03:22.946 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:23.142 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629307 2024-04-26T21:03:23.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:23.157 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:23.157 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:23.157 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:23.157 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:23.157 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/fast").st_ino) 2024-04-26T21:03:23.158 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:23.354 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629310 2024-04-26T21:03:23.359 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:23.369 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:23.369 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:23.369 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:23.369 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:23.369 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/fast/normal.yaml").st_ino) 2024-04-26T21:03:23.369 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:23.565 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629312 2024-04-26T21:03:23.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:23.580 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:23.580 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:23.580 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:23.581 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:23.581 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/fast/fast.yaml").st_ino) 2024-04-26T21:03:23.581 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:23.784 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629311 2024-04-26T21:03:23.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:23.798 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:23.798 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:23.798 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:23.799 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:23.799 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/leveldb.yaml").st_ino) 2024-04-26T21:03:23.799 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:24.004 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629314 2024-04-26T21:03:24.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:24.020 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:24.020 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:24.020 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:24.020 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:24.021 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/clusters").st_ino) 2024-04-26T21:03:24.021 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:24.205 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629309 2024-04-26T21:03:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:24.219 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:24.219 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:24.219 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:24.220 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:24.220 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/msgr-failures").st_ino) 2024-04-26T21:03:24.220 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:24.414 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629315 2024-04-26T21:03:24.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:24.429 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:24.429 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:24.429 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:24.429 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:24.429 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/objectstore").st_ino) 2024-04-26T21:03:24.429 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:24.626 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629316 2024-04-26T21:03:24.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:24.641 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:24.641 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:24.641 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:24.641 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:24.641 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/fs").st_ino) 2024-04-26T21:03:24.642 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:24.833 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629313 2024-04-26T21:03:24.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:24.848 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:24.848 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:24.848 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:24.848 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:24.849 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/workloads").st_ino) 2024-04-26T21:03:24.849 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:25.042 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629324 2024-04-26T21:03:25.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:25.058 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:25.058 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:25.058 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:25.058 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:25.058 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/workloads/ec-small-objects-fast-read-overwrites.yaml").st_ino) 2024-04-26T21:03:25.059 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:25.256 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629329 2024-04-26T21:03:25.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:25.271 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:25.271 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:25.271 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:25.271 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:25.271 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/workloads/ec-small-objects-overwrites.yaml").st_ino) 2024-04-26T21:03:25.271 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:25.456 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629331 2024-04-26T21:03:25.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:25.470 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:25.470 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:25.470 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:25.470 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:25.470 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/workloads/ec-rados-plugin=jerasure-k=3-m=1.yaml").st_ino) 2024-04-26T21:03:25.470 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:25.660 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629327 2024-04-26T21:03:25.665 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:25.675 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:25.676 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:25.676 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:25.676 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:25.676 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/workloads/ec-snaps-few-objects-overwrites.yaml").st_ino) 2024-04-26T21:03:25.676 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:25.868 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629333 2024-04-26T21:03:25.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:25.883 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:25.884 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:25.884 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:25.884 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:25.884 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/workloads/ec-radosbench.yaml").st_ino) 2024-04-26T21:03:25.884 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:26.080 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629328 2024-04-26T21:03:26.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:26.095 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:26.095 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:26.096 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:26.096 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:26.096 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/workloads/ec-rados-plugin=jerasure-k=2-m=1.yaml").st_ino) 2024-04-26T21:03:26.096 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:26.293 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629326 2024-04-26T21:03:26.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:26.309 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:26.309 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:26.309 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:26.309 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:26.309 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/workloads/ec-small-objects.yaml").st_ino) 2024-04-26T21:03:26.309 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:26.499 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629332 2024-04-26T21:03:26.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:26.514 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:26.514 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:26.514 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:26.514 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:26.514 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/workloads/ec-small-objects-fast-read.yaml").st_ino) 2024-04-26T21:03:26.514 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:26.709 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629330 2024-04-26T21:03:26.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:26.724 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:26.724 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:26.724 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:26.724 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:26.724 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/workloads/ec-pool-snaps-few-objects-overwrites.yaml").st_ino) 2024-04-26T21:03:26.724 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:26.920 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629325 2024-04-26T21:03:26.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:26.933 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:26.934 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:26.934 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:26.934 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:26.934 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/%").st_ino) 2024-04-26T21:03:26.934 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:27.130 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629308 2024-04-26T21:03:27.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:27.145 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:27.145 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:27.145 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:27.145 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:27.145 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/thrashers").st_ino) 2024-04-26T21:03:27.145 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:27.341 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629318 2024-04-26T21:03:27.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:27.355 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:27.356 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:27.356 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:27.356 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:27.356 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/thrashers/default.yaml").st_ino) 2024-04-26T21:03:27.356 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:27.554 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629319 2024-04-26T21:03:27.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:27.570 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:27.570 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:27.570 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:27.570 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:27.571 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/thrashers/fastread.yaml").st_ino) 2024-04-26T21:03:27.571 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:27.767 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629320 2024-04-26T21:03:27.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:27.783 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:27.783 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:27.783 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:27.783 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:27.783 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/thrashers/morepggrow.yaml").st_ino) 2024-04-26T21:03:27.783 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:27.980 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629322 2024-04-26T21:03:27.984 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:27.995 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:27.995 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:27.995 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:27.995 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:27.995 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/thrashers/pggrow.yaml").st_ino) 2024-04-26T21:03:27.995 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:28.191 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629323 2024-04-26T21:03:28.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:28.206 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:28.206 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:28.207 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:28.207 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:28.207 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/thrashers/mapgap.yaml").st_ino) 2024-04-26T21:03:28.207 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:28.405 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629321 2024-04-26T21:03:28.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:28.419 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:28.419 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:28.419 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:28.419 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:28.419 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code/rados.yaml").st_ino) 2024-04-26T21:03:28.419 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:28.603 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629317 2024-04-26T21:03:28.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:28.617 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:28.617 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:28.617 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:28.617 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:28.617 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton-nomsgr").st_ino) 2024-04-26T21:03:28.617 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:28.818 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629211 2024-04-26T21:03:28.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:28.833 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:28.833 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:28.833 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:28.833 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:28.833 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton-nomsgr/%").st_ino) 2024-04-26T21:03:28.833 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:29.037 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629212 2024-04-26T21:03:29.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:29.052 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:29.052 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:29.052 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:29.052 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:29.052 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton-nomsgr/all").st_ino) 2024-04-26T21:03:29.052 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:29.247 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629213 2024-04-26T21:03:29.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:29.262 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:29.263 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:29.263 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:29.263 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:29.263 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton-nomsgr/all/multi-backfill-reject.yaml").st_ino) 2024-04-26T21:03:29.263 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:29.458 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629219 2024-04-26T21:03:29.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:29.473 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:29.474 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:29.474 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:29.474 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:29.474 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton-nomsgr/all/full-tiering.yaml").st_ino) 2024-04-26T21:03:29.474 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:29.669 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629217 2024-04-26T21:03:29.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:29.684 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:29.684 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:29.684 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:29.684 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:29.684 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton-nomsgr/all/cache-fs-trunc.yaml").st_ino) 2024-04-26T21:03:29.685 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:29.884 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629214 2024-04-26T21:03:29.888 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:29.899 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:29.899 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:29.899 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:29.899 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:29.899 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton-nomsgr/all/msgr.yaml").st_ino) 2024-04-26T21:03:29.899 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:30.095 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629218 2024-04-26T21:03:30.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:30.110 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:30.110 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:30.110 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:30.110 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:30.110 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton-nomsgr/all/ceph-post-file.yaml").st_ino) 2024-04-26T21:03:30.110 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:30.303 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629215 2024-04-26T21:03:30.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:30.320 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:30.320 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:30.320 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:30.320 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:30.320 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton-nomsgr/all/export-after-evict.yaml").st_ino) 2024-04-26T21:03:30.320 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:30.520 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629216 2024-04-26T21:03:30.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:30.536 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:30.536 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:30.537 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:30.537 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:30.537 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton-nomsgr/all/valgrind-leaks.yaml").st_ino) 2024-04-26T21:03:30.537 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:30.741 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629220 2024-04-26T21:03:30.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:30.756 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:30.756 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:30.756 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:30.756 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:30.756 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/singleton-nomsgr/rados.yaml").st_ino) 2024-04-26T21:03:30.756 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:30.950 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629221 2024-04-26T21:03:30.954 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:30.964 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:30.964 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:30.964 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:30.965 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:30.965 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade").st_ino) 2024-04-26T21:03:30.965 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:31.166 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629378 2024-04-26T21:03:31.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:31.181 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:31.181 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:31.181 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:31.181 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:31.181 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton").st_ino) 2024-04-26T21:03:31.181 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:31.385 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629379 2024-04-26T21:03:31.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:31.399 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:31.399 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:31.399 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:31.399 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:31.400 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/6-next-mon").st_ino) 2024-04-26T21:03:31.400 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:31.599 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629399 2024-04-26T21:03:31.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:31.613 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:31.613 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:31.613 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:31.613 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:31.614 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/6-next-mon/monb.yaml").st_ino) 2024-04-26T21:03:31.614 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:31.813 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629400 2024-04-26T21:03:31.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:31.829 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:31.829 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:31.829 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:31.829 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:31.829 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/8-next-mon").st_ino) 2024-04-26T21:03:31.829 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:32.027 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629405 2024-04-26T21:03:32.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:32.042 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:32.042 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:32.042 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:32.042 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:32.042 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/8-next-mon/monc.yaml").st_ino) 2024-04-26T21:03:32.042 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:32.239 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629406 2024-04-26T21:03:32.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:32.253 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:32.253 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:32.254 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:32.254 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:32.254 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/1-jewel-install").st_ino) 2024-04-26T21:03:32.254 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:32.447 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629385 2024-04-26T21:03:32.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:32.463 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:32.463 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:32.463 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:32.463 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:32.463 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/1-jewel-install/jewel.yaml").st_ino) 2024-04-26T21:03:32.463 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:32.667 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629386 2024-04-26T21:03:32.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:32.682 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:32.682 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:32.682 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:32.682 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:32.682 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/5-workload").st_ino) 2024-04-26T21:03:32.682 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:32.879 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629393 2024-04-26T21:03:32.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:32.894 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:32.894 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:32.894 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:32.894 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:32.894 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/5-workload/readwrite.yaml").st_ino) 2024-04-26T21:03:32.894 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:33.093 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629397 2024-04-26T21:03:33.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:33.108 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:33.108 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:33.108 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:33.108 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:33.109 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/5-workload/snaps-few-objects.yaml").st_ino) 2024-04-26T21:03:33.109 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:33.305 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629398 2024-04-26T21:03:33.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:33.320 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:33.320 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:33.320 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:33.320 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:33.320 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/5-workload/rbd-cls.yaml").st_ino) 2024-04-26T21:03:33.320 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:33.516 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629395 2024-04-26T21:03:33.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:33.531 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:33.531 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:33.531 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:33.531 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:33.531 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/5-workload/rbd-import-export.yaml").st_ino) 2024-04-26T21:03:33.531 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:33.730 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629396 2024-04-26T21:03:33.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:33.745 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:33.745 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:33.745 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:33.746 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:33.746 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/5-workload/+").st_ino) 2024-04-26T21:03:33.746 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:33.945 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629394 2024-04-26T21:03:33.949 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:33.960 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:33.961 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:33.961 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:33.961 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:33.961 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/4-mon").st_ino) 2024-04-26T21:03:33.961 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:34.158 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629391 2024-04-26T21:03:34.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:34.172 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:34.172 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:34.172 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:34.173 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:34.173 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/4-mon/mona.yaml").st_ino) 2024-04-26T21:03:34.173 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:34.381 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629392 2024-04-26T21:03:34.385 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:34.396 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:34.396 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:34.397 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:34.397 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:34.397 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/%").st_ino) 2024-04-26T21:03:34.397 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:34.585 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629380 2024-04-26T21:03:34.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:34.599 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:34.599 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:34.599 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:34.599 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:34.599 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/9-workload").st_ino) 2024-04-26T21:03:34.599 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:34.794 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629407 2024-04-26T21:03:34.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:34.808 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:34.809 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:34.809 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:34.809 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:34.809 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/9-workload/rbd-python.yaml").st_ino) 2024-04-26T21:03:34.809 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:35.005 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629409 2024-04-26T21:03:35.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:35.018 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:35.019 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:35.019 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:35.019 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:35.019 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/9-workload/rgw-swift.yaml").st_ino) 2024-04-26T21:03:35.019 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:35.214 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629410 2024-04-26T21:03:35.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:35.229 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:35.229 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:35.229 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:35.229 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:35.229 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/9-workload/snaps-many-objects.yaml").st_ino) 2024-04-26T21:03:35.230 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:35.428 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629411 2024-04-26T21:03:35.433 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:35.443 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:35.443 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:35.444 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:35.444 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:35.444 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/9-workload/+").st_ino) 2024-04-26T21:03:35.444 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:35.652 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629408 2024-04-26T21:03:35.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:35.667 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:35.667 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:35.667 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:35.667 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:35.667 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/2-partial-upgrade").st_ino) 2024-04-26T21:03:35.667 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:35.864 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629387 2024-04-26T21:03:35.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:35.878 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:35.878 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:35.878 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:35.878 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:35.878 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/2-partial-upgrade/firsthalf.yaml").st_ino) 2024-04-26T21:03:35.879 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:36.065 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629388 2024-04-26T21:03:36.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:36.079 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:36.079 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:36.079 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:36.079 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:36.079 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/3-thrash").st_ino) 2024-04-26T21:03:36.079 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:36.279 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629389 2024-04-26T21:03:36.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:36.294 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:36.294 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:36.295 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:36.295 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:36.295 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/3-thrash/default.yaml").st_ino) 2024-04-26T21:03:36.295 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:36.494 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629390 2024-04-26T21:03:36.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:36.510 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:36.510 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:36.510 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:36.510 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:36.510 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/7-workload").st_ino) 2024-04-26T21:03:36.510 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:36.706 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629401 2024-04-26T21:03:36.711 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:36.721 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:36.721 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:36.722 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:36.722 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:36.722 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/7-workload/radosbench.yaml").st_ino) 2024-04-26T21:03:36.722 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:36.917 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629403 2024-04-26T21:03:36.922 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:36.932 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:36.932 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:36.932 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:36.932 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:36.932 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/7-workload/rbd_api.yaml").st_ino) 2024-04-26T21:03:36.932 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:37.134 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629404 2024-04-26T21:03:37.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:37.149 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:37.149 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:37.149 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:37.149 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:37.149 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/7-workload/+").st_ino) 2024-04-26T21:03:37.149 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:37.339 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629402 2024-04-26T21:03:37.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:37.355 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:37.355 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:37.355 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:37.355 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:37.356 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/0-cluster").st_ino) 2024-04-26T21:03:37.356 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:37.550 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629381 2024-04-26T21:03:37.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:37.565 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:37.565 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:37.565 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:37.565 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:37.565 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/0-cluster/openstack.yaml").st_ino) 2024-04-26T21:03:37.565 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:37.764 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629383 2024-04-26T21:03:37.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:37.779 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:37.780 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:37.780 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:37.780 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:37.780 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/0-cluster/+").st_ino) 2024-04-26T21:03:37.780 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:37.980 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629382 2024-04-26T21:03:37.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:37.994 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:37.994 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:37.994 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:37.994 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:37.994 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/upgrade/jewel-x-singleton/0-cluster/start.yaml").st_ino) 2024-04-26T21:03:37.994 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:38.190 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629384 2024-04-26T21:03:38.194 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:38.204 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:38.204 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:38.205 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:38.205 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:38.205 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash").st_ino) 2024-04-26T21:03:38.205 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:38.407 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629153 2024-04-26T21:03:38.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:38.422 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:38.422 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:38.422 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:38.422 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:38.422 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/msgr").st_ino) 2024-04-26T21:03:38.423 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:38.619 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629163 2024-04-26T21:03:38.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:38.634 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:38.634 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:38.634 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:38.634 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:38.634 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/clusters").st_ino) 2024-04-26T21:03:38.634 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:38.831 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629157 2024-04-26T21:03:38.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:38.847 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:38.847 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:38.847 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:38.847 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:38.847 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/clusters/3-mons.yaml").st_ino) 2024-04-26T21:03:38.848 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:39.043 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629158 2024-04-26T21:03:39.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:39.060 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:39.060 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:39.060 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:39.060 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:39.060 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/clusters/9-mons.yaml").st_ino) 2024-04-26T21:03:39.060 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:39.254 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629159 2024-04-26T21:03:39.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:39.269 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:39.269 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:39.269 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:39.269 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:39.269 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/msgr-failures").st_ino) 2024-04-26T21:03:39.269 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:39.465 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629164 2024-04-26T21:03:39.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:39.480 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:39.481 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:39.481 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:39.481 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:39.481 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/msgr-failures/mon-delay.yaml").st_ino) 2024-04-26T21:03:39.481 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:39.679 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629166 2024-04-26T21:03:39.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:39.693 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:39.693 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:39.693 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:39.694 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:39.694 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/msgr-failures/few.yaml").st_ino) 2024-04-26T21:03:39.694 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:39.892 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629165 2024-04-26T21:03:39.896 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:39.906 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:39.906 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:39.906 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:39.906 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:39.906 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/objectstore").st_ino) 2024-04-26T21:03:39.907 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:40.106 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629167 2024-04-26T21:03:40.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:40.121 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:40.121 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:40.121 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:40.121 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:40.121 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/fs").st_ino) 2024-04-26T21:03:40.121 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:40.317 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629160 2024-04-26T21:03:40.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:40.333 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:40.333 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:40.333 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:40.333 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:40.333 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/fs/xfs.yaml").st_ino) 2024-04-26T21:03:40.333 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:40.530 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629161 2024-04-26T21:03:40.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:40.544 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:40.544 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:40.544 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:40.545 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:40.545 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/workloads").st_ino) 2024-04-26T21:03:40.545 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:40.741 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629175 2024-04-26T21:03:40.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:40.755 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:40.755 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:40.755 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:40.756 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:40.756 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/workloads/pool-create-delete.yaml").st_ino) 2024-04-26T21:03:40.756 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:40.953 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629176 2024-04-26T21:03:40.958 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:40.968 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:40.968 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:40.968 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:40.968 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:40.968 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/workloads/snaps-few-objects.yaml").st_ino) 2024-04-26T21:03:40.968 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:41.164 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629180 2024-04-26T21:03:41.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:41.179 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:41.180 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:41.180 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:41.180 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:41.180 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/workloads/rados_mon_workunits.yaml").st_ino) 2024-04-26T21:03:41.180 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:41.384 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629179 2024-04-26T21:03:41.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:41.399 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:41.399 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:41.399 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:41.399 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:41.399 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/workloads/rados_5925.yaml").st_ino) 2024-04-26T21:03:41.399 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:41.596 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629177 2024-04-26T21:03:41.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:41.612 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:41.612 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:41.612 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:41.612 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:41.612 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/workloads/rados_api_tests.yaml").st_ino) 2024-04-26T21:03:41.612 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:41.810 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629178 2024-04-26T21:03:41.815 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:41.825 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:41.825 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:41.825 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:41.825 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:41.825 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/ceph").st_ino) 2024-04-26T21:03:41.825 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:42.023 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629155 2024-04-26T21:03:42.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:42.037 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:42.038 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:42.038 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:42.038 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:42.038 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/ceph/ceph.yaml").st_ino) 2024-04-26T21:03:42.038 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:42.235 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629156 2024-04-26T21:03:42.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:42.250 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:42.250 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:42.250 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:42.250 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:42.251 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/%").st_ino) 2024-04-26T21:03:42.251 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:42.448 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629154 2024-04-26T21:03:42.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:42.463 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:42.463 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:42.463 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:42.463 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:42.463 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/mon_kv_backend").st_ino) 2024-04-26T21:03:42.463 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:42.659 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629162 2024-04-26T21:03:42.663 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:42.673 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:42.673 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:42.673 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:42.674 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:42.674 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/thrashers").st_ino) 2024-04-26T21:03:42.674 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:42.862 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629169 2024-04-26T21:03:42.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:42.878 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:42.878 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:42.878 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:42.878 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:42.878 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/thrashers/many.yaml").st_ino) 2024-04-26T21:03:42.878 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:43.074 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629171 2024-04-26T21:03:43.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:43.089 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:43.089 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:43.089 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:43.089 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:43.090 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/thrashers/one.yaml").st_ino) 2024-04-26T21:03:43.090 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:43.286 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629172 2024-04-26T21:03:43.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:43.301 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:43.301 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:43.301 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:43.301 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:43.301 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/thrashers/sync.yaml").st_ino) 2024-04-26T21:03:43.302 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:43.499 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629174 2024-04-26T21:03:43.506 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:43.515 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:43.515 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:43.515 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:43.515 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:43.515 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/thrashers/sync-many.yaml").st_ino) 2024-04-26T21:03:43.515 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:43.717 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629173 2024-04-26T21:03:43.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:43.732 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:43.732 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:43.732 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:43.732 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:43.732 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/thrashers/force-sync-many.yaml").st_ino) 2024-04-26T21:03:43.732 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:43.924 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629170 2024-04-26T21:03:43.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:43.939 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:43.939 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:43.939 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:43.939 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:43.939 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/monthrash/rados.yaml").st_ino) 2024-04-26T21:03:43.939 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:44.138 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629168 2024-04-26T21:03:44.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:44.153 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:44.153 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:44.154 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:44.154 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:44.154 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash").st_ino) 2024-04-26T21:03:44.154 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:44.349 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629334 2024-04-26T21:03:44.353 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:44.364 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:44.364 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:44.364 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:44.364 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:44.364 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/msgr").st_ino) 2024-04-26T21:03:44.365 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:44.561 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629351 2024-04-26T21:03:44.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:44.577 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:44.578 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:44.578 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:44.578 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:44.578 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/clusters").st_ino) 2024-04-26T21:03:44.578 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:44.765 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629343 2024-04-26T21:03:44.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:44.781 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:44.781 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:44.781 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:44.781 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:44.781 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/clusters/openstack.yaml").st_ino) 2024-04-26T21:03:44.781 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:44.978 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629346 2024-04-26T21:03:44.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:44.994 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:44.994 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:44.994 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:44.994 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:44.994 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/clusters/fixed-2.yaml").st_ino) 2024-04-26T21:03:44.994 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:45.191 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629345 2024-04-26T21:03:45.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:45.205 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:45.205 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:45.205 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:45.205 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:45.205 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/clusters/+").st_ino) 2024-04-26T21:03:45.205 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:45.403 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629344 2024-04-26T21:03:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:45.419 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:45.419 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:45.419 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:45.419 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:45.419 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/1-pg-log-overrides").st_ino) 2024-04-26T21:03:45.419 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:45.607 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629340 2024-04-26T21:03:45.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:45.621 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:45.621 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:45.621 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:45.622 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:45.622 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/1-pg-log-overrides/normal_pg_log.yaml").st_ino) 2024-04-26T21:03:45.622 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:45.817 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629341 2024-04-26T21:03:45.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:45.833 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:45.833 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:45.833 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:45.833 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:45.833 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/1-pg-log-overrides/short_pg_log.yaml").st_ino) 2024-04-26T21:03:45.833 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:46.030 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629342 2024-04-26T21:03:46.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:46.044 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:46.044 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:46.044 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:46.044 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:46.044 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/msgr-failures").st_ino) 2024-04-26T21:03:46.044 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:46.245 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629352 2024-04-26T21:03:46.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:46.260 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:46.260 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:46.261 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:46.261 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:46.261 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/msgr-failures/few.yaml").st_ino) 2024-04-26T21:03:46.261 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:46.456 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629354 2024-04-26T21:03:46.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:46.471 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:46.471 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:46.472 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:46.472 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:46.472 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/msgr-failures/fastclose.yaml").st_ino) 2024-04-26T21:03:46.472 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:46.670 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629353 2024-04-26T21:03:46.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:46.685 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:46.685 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:46.685 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:46.685 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:46.685 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/msgr-failures/osd-delay.yaml").st_ino) 2024-04-26T21:03:46.685 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:46.880 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629355 2024-04-26T21:03:46.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:46.895 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:46.895 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:46.895 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:46.895 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:46.895 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/0-size-min-size-overrides").st_ino) 2024-04-26T21:03:46.895 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:47.092 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629336 2024-04-26T21:03:47.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:47.106 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:47.106 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:47.107 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:47.107 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:47.107 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/0-size-min-size-overrides/3-size-2-min-size.yaml").st_ino) 2024-04-26T21:03:47.107 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:47.307 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629339 2024-04-26T21:03:47.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:47.322 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:47.323 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:47.323 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:47.323 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:47.323 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/0-size-min-size-overrides/2-size-2-min-size.yaml").st_ino) 2024-04-26T21:03:47.323 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:47.418 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:03:47.424 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:03:47.518 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629338 2024-04-26T21:03:47.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:47.534 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:47.534 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:47.534 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:47.535 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:47.535 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/0-size-min-size-overrides/2-size-1-min-size.yaml").st_ino) 2024-04-26T21:03:47.535 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:47.729 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629337 2024-04-26T21:03:47.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:47.745 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:47.745 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:47.745 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:47.745 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:47.745 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/objectstore").st_ino) 2024-04-26T21:03:47.745 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:47.941 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629356 2024-04-26T21:03:47.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:47.957 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:47.957 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:47.957 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:47.957 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:47.957 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/fs").st_ino) 2024-04-26T21:03:47.957 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:48.162 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629347 2024-04-26T21:03:48.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:48.177 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:48.177 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:48.177 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:48.177 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:48.177 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/fs/btrfs.yaml").st_ino) 2024-04-26T21:03:48.177 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:48.377 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629348 2024-04-26T21:03:48.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:48.392 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:48.392 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:48.392 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:48.392 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:48.392 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/fs/xfs.yaml").st_ino) 2024-04-26T21:03:48.392 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:48.590 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629349 2024-04-26T21:03:48.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:48.606 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:48.606 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:48.606 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:48.606 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:48.607 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/workloads").st_ino) 2024-04-26T21:03:48.607 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:48.804 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629364 2024-04-26T21:03:48.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:48.819 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:48.819 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:48.819 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:48.819 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:48.819 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/workloads/radosbench.yaml").st_ino) 2024-04-26T21:03:48.819 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:49.025 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629374 2024-04-26T21:03:49.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:49.041 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:49.041 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:49.041 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:49.041 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:49.041 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/workloads/cache-agent-big.yaml").st_ino) 2024-04-26T21:03:49.041 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:49.239 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629366 2024-04-26T21:03:49.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:49.254 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:49.254 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:49.254 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:49.254 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:49.254 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/workloads/cache.yaml").st_ino) 2024-04-26T21:03:49.254 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:49.457 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629371 2024-04-26T21:03:49.462 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:49.472 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:49.472 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:49.472 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:49.472 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:49.472 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/workloads/cache-agent-small.yaml").st_ino) 2024-04-26T21:03:49.472 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:49.669 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629367 2024-04-26T21:03:49.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:49.685 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:49.685 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:49.685 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:49.685 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:49.685 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/workloads/snaps-few-objects.yaml").st_ino) 2024-04-26T21:03:49.685 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:49.882 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629376 2024-04-26T21:03:49.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:49.897 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:49.898 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:49.898 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:49.898 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:49.898 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/workloads/write_fadvise_dontneed.yaml").st_ino) 2024-04-26T21:03:49.898 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:50.081 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629377 2024-04-26T21:03:50.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:50.097 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:50.097 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:50.097 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:50.097 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:50.097 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/workloads/cache-pool-snaps.yaml").st_ino) 2024-04-26T21:03:50.097 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:50.295 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629369 2024-04-26T21:03:50.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:50.310 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:50.310 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:50.310 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:50.310 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:50.310 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/workloads/admin_socket_objecter_requests.yaml").st_ino) 2024-04-26T21:03:50.310 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:50.506 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629365 2024-04-26T21:03:50.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:50.521 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:50.521 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:50.521 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:50.521 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:50.522 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/workloads/cache-pool-snaps-readproxy.yaml").st_ino) 2024-04-26T21:03:50.522 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:50.710 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629368 2024-04-26T21:03:50.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:50.725 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:50.725 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:50.725 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:50.725 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:50.726 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/workloads/cache-snaps.yaml").st_ino) 2024-04-26T21:03:50.726 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:50.927 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629370 2024-04-26T21:03:50.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:50.942 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:50.942 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:50.942 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:50.942 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:50.942 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/workloads/pool-snaps-few-objects.yaml").st_ino) 2024-04-26T21:03:50.942 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:51.123 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629372 2024-04-26T21:03:51.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:51.138 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:51.138 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:51.138 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:51.138 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:51.138 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/workloads/rados_api_tests.yaml").st_ino) 2024-04-26T21:03:51.138 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:51.336 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629373 2024-04-26T21:03:51.341 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:51.352 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:51.352 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:51.353 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:51.353 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:51.353 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/workloads/small-objects.yaml").st_ino) 2024-04-26T21:03:51.353 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:51.548 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629375 2024-04-26T21:03:51.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:51.563 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:51.563 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:51.564 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:51.564 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:51.564 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/hobj-sort.yaml").st_ino) 2024-04-26T21:03:51.564 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:51.763 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629350 2024-04-26T21:03:51.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:51.777 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:51.778 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:51.778 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:51.778 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:51.778 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/%").st_ino) 2024-04-26T21:03:51.778 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:51.972 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629335 2024-04-26T21:03:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:51.988 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:51.988 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:51.988 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:51.988 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:51.988 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/thrashers").st_ino) 2024-04-26T21:03:51.988 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:52.181 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629359 2024-04-26T21:03:52.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:52.196 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:52.197 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:52.197 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:52.197 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:52.197 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/thrashers/default.yaml").st_ino) 2024-04-26T21:03:52.197 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:52.395 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629360 2024-04-26T21:03:52.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:52.410 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:52.410 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:52.410 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:52.410 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:52.410 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/thrashers/morepggrow.yaml").st_ino) 2024-04-26T21:03:52.410 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:52.608 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629362 2024-04-26T21:03:52.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:52.624 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:52.625 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:52.625 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:52.625 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:52.625 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/thrashers/pggrow.yaml").st_ino) 2024-04-26T21:03:52.625 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:52.823 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629363 2024-04-26T21:03:52.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:52.838 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:52.838 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:52.838 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:52.838 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:52.839 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/thrashers/mapgap.yaml").st_ino) 2024-04-26T21:03:52.839 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:53.034 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629361 2024-04-26T21:03:53.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:53.050 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:53.050 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:53.050 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:53.050 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:53.050 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/rocksdb.yaml").st_ino) 2024-04-26T21:03:53.050 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:53.248 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629358 2024-04-26T21:03:53.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:53.264 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:53.264 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:53.264 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:53.264 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:53.264 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash/rados.yaml").st_ino) 2024-04-26T21:03:53.264 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:53.464 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629357 2024-04-26T21:03:53.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:53.480 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:53.480 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:53.480 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:53.480 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:53.480 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify").st_ino) 2024-04-26T21:03:53.480 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:53.678 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629412 2024-04-26T21:03:53.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:53.693 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:53.693 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:53.693 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:53.694 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:53.694 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/msgr").st_ino) 2024-04-26T21:03:53.694 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:53.890 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629424 2024-04-26T21:03:53.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:53.905 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:53.905 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:53.905 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:53.905 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:53.905 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/validater").st_ino) 2024-04-26T21:03:53.905 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:54.101 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629433 2024-04-26T21:03:54.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:54.117 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:54.117 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:54.117 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:54.117 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:54.117 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/validater/valgrind.yaml").st_ino) 2024-04-26T21:03:54.117 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:54.298 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629435 2024-04-26T21:03:54.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:54.312 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:54.313 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:54.313 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:54.313 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:54.313 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/validater/lockdep.yaml").st_ino) 2024-04-26T21:03:54.313 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:54.511 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629434 2024-04-26T21:03:54.516 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:54.527 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:54.527 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:54.527 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:54.527 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:54.527 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/clusters").st_ino) 2024-04-26T21:03:54.527 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:54.719 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629417 2024-04-26T21:03:54.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:54.733 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:54.733 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:54.733 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:54.733 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:54.733 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/clusters/openstack.yaml").st_ino) 2024-04-26T21:03:54.734 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:54.929 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629420 2024-04-26T21:03:54.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:54.943 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:54.943 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:54.943 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:54.943 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:54.944 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/clusters/fixed-2.yaml").st_ino) 2024-04-26T21:03:54.944 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:55.143 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629419 2024-04-26T21:03:55.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:55.158 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:55.158 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:55.158 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:55.158 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:55.158 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/clusters/+").st_ino) 2024-04-26T21:03:55.158 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:55.355 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629418 2024-04-26T21:03:55.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:55.372 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:55.372 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:55.372 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:55.372 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:55.372 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/msgr-failures").st_ino) 2024-04-26T21:03:55.372 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:55.570 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629425 2024-04-26T21:03:55.575 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:55.585 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:55.585 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:55.585 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:55.585 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:55.585 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/msgr-failures/few.yaml").st_ino) 2024-04-26T21:03:55.585 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:55.784 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629426 2024-04-26T21:03:55.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:55.798 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:55.799 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:55.799 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:55.799 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:55.799 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/objectstore").st_ino) 2024-04-26T21:03:55.799 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:55.996 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629427 2024-04-26T21:03:56.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:56.011 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:56.011 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:56.011 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:56.011 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:56.011 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/fs").st_ino) 2024-04-26T21:03:56.011 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:56.195 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629421 2024-04-26T21:03:56.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:56.209 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:56.209 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:56.209 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:56.209 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:56.209 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/fs/btrfs.yaml").st_ino) 2024-04-26T21:03:56.209 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:56.397 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629422 2024-04-26T21:03:56.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:56.412 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:56.412 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:56.412 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:56.412 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:56.412 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/1thrash").st_ino) 2024-04-26T21:03:56.412 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:56.606 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629414 2024-04-26T21:03:56.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:56.621 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:56.621 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:56.621 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:56.621 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:56.621 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/1thrash/default.yaml").st_ino) 2024-04-26T21:03:56.621 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:56.821 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629415 2024-04-26T21:03:56.826 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:56.836 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:56.836 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:56.837 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:56.837 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:56.837 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/1thrash/none.yaml").st_ino) 2024-04-26T21:03:56.837 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:57.034 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629416 2024-04-26T21:03:57.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:57.049 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:57.049 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:57.049 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:57.049 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:57.049 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/%").st_ino) 2024-04-26T21:03:57.049 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:57.246 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629413 2024-04-26T21:03:57.251 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:57.262 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:57.262 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:57.262 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:57.262 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:57.262 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/mon_kv_backend").st_ino) 2024-04-26T21:03:57.262 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:57.440 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629423 2024-04-26T21:03:57.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:57.454 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:57.455 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:57.455 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:57.455 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:57.455 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/tasks").st_ino) 2024-04-26T21:03:57.455 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:57.652 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629429 2024-04-26T21:03:57.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:57.670 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:57.670 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:57.670 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:57.670 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:57.670 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/tasks/rados_cls_all.yaml").st_ino) 2024-04-26T21:03:57.670 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:57.857 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629432 2024-04-26T21:03:57.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:57.871 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:57.873 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:57.873 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:57.873 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:57.874 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/tasks/mon_recovery.yaml").st_ino) 2024-04-26T21:03:57.874 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:58.061 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629430 2024-04-26T21:03:58.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:58.076 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:58.077 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:58.077 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:58.077 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:58.077 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/tasks/rados_api_tests.yaml").st_ino) 2024-04-26T21:03:58.077 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:58.279 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629431 2024-04-26T21:03:58.285 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:58.294 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:58.295 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:58.295 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:58.295 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:58.295 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/verify/rados.yaml").st_ino) 2024-04-26T21:03:58.295 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:58.494 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629428 2024-04-26T21:03:58.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:58.509 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:58.509 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:58.509 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:58.509 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:58.509 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/multimon").st_ino) 2024-04-26T21:03:58.509 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:58.705 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629181 2024-04-26T21:03:58.710 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:58.721 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:58.721 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:58.721 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:58.721 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:58.721 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/multimon/msgr").st_ino) 2024-04-26T21:03:58.721 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:58.919 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629191 2024-04-26T21:03:58.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:58.935 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:58.935 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:58.935 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:58.935 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:58.935 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/multimon/clusters").st_ino) 2024-04-26T21:03:58.935 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:59.127 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629183 2024-04-26T21:03:59.131 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:59.141 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:59.142 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:59.142 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:59.142 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:59.142 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/multimon/clusters/21.yaml").st_ino) 2024-04-26T21:03:59.142 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:59.334 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629184 2024-04-26T21:03:59.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:59.350 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:59.350 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:59.350 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:59.350 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:59.350 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/multimon/clusters/9.yaml").st_ino) 2024-04-26T21:03:59.350 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:59.550 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629187 2024-04-26T21:03:59.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:59.565 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:59.566 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:59.566 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:59.566 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:59.566 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/multimon/clusters/3.yaml").st_ino) 2024-04-26T21:03:59.566 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:59.764 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629185 2024-04-26T21:03:59.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:59.780 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:59.780 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:59.780 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:59.780 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:59.780 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/multimon/clusters/6.yaml").st_ino) 2024-04-26T21:03:59.780 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:03:59.978 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629186 2024-04-26T21:03:59.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:03:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:03:59.993 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:03:59.993 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:03:59.993 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:03:59.993 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:03:59.993 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/multimon/msgr-failures").st_ino) 2024-04-26T21:03:59.993 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:00.189 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629192 2024-04-26T21:04:00.193 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:00.203 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:00.203 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:00.203 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:00.203 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:00.203 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/multimon/msgr-failures/many.yaml").st_ino) 2024-04-26T21:04:00.203 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:00.403 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629194 2024-04-26T21:04:00.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:00.417 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:00.417 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:00.417 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:00.418 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:00.418 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/multimon/msgr-failures/few.yaml").st_ino) 2024-04-26T21:04:00.418 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:00.616 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629193 2024-04-26T21:04:00.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:00.630 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:00.631 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:00.631 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:00.631 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:00.631 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/multimon/objectstore").st_ino) 2024-04-26T21:04:00.631 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:00.827 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629195 2024-04-26T21:04:00.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:00.842 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:00.842 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:00.842 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:00.842 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:00.842 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/multimon/fs").st_ino) 2024-04-26T21:04:00.842 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:01.037 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629188 2024-04-26T21:04:01.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:01.051 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:01.051 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:01.051 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:01.051 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:01.051 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/multimon/fs/xfs.yaml").st_ino) 2024-04-26T21:04:01.051 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:01.247 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629189 2024-04-26T21:04:01.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:01.261 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:01.261 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:01.261 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:01.261 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:01.261 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/multimon/%").st_ino) 2024-04-26T21:04:01.261 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:01.460 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629182 2024-04-26T21:04:01.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:01.475 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:01.475 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:01.475 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:01.475 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:01.475 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/multimon/mon_kv_backend").st_ino) 2024-04-26T21:04:01.475 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:01.670 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629190 2024-04-26T21:04:01.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:01.685 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:01.685 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:01.685 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:01.685 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:01.685 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/multimon/tasks").st_ino) 2024-04-26T21:04:01.686 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:01.886 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629197 2024-04-26T21:04:01.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:01.901 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:01.901 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:01.901 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:01.901 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:01.901 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/multimon/tasks/mon_clock_with_skews.yaml").st_ino) 2024-04-26T21:04:01.901 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:02.095 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629199 2024-04-26T21:04:02.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:02.110 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:02.110 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:02.111 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:02.111 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:02.111 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/multimon/tasks/mon_recovery.yaml").st_ino) 2024-04-26T21:04:02.111 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:02.302 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629200 2024-04-26T21:04:02.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:02.316 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:02.316 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:02.316 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:02.316 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:02.317 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/multimon/tasks/mon_clock_no_skews.yaml").st_ino) 2024-04-26T21:04:02.317 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:02.515 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629198 2024-04-26T21:04:02.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:02.530 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:02.530 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:02.530 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:02.530 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:02.531 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/multimon/rados.yaml").st_ino) 2024-04-26T21:04:02.531 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:02.727 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629196 2024-04-26T21:04:02.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:02.743 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:02.743 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:02.743 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:02.743 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:02.743 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-isa").st_ino) 2024-04-26T21:04:02.744 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:02.941 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629277 2024-04-26T21:04:02.946 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:02.956 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:02.956 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:02.956 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:02.956 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:02.956 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/leveldb.yaml").st_ino) 2024-04-26T21:04:02.956 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:03.154 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629283 2024-04-26T21:04:03.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:03.169 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:03.169 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:03.169 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:03.169 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:03.169 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/clusters").st_ino) 2024-04-26T21:04:03.169 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:03.372 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629281 2024-04-26T21:04:03.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:03.388 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:03.388 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:03.388 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:03.388 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:03.388 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/msgr-failures").st_ino) 2024-04-26T21:04:03.388 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:03.583 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629284 2024-04-26T21:04:03.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:03.598 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:03.598 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:03.598 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:03.598 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:03.598 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/objectstore").st_ino) 2024-04-26T21:04:03.598 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:03.785 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629285 2024-04-26T21:04:03.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:03.801 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:03.801 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:03.801 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:03.801 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:03.801 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/supported").st_ino) 2024-04-26T21:04:03.801 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:04.001 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629287 2024-04-26T21:04:04.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:04.015 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:04.016 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:04.016 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:04.016 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:04.016 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/fs").st_ino) 2024-04-26T21:04:04.016 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:04.212 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629282 2024-04-26T21:04:04.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:04.226 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:04.226 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:04.226 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:04.226 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:04.227 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/workloads").st_ino) 2024-04-26T21:04:04.227 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:04.423 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629289 2024-04-26T21:04:04.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:04.438 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:04.438 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:04.439 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:04.439 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:04.439 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/workloads/ec-rados-plugin=isa-k=2-m=1.yaml").st_ino) 2024-04-26T21:04:04.439 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:04.637 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629290 2024-04-26T21:04:04.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:04.651 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:04.651 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:04.651 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:04.652 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:04.652 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/arch").st_ino) 2024-04-26T21:04:04.652 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:04.853 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629279 2024-04-26T21:04:04.857 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:04.868 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:04.868 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:04.868 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:04.868 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:04.868 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/arch/x86_64.yaml").st_ino) 2024-04-26T21:04:04.868 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:05.064 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629280 2024-04-26T21:04:05.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:05.079 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:05.079 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:05.079 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:05.079 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:05.079 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/%").st_ino) 2024-04-26T21:04:05.079 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:05.274 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629278 2024-04-26T21:04:05.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:05.289 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:05.289 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:05.289 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:05.289 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:05.289 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/thrashers").st_ino) 2024-04-26T21:04:05.289 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:05.474 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629288 2024-04-26T21:04:05.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:05.489 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:05.489 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:05.489 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:05.489 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:05.489 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-isa/rados.yaml").st_ino) 2024-04-26T21:04:05.489 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:05.684 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629286 2024-04-26T21:04:05.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:05.698 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:05.698 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:05.698 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:05.699 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:05.699 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-big").st_ino) 2024-04-26T21:04:05.699 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:05.893 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629258 2024-04-26T21:04:05.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:05.908 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:05.908 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:05.908 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:05.908 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:05.909 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-big/cluster").st_ino) 2024-04-26T21:04:05.909 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:06.104 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629260 2024-04-26T21:04:06.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:06.119 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:06.119 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:06.120 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:06.120 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:06.120 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-big/cluster/openstack.yaml").st_ino) 2024-04-26T21:04:06.120 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:06.317 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629263 2024-04-26T21:04:06.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:06.331 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:06.331 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:06.331 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:06.331 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:06.331 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-big/cluster/12-osds.yaml").st_ino) 2024-04-26T21:04:06.331 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:06.520 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629262 2024-04-26T21:04:06.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:06.535 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:06.535 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:06.535 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:06.535 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:06.535 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-big/cluster/+").st_ino) 2024-04-26T21:04:06.535 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:06.737 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629261 2024-04-26T21:04:06.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:06.752 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:06.752 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:06.752 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:06.752 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:06.752 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-big/leveldb.yaml").st_ino) 2024-04-26T21:04:06.752 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:06.947 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629265 2024-04-26T21:04:06.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:06.962 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:06.962 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:06.962 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:06.962 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:06.963 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-big/msgr-failures").st_ino) 2024-04-26T21:04:06.963 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:07.158 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629266 2024-04-26T21:04:07.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:07.173 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:07.173 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:07.173 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:07.173 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:07.173 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-big/objectstore").st_ino) 2024-04-26T21:04:07.173 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:07.362 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629267 2024-04-26T21:04:07.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:07.377 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:07.377 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:07.377 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:07.377 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:07.377 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-big/fs").st_ino) 2024-04-26T21:04:07.377 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:07.572 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629264 2024-04-26T21:04:07.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:07.587 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:07.587 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:07.587 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:07.587 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:07.587 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-big/workloads").st_ino) 2024-04-26T21:04:07.587 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:07.775 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629275 2024-04-26T21:04:07.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:07.789 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:07.790 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:07.790 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:07.790 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:07.790 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-big/workloads/ec-rados-plugin=lrc-k=4-m=2-l=3.yaml").st_ino) 2024-04-26T21:04:07.790 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:07.987 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629276 2024-04-26T21:04:07.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:08.004 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:08.004 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:08.004 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:08.004 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:08.004 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-big/%").st_ino) 2024-04-26T21:04:08.004 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:08.191 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629259 2024-04-26T21:04:08.196 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:08.207 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:08.207 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:08.207 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:08.207 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:08.207 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-big/thrashers").st_ino) 2024-04-26T21:04:08.207 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:08.399 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629269 2024-04-26T21:04:08.404 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:08.414 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:08.415 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:08.415 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:08.415 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:08.415 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-big/thrashers/default.yaml").st_ino) 2024-04-26T21:04:08.415 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:08.616 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629270 2024-04-26T21:04:08.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:08.631 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:08.631 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:08.631 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:08.632 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:08.632 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-big/thrashers/fastread.yaml").st_ino) 2024-04-26T21:04:08.632 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:08.831 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629271 2024-04-26T21:04:08.836 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:08.846 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:08.846 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:08.847 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:08.847 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:08.847 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-big/thrashers/morepggrow.yaml").st_ino) 2024-04-26T21:04:08.847 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:09.043 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629273 2024-04-26T21:04:09.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:09.057 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:09.057 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:09.058 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:09.058 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:09.058 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-big/thrashers/pggrow.yaml").st_ino) 2024-04-26T21:04:09.058 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:09.251 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629274 2024-04-26T21:04:09.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:09.265 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:09.265 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:09.266 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:09.266 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:09.266 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-big/thrashers/mapgap.yaml").st_ino) 2024-04-26T21:04:09.266 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:09.462 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629272 2024-04-26T21:04:09.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:09.478 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:09.478 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:09.478 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:09.478 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:09.478 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/rados/thrash-erasure-code-big/rados.yaml").st_ino) 2024-04-26T21:04:09.478 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:09.673 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629268 2024-04-26T21:04:09.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:09.688 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:09.688 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:09.688 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:09.688 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:09.688 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/knfs").st_ino) 2024-04-26T21:04:09.688 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:09.881 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628895 2024-04-26T21:04:09.886 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:09.897 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:09.897 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:09.897 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:09.897 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:09.897 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/knfs/basic").st_ino) 2024-04-26T21:04:09.897 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:10.092 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628896 2024-04-26T21:04:10.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:10.107 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:10.107 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:10.107 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:10.107 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:10.107 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/knfs/basic/clusters").st_ino) 2024-04-26T21:04:10.107 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:10.302 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628900 2024-04-26T21:04:10.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:10.317 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:10.317 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:10.318 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:10.318 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:10.318 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/knfs/basic/clusters/extra-client.yaml").st_ino) 2024-04-26T21:04:10.318 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:10.513 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628901 2024-04-26T21:04:10.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:10.529 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:10.529 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:10.529 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:10.529 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:10.529 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/knfs/basic/fs").st_ino) 2024-04-26T21:04:10.529 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:10.712 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628902 2024-04-26T21:04:10.717 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:10.726 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:10.726 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:10.727 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:10.727 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:10.727 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/knfs/basic/fs/xfs.yaml").st_ino) 2024-04-26T21:04:10.727 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:10.922 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628903 2024-04-26T21:04:10.926 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:10.936 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:10.936 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:10.936 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:10.936 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:10.936 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/knfs/basic/mount").st_ino) 2024-04-26T21:04:10.936 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:11.135 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628904 2024-04-26T21:04:11.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:11.150 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:11.150 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:11.151 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:11.151 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:11.151 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/knfs/basic/mount/v4.yaml").st_ino) 2024-04-26T21:04:11.151 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:11.335 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628906 2024-04-26T21:04:11.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:11.350 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:11.350 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:11.350 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:11.350 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:11.350 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/knfs/basic/mount/v3.yaml").st_ino) 2024-04-26T21:04:11.350 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:11.546 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628905 2024-04-26T21:04:11.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:11.561 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:11.561 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:11.561 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:11.561 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:11.561 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/knfs/basic/ceph").st_ino) 2024-04-26T21:04:11.562 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:11.757 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628898 2024-04-26T21:04:11.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:11.771 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:11.771 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:11.771 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:11.771 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:11.771 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/knfs/basic/ceph/base.yaml").st_ino) 2024-04-26T21:04:11.772 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:11.972 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628899 2024-04-26T21:04:11.977 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:11.987 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:11.987 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:11.987 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:11.988 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:11.988 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/knfs/basic/%").st_ino) 2024-04-26T21:04:11.988 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:12.194 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628897 2024-04-26T21:04:12.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:12.209 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:12.209 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:12.209 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:12.209 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:12.209 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/knfs/basic/tasks").st_ino) 2024-04-26T21:04:12.209 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:12.405 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628907 2024-04-26T21:04:12.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:12.420 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:12.420 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:12.420 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:12.421 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:12.421 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/knfs/basic/tasks/nfs_workunit_suites_dbench.yaml").st_ino) 2024-04-26T21:04:12.421 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:12.619 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628911 2024-04-26T21:04:12.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:12.635 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:12.636 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:12.636 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:12.636 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:12.636 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/knfs/basic/tasks/nfs-workunit-kernel-untar-build.yaml").st_ino) 2024-04-26T21:04:12.636 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:12.832 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628908 2024-04-26T21:04:12.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:12.847 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:12.847 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:12.847 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:12.847 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:12.847 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/knfs/basic/tasks/nfs_workunit_suites_ffsb.yaml").st_ino) 2024-04-26T21:04:12.847 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:13.046 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628912 2024-04-26T21:04:13.050 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:13.060 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:13.060 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:13.060 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:13.060 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:13.061 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/knfs/basic/tasks/nfs_workunit_suites_blogbench.yaml").st_ino) 2024-04-26T21:04:13.061 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:13.258 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628910 2024-04-26T21:04:13.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:13.273 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:13.274 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:13.274 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:13.274 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:13.274 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/knfs/basic/tasks/nfs_workunit_misc.yaml").st_ino) 2024-04-26T21:04:13.274 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:13.473 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628909 2024-04-26T21:04:13.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:13.488 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:13.488 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:13.488 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:13.488 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:13.488 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/knfs/basic/tasks/nfs_workunit_suites_fsstress.yaml").st_ino) 2024-04-26T21:04:13.488 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:13.679 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628913 2024-04-26T21:04:13.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:13.693 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:13.693 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:13.693 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:13.693 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:13.693 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/knfs/basic/tasks/nfs_workunit_suites_iozone.yaml").st_ino) 2024-04-26T21:04:13.693 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:13.889 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628914 2024-04-26T21:04:13.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:13.903 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:13.903 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:13.903 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:13.903 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:13.903 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/experimental").st_ino) 2024-04-26T21:04:13.903 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:14.089 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628544 2024-04-26T21:04:14.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:14.103 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:14.103 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:14.104 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:14.104 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:14.104 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/experimental/multimds").st_ino) 2024-04-26T21:04:14.104 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:14.299 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628545 2024-04-26T21:04:14.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:14.314 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:14.314 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:14.314 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:14.314 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:14.314 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/experimental/multimds/clusters").st_ino) 2024-04-26T21:04:14.314 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:14.513 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628547 2024-04-26T21:04:14.518 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:14.528 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:14.528 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:14.528 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:14.529 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:14.529 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/experimental/multimds/clusters/7-multimds.yaml").st_ino) 2024-04-26T21:04:14.529 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:14.725 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628548 2024-04-26T21:04:14.729 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:14.739 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:14.739 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:14.740 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:14.740 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:14.740 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/experimental/multimds/%").st_ino) 2024-04-26T21:04:14.740 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:14.936 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628546 2024-04-26T21:04:14.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:14.951 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:14.951 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:14.951 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:14.951 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:14.951 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/experimental/multimds/tasks").st_ino) 2024-04-26T21:04:14.951 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:15.148 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628549 2024-04-26T21:04:15.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:15.164 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:15.164 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:15.164 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:15.164 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:15.164 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/experimental/multimds/tasks/fsstress_thrash_subtrees.yaml").st_ino) 2024-04-26T21:04:15.164 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:15.360 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628550 2024-04-26T21:04:15.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:15.374 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:15.374 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:15.374 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:15.374 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:15.374 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal").st_ino) 2024-04-26T21:04:15.374 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:15.574 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628996 2024-04-26T21:04:15.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:15.589 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:15.590 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:15.590 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:15.590 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:15.590 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/fs-misc").st_ino) 2024-04-26T21:04:15.590 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:15.784 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629006 2024-04-26T21:04:15.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:15.799 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:15.799 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:15.799 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:15.799 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:15.800 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/fs-misc/clusters").st_ino) 2024-04-26T21:04:15.800 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:15.996 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629008 2024-04-26T21:04:16.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:16.011 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:16.011 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:16.011 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:16.011 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:16.011 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/fs-misc/clusters/two_clients.yaml").st_ino) 2024-04-26T21:04:16.011 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:16.209 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629009 2024-04-26T21:04:16.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:16.224 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:16.224 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:16.224 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:16.224 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:16.224 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/fs-misc/fs").st_ino) 2024-04-26T21:04:16.224 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:16.423 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629010 2024-04-26T21:04:16.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:16.438 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:16.438 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:16.438 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:16.439 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:16.439 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/fs-misc/fs/btrfs.yaml").st_ino) 2024-04-26T21:04:16.439 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:16.635 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629011 2024-04-26T21:04:16.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:16.650 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:16.650 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:16.650 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:16.651 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:16.651 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/fs-misc/%").st_ino) 2024-04-26T21:04:16.651 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:16.846 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629007 2024-04-26T21:04:16.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:16.860 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:16.861 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:16.861 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:16.861 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:16.861 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/fs-misc/tasks").st_ino) 2024-04-26T21:04:16.861 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:17.056 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629012 2024-04-26T21:04:17.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:17.071 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:17.071 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:17.071 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:17.071 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:17.072 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/fs-misc/tasks/locktest.yaml").st_ino) 2024-04-26T21:04:17.072 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:17.271 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629013 2024-04-26T21:04:17.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:17.287 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:17.287 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:17.287 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:17.287 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:17.287 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/basic").st_ino) 2024-04-26T21:04:17.287 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:17.487 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628997 2024-04-26T21:04:17.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:17.502 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:17.502 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:17.502 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:17.502 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:17.502 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/basic/clusters").st_ino) 2024-04-26T21:04:17.502 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:17.542 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:04:17.555 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:04:17.677 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628999 2024-04-26T21:04:17.681 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:17.691 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:17.691 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:17.691 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:17.691 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:17.691 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/basic/clusters/fixed-3.yaml").st_ino) 2024-04-26T21:04:17.691 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:17.889 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629000 2024-04-26T21:04:17.894 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:17.904 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:17.905 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:17.905 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:17.905 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:17.905 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/basic/fs").st_ino) 2024-04-26T21:04:17.905 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:18.101 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629001 2024-04-26T21:04:18.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:18.116 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:18.116 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:18.116 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:18.116 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:18.116 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/basic/fs/btrfs.yaml").st_ino) 2024-04-26T21:04:18.116 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:18.311 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629002 2024-04-26T21:04:18.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:18.326 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:18.326 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:18.326 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:18.326 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:18.327 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/basic/%").st_ino) 2024-04-26T21:04:18.327 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:18.523 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628998 2024-04-26T21:04:18.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:18.537 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:18.537 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:18.538 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:18.538 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:18.538 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/basic/tasks").st_ino) 2024-04-26T21:04:18.538 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:18.732 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629003 2024-04-26T21:04:18.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:18.748 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:18.748 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:18.748 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:18.748 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:18.748 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/basic/tasks/kclient_workunit_suites_blogbench.yaml").st_ino) 2024-04-26T21:04:18.748 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:18.946 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629004 2024-04-26T21:04:18.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:18.960 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:18.960 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:18.960 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:18.960 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:18.960 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/basic/tasks/kclient_workunit_suites_fsx.yaml").st_ino) 2024-04-26T21:04:18.960 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:19.158 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629005 2024-04-26T21:04:19.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:19.173 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:19.173 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:19.173 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:19.173 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:19.173 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/mds_restart").st_ino) 2024-04-26T21:04:19.173 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:19.369 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629014 2024-04-26T21:04:19.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:19.383 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:19.383 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:19.384 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:19.384 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:19.384 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/mds_restart/clusters").st_ino) 2024-04-26T21:04:19.384 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:19.580 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629016 2024-04-26T21:04:19.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:19.596 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:19.596 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:19.596 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:19.596 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:19.596 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/mds_restart/clusters/one_mds.yaml").st_ino) 2024-04-26T21:04:19.596 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:19.797 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629017 2024-04-26T21:04:19.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:19.814 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:19.814 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:19.814 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:19.814 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:19.815 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/mds_restart/%").st_ino) 2024-04-26T21:04:19.815 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:20.013 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629015 2024-04-26T21:04:20.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:20.029 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:20.029 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:20.029 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:20.029 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:20.029 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/mds_restart/tasks").st_ino) 2024-04-26T21:04:20.029 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:20.228 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629018 2024-04-26T21:04:20.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:20.243 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:20.243 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:20.244 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:20.244 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:20.244 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/mds_restart/tasks/restart-workunit-backtraces.yaml").st_ino) 2024-04-26T21:04:20.244 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:20.441 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629019 2024-04-26T21:04:20.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:20.456 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:20.457 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:20.457 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:20.457 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:20.457 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds").st_ino) 2024-04-26T21:04:20.457 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:20.651 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629020 2024-04-26T21:04:20.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:20.666 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:20.666 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:20.666 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:20.666 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:20.666 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds/clusters").st_ino) 2024-04-26T21:04:20.667 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:20.864 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629022 2024-04-26T21:04:20.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:20.878 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:20.879 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:20.879 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:20.879 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:20.879 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds/clusters/3-node-3-mds.yaml").st_ino) 2024-04-26T21:04:20.879 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:21.076 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629023 2024-04-26T21:04:21.080 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:21.091 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:21.091 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:21.091 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:21.091 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:21.091 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds/clusters/3-node-9-mds.yaml").st_ino) 2024-04-26T21:04:21.091 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:21.290 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629024 2024-04-26T21:04:21.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:21.305 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:21.305 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:21.305 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:21.305 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:21.305 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds/fs").st_ino) 2024-04-26T21:04:21.305 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:21.502 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629025 2024-04-26T21:04:21.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:21.518 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:21.518 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:21.518 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:21.518 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:21.518 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds/fs/btrfs.yaml").st_ino) 2024-04-26T21:04:21.518 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:21.716 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629026 2024-04-26T21:04:21.721 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:21.733 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:21.733 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:21.733 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:21.733 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:21.733 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds/%").st_ino) 2024-04-26T21:04:21.733 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:21.920 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629021 2024-04-26T21:04:21.924 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:21.934 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:21.934 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:21.934 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:21.934 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:21.934 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds/mounts").st_ino) 2024-04-26T21:04:21.934 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:22.131 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629027 2024-04-26T21:04:22.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:22.146 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:22.146 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:22.146 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:22.146 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:22.146 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds/mounts/ceph-fuse.yaml").st_ino) 2024-04-26T21:04:22.146 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:22.345 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629028 2024-04-26T21:04:22.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:22.361 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:22.361 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:22.361 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:22.361 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:22.361 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds/mounts/kclient.yaml").st_ino) 2024-04-26T21:04:22.361 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:22.558 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629029 2024-04-26T21:04:22.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:22.573 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:22.573 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:22.573 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:22.573 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:22.573 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds/tasks").st_ino) 2024-04-26T21:04:22.574 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:22.760 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629030 2024-04-26T21:04:22.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:22.775 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:22.775 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:22.775 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:22.775 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:22.775 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds/tasks/workunit_suites_pjd.yaml").st_ino) 2024-04-26T21:04:22.775 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:22.971 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629036 2024-04-26T21:04:22.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:22.985 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:22.985 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:22.985 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:22.986 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:22.986 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds/tasks/workunit_suites_fsstress.yaml").st_ino) 2024-04-26T21:04:22.986 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:23.182 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629034 2024-04-26T21:04:23.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:23.197 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:23.197 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:23.197 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:23.197 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:23.197 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds/tasks/workunit_suites_truncate_delay.yaml").st_ino) 2024-04-26T21:04:23.197 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:23.394 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629037 2024-04-26T21:04:23.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:23.409 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:23.409 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:23.410 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:23.410 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:23.410 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds/tasks/workunit_misc.yaml").st_ino) 2024-04-26T21:04:23.410 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:23.607 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629031 2024-04-26T21:04:23.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:23.622 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:23.622 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:23.622 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:23.622 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:23.623 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds/tasks/workunit_suites_blogbench.yaml").st_ino) 2024-04-26T21:04:23.623 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:23.806 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629032 2024-04-26T21:04:23.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:23.821 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:23.821 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:23.821 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:23.821 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:23.821 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds/tasks/workunit_suites_fsync.yaml").st_ino) 2024-04-26T21:04:23.821 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:24.017 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629035 2024-04-26T21:04:24.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:24.032 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:24.032 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:24.032 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:24.032 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:24.032 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds/tasks/workunit_suites_dbench.yaml").st_ino) 2024-04-26T21:04:24.032 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:24.227 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629033 2024-04-26T21:04:24.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:24.241 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:24.242 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:24.242 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:24.242 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:24.242 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds/thrash").st_ino) 2024-04-26T21:04:24.242 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:24.440 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629038 2024-04-26T21:04:24.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:24.455 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:24.455 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:24.455 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:24.455 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:24.456 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds/thrash/normal.yaml").st_ino) 2024-04-26T21:04:24.456 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:24.657 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629040 2024-04-26T21:04:24.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:24.673 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:24.673 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:24.673 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:24.673 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:24.673 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/marginal/multimds/thrash/exports.yaml").st_ino) 2024-04-26T21:04:24.673 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:24.873 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629039 2024-04-26T21:04:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:24.888 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:24.888 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:24.888 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:24.888 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:24.888 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs").st_ino) 2024-04-26T21:04:24.888 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:25.085 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628819 2024-04-26T21:04:25.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:25.099 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:25.100 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:25.100 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:25.100 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:25.100 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs").st_ino) 2024-04-26T21:04:25.100 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:25.299 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628820 2024-04-26T21:04:25.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:25.316 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:25.316 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:25.316 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:25.316 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:25.316 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/clusters").st_ino) 2024-04-26T21:04:25.316 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:25.514 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628822 2024-04-26T21:04:25.519 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:25.529 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:25.529 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:25.530 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:25.530 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:25.530 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/clusters/fixed-3-cephfs.yaml").st_ino) 2024-04-26T21:04:25.530 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:25.728 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628823 2024-04-26T21:04:25.732 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:25.743 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:25.743 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:25.743 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:25.743 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:25.743 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/fs").st_ino) 2024-04-26T21:04:25.743 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:25.934 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628825 2024-04-26T21:04:25.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:25.950 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:25.950 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:25.951 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:25.951 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:25.951 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/fs/xfs.yaml").st_ino) 2024-04-26T21:04:25.951 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:26.147 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628826 2024-04-26T21:04:26.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:26.163 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:26.163 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:26.163 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:26.163 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:26.163 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/inline").st_ino) 2024-04-26T21:04:26.163 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:26.359 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628827 2024-04-26T21:04:26.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:26.374 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:26.375 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:26.375 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:26.375 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:26.375 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/inline/yes.yaml").st_ino) 2024-04-26T21:04:26.375 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:26.570 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628829 2024-04-26T21:04:26.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:26.583 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:26.584 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:26.584 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:26.584 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:26.584 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/inline/no.yaml").st_ino) 2024-04-26T21:04:26.584 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:26.782 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628828 2024-04-26T21:04:26.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:26.797 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:26.797 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:26.797 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:26.797 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:26.797 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/conf.yaml").st_ino) 2024-04-26T21:04:26.797 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:26.997 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628824 2024-04-26T21:04:27.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:27.012 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:27.012 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:27.012 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:27.012 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:27.012 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/%").st_ino) 2024-04-26T21:04:27.012 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:27.209 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628821 2024-04-26T21:04:27.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:27.224 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:27.224 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:27.224 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:27.225 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:27.225 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/tasks").st_ino) 2024-04-26T21:04:27.225 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:27.423 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628830 2024-04-26T21:04:27.428 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:27.439 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:27.439 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:27.439 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:27.439 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:27.439 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_misc.yaml").st_ino) 2024-04-26T21:04:27.439 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:27.636 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628833 2024-04-26T21:04:27.641 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:27.651 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:27.652 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:27.652 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:27.652 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:27.652 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_suites_fsstress.yaml").st_ino) 2024-04-26T21:04:27.652 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:27.842 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628838 2024-04-26T21:04:27.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:27.857 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:27.857 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:27.858 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:27.858 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:27.858 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_trivial_sync.yaml").st_ino) 2024-04-26T21:04:27.858 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:28.058 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628843 2024-04-26T21:04:28.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:28.073 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:28.073 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:28.073 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:28.073 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:28.073 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_snaps.yaml").st_ino) 2024-04-26T21:04:28.073 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:28.264 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628835 2024-04-26T21:04:28.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:28.279 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:28.280 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:28.280 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:28.280 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:28.280 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_direct_io.yaml").st_ino) 2024-04-26T21:04:28.280 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:28.488 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628831 2024-04-26T21:04:28.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:28.504 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:28.504 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:28.505 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:28.505 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:28.505 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_suites_fsx.yaml").st_ino) 2024-04-26T21:04:28.505 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:28.703 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628839 2024-04-26T21:04:28.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:28.718 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:28.718 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:28.718 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:28.718 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:28.718 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_kernel_untar_build.yaml").st_ino) 2024-04-26T21:04:28.719 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:28.909 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628832 2024-04-26T21:04:28.913 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:28.923 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:28.924 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:28.924 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:28.924 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:28.924 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_suites_ffsb.yaml").st_ino) 2024-04-26T21:04:28.924 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:29.118 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628837 2024-04-26T21:04:29.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:29.134 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:29.134 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:29.134 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:29.134 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:29.134 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_suites_dbench.yaml").st_ino) 2024-04-26T21:04:29.134 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:29.332 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628836 2024-04-26T21:04:29.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:29.346 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:29.346 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:29.347 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:29.347 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:29.347 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_suites_fsync.yaml").st_ino) 2024-04-26T21:04:29.347 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:29.544 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628840 2024-04-26T21:04:29.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:29.559 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:29.560 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:29.560 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:29.560 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:29.560 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_suites_pjd.yaml").st_ino) 2024-04-26T21:04:29.560 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:29.760 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628842 2024-04-26T21:04:29.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:29.775 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:29.775 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:29.775 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:29.775 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:29.775 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_suites_iozone.yaml").st_ino) 2024-04-26T21:04:29.775 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:29.971 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628841 2024-04-26T21:04:29.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:29.986 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:29.986 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:29.986 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:29.986 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:29.986 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/cephfs/tasks/kclient_workunit_o_trunc.yaml").st_ino) 2024-04-26T21:04:29.986 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:30.172 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628834 2024-04-26T21:04:30.176 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:30.186 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:30.187 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:30.187 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:30.187 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:30.187 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/mixed-clients").st_ino) 2024-04-26T21:04:30.187 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:30.385 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628844 2024-04-26T21:04:30.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:30.399 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:30.400 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:30.400 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:30.400 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:30.400 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/mixed-clients/clusters").st_ino) 2024-04-26T21:04:30.400 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:30.599 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628846 2024-04-26T21:04:30.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:30.614 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:30.614 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:30.614 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:30.614 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:30.614 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/mixed-clients/clusters/2-clients.yaml").st_ino) 2024-04-26T21:04:30.615 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:30.810 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628847 2024-04-26T21:04:30.814 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:30.825 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:30.825 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:30.825 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:30.825 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:30.825 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/mixed-clients/fs").st_ino) 2024-04-26T21:04:30.825 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:31.025 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628849 2024-04-26T21:04:31.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:31.040 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:31.040 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:31.040 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:31.040 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:31.040 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/mixed-clients/fs/xfs.yaml").st_ino) 2024-04-26T21:04:31.041 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:31.241 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628850 2024-04-26T21:04:31.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:31.257 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:31.257 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:31.257 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:31.257 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:31.257 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/mixed-clients/conf.yaml").st_ino) 2024-04-26T21:04:31.257 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:31.455 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628848 2024-04-26T21:04:31.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:31.470 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:31.470 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:31.470 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:31.470 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:31.470 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/mixed-clients/%").st_ino) 2024-04-26T21:04:31.470 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:31.666 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628845 2024-04-26T21:04:31.671 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:31.681 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:31.681 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:31.681 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:31.681 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:31.682 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/mixed-clients/tasks").st_ino) 2024-04-26T21:04:31.682 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:31.878 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628851 2024-04-26T21:04:31.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:31.893 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:31.893 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:31.893 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:31.893 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:31.894 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/mixed-clients/tasks/kernel_cfuse_workunits_untarbuild_blogbench.yaml").st_ino) 2024-04-26T21:04:31.894 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:32.090 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628853 2024-04-26T21:04:32.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:32.104 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:32.104 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:32.104 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:32.104 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:32.104 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/mixed-clients/tasks/kernel_cfuse_workunits_dbench_iozone.yaml").st_ino) 2024-04-26T21:04:32.105 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:32.305 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628852 2024-04-26T21:04:32.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:32.320 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:32.320 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:32.321 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:32.321 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:32.321 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery").st_ino) 2024-04-26T21:04:32.321 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:32.518 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628854 2024-04-26T21:04:32.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:32.534 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:32.535 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:32.535 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:32.535 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:32.535 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/dirfrag").st_ino) 2024-04-26T21:04:32.535 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:32.717 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628860 2024-04-26T21:04:32.722 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:32.731 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:32.732 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:32.732 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:32.732 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:32.732 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/dirfrag/frag_enable.yaml").st_ino) 2024-04-26T21:04:32.732 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:32.929 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628861 2024-04-26T21:04:32.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:32.945 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:32.945 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:32.945 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:32.946 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:32.946 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/clusters").st_ino) 2024-04-26T21:04:32.946 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:33.140 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628856 2024-04-26T21:04:33.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:33.155 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:33.155 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:33.155 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:33.156 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:33.156 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/clusters/4-remote-clients.yaml").st_ino) 2024-04-26T21:04:33.156 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:33.344 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628857 2024-04-26T21:04:33.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:33.358 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:33.359 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:33.359 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:33.359 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:33.359 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/%").st_ino) 2024-04-26T21:04:33.359 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:33.556 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628855 2024-04-26T21:04:33.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:33.572 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:33.573 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:33.573 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:33.573 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:33.573 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/mounts").st_ino) 2024-04-26T21:04:33.573 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:33.773 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628862 2024-04-26T21:04:33.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:33.788 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:33.788 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:33.788 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:33.788 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:33.788 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/mounts/kmounts.yaml").st_ino) 2024-04-26T21:04:33.789 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:33.982 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628863 2024-04-26T21:04:33.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:33.997 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:33.998 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:33.998 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:33.998 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:33.998 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/xfs.yaml").st_ino) 2024-04-26T21:04:33.998 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:34.195 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628880 2024-04-26T21:04:34.199 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:34.210 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:34.210 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:34.210 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:34.210 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:34.210 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/tasks").st_ino) 2024-04-26T21:04:34.210 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:34.405 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628864 2024-04-26T21:04:34.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:34.420 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:34.420 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:34.420 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:34.420 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:34.420 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/tasks/backtrace.yaml").st_ino) 2024-04-26T21:04:34.420 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:34.620 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628866 2024-04-26T21:04:34.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:34.636 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:34.636 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:34.636 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:34.636 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:34.636 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/tasks/forward-scrub.yaml").st_ino) 2024-04-26T21:04:34.636 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:34.834 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628873 2024-04-26T21:04:34.838 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:34.849 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:34.849 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:34.849 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:34.849 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:34.849 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/tasks/volume-client.yaml").st_ino) 2024-04-26T21:04:34.849 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:35.050 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628879 2024-04-26T21:04:35.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:35.066 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:35.066 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:35.066 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:35.066 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:35.066 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/tasks/config-commands.yaml").st_ino) 2024-04-26T21:04:35.066 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:35.264 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628869 2024-04-26T21:04:35.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:35.278 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:35.278 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:35.278 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:35.279 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:35.279 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/tasks/sessionmap.yaml").st_ino) 2024-04-26T21:04:35.279 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:35.474 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628877 2024-04-26T21:04:35.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:35.490 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:35.490 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:35.490 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:35.490 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:35.490 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/tasks/failover.yaml.disabled").st_ino) 2024-04-26T21:04:35.490 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:35.686 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628872 2024-04-26T21:04:35.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:35.701 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:35.701 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:35.701 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:35.701 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:35.701 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/tasks/data-scan.yaml").st_ino) 2024-04-26T21:04:35.701 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:35.898 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628871 2024-04-26T21:04:35.902 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:35.913 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:35.913 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:35.913 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:35.913 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:35.913 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/tasks/mds-flush.yaml").st_ino) 2024-04-26T21:04:35.913 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:36.115 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628875 2024-04-26T21:04:36.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:36.130 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:36.130 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:36.130 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:36.130 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:36.130 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/tasks/auto-repair.yaml").st_ino) 2024-04-26T21:04:36.130 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:36.326 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628865 2024-04-26T21:04:36.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:36.341 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:36.341 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:36.342 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:36.342 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:36.342 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/tasks/damage.yaml").st_ino) 2024-04-26T21:04:36.342 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:36.543 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628870 2024-04-26T21:04:36.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:36.558 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:36.558 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:36.558 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:36.558 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:36.558 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/tasks/client-recovery.yaml").st_ino) 2024-04-26T21:04:36.558 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:36.760 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628868 2024-04-26T21:04:36.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:36.774 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:36.774 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:36.775 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:36.775 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:36.775 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/tasks/journal-repair.yaml").st_ino) 2024-04-26T21:04:36.775 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:36.974 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628874 2024-04-26T21:04:36.979 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:36.988 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:36.989 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:36.989 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:36.989 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:36.989 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/tasks/client-limits.yaml").st_ino) 2024-04-26T21:04:36.989 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:37.174 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628867 2024-04-26T21:04:37.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:37.189 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:37.189 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:37.189 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:37.189 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:37.189 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/tasks/strays.yaml.disabled").st_ino) 2024-04-26T21:04:37.189 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:37.385 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628878 2024-04-26T21:04:37.389 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:37.400 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:37.401 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:37.401 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:37.401 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:37.401 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/tasks/pool-perm.yaml").st_ino) 2024-04-26T21:04:37.401 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:37.623 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628876 2024-04-26T21:04:37.629 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:37.640 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:37.640 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:37.640 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:37.640 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:37.640 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/debug").st_ino) 2024-04-26T21:04:37.640 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:37.842 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628858 2024-04-26T21:04:37.847 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:37.857 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:37.858 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:37.858 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:37.858 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:37.858 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/recovery/debug/mds_client.yaml").st_ino) 2024-04-26T21:04:37.858 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:38.051 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628859 2024-04-26T21:04:38.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:38.065 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:38.065 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:38.065 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:38.065 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:38.065 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/thrash").st_ino) 2024-04-26T21:04:38.065 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:38.249 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628881 2024-04-26T21:04:38.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:38.263 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:38.263 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:38.263 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:38.263 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:38.264 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/thrash/clusters").st_ino) 2024-04-26T21:04:38.264 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:38.461 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628883 2024-04-26T21:04:38.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:38.476 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:38.476 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:38.476 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:38.476 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:38.476 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/thrash/clusters/fixed-3-cephfs.yaml").st_ino) 2024-04-26T21:04:38.476 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:38.676 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628884 2024-04-26T21:04:38.680 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:38.691 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:38.692 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:38.692 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:38.692 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:38.692 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/thrash/fs").st_ino) 2024-04-26T21:04:38.692 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:38.890 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628886 2024-04-26T21:04:38.895 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:38.905 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:38.906 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:38.906 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:38.906 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:38.906 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/thrash/fs/xfs.yaml").st_ino) 2024-04-26T21:04:38.906 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:39.101 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628887 2024-04-26T21:04:39.106 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:39.116 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:39.116 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:39.116 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:39.116 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:39.116 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/thrash/workloads").st_ino) 2024-04-26T21:04:39.116 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:39.314 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628892 2024-04-26T21:04:39.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:39.330 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:39.330 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:39.330 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:39.330 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:39.331 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/thrash/workloads/kclient_workunit_suites_ffsb.yaml").st_ino) 2024-04-26T21:04:39.331 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:39.529 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628893 2024-04-26T21:04:39.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:39.544 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:39.545 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:39.545 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:39.545 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:39.545 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/thrash/workloads/kclient_workunit_suites_iozone.yaml").st_ino) 2024-04-26T21:04:39.545 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:39.744 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628894 2024-04-26T21:04:39.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:39.759 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:39.759 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:39.759 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:39.759 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:39.759 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/thrash/conf.yaml").st_ino) 2024-04-26T21:04:39.759 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:39.952 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628885 2024-04-26T21:04:39.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:39.967 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:39.967 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:39.967 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:39.967 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:39.967 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/thrash/%").st_ino) 2024-04-26T21:04:39.967 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:40.163 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628882 2024-04-26T21:04:40.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:40.178 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:40.178 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:40.178 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:40.178 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:40.178 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/thrash/thrashers").st_ino) 2024-04-26T21:04:40.178 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:40.374 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628888 2024-04-26T21:04:40.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:40.390 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:40.390 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:40.390 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:40.390 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:40.390 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/thrash/thrashers/default.yaml").st_ino) 2024-04-26T21:04:40.390 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:40.587 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628889 2024-04-26T21:04:40.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:40.602 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:40.602 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:40.602 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:40.602 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:40.602 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/thrash/thrashers/mon.yaml").st_ino) 2024-04-26T21:04:40.602 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:40.797 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628891 2024-04-26T21:04:40.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:40.812 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:40.813 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:40.813 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:40.813 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:40.813 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/kcephfs/thrash/thrashers/mds.yaml").st_ino) 2024-04-26T21:04:40.813 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:41.010 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628890 2024-04-26T21:04:41.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:41.025 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:41.026 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:41.026 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:41.026 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:41.026 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade").st_ino) 2024-04-26T21:04:41.026 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:41.226 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629876 2024-04-26T21:04:41.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:41.241 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:41.241 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:41.241 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:41.241 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:41.241 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x").st_ino) 2024-04-26T21:04:41.241 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:41.434 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629973 2024-04-26T21:04:41.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:41.449 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:41.449 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:41.449 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:41.449 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:41.449 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/point-to-point-x").st_ino) 2024-04-26T21:04:41.449 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:41.645 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630005 2024-04-26T21:04:41.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:41.659 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:41.659 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:41.659 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:41.659 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:41.659 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/point-to-point-x/point-to-point-upgrade.yaml").st_ino) 2024-04-26T21:04:41.660 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:41.855 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630010 2024-04-26T21:04:41.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:41.870 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:41.870 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:41.870 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:41.870 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:41.870 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/point-to-point-x/distros").st_ino) 2024-04-26T21:04:41.870 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:42.067 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630007 2024-04-26T21:04:42.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:42.082 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:42.082 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:42.082 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:42.082 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:42.082 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/point-to-point-x/distros/centos_7.2.yaml").st_ino) 2024-04-26T21:04:42.082 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:42.282 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630008 2024-04-26T21:04:42.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:42.297 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:42.297 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:42.297 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:42.297 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:42.297 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/point-to-point-x/distros/ubuntu_14.04.yaml").st_ino) 2024-04-26T21:04:42.297 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:42.493 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630009 2024-04-26T21:04:42.498 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:42.509 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:42.509 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:42.509 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:42.509 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:42.509 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/point-to-point-x/%").st_ino) 2024-04-26T21:04:42.509 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:42.704 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630006 2024-04-26T21:04:42.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:42.720 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:42.720 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:42.720 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:42.720 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:42.720 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel").st_ino) 2024-04-26T21:04:42.720 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:42.916 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629974 2024-04-26T21:04:42.921 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:42.932 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:42.932 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:42.932 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:42.932 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:42.932 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/kraken.yaml").st_ino) 2024-04-26T21:04:42.932 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:43.129 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630004 2024-04-26T21:04:43.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:43.144 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:43.144 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:43.144 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:43.144 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:43.144 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/1-jewel-install").st_ino) 2024-04-26T21:04:43.144 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:43.339 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629980 2024-04-26T21:04:43.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:43.354 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:43.355 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:43.355 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:43.355 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:43.355 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/1-jewel-install/jewel.yaml").st_ino) 2024-04-26T21:04:43.355 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:43.537 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629981 2024-04-26T21:04:43.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:43.551 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:43.551 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:43.552 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:43.552 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:43.552 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/4-kraken.yaml").st_ino) 2024-04-26T21:04:43.552 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:43.748 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629993 2024-04-26T21:04:43.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:43.763 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:43.763 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:43.763 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:43.763 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:43.763 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/3-upgrade-sequence").st_ino) 2024-04-26T21:04:43.764 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:43.960 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629990 2024-04-26T21:04:43.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:43.974 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:43.975 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:43.975 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:43.975 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:43.975 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/3-upgrade-sequence/upgrade-all.yaml").st_ino) 2024-04-26T21:04:43.975 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:44.177 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629991 2024-04-26T21:04:44.181 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:44.192 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:44.192 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:44.192 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:44.192 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:44.192 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/3-upgrade-sequence/upgrade-mon-osd-mds.yaml").st_ino) 2024-04-26T21:04:44.193 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:44.390 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629992 2024-04-26T21:04:44.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:44.405 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:44.405 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:44.405 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:44.405 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:44.406 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/distros").st_ino) 2024-04-26T21:04:44.406 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:44.603 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630003 2024-04-26T21:04:44.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:44.619 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:44.619 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:44.619 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:44.619 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:44.619 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/%").st_ino) 2024-04-26T21:04:44.619 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:44.818 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629975 2024-04-26T21:04:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:44.834 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:44.834 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:44.834 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:44.834 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:44.834 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/2-workload").st_ino) 2024-04-26T21:04:44.834 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:45.036 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629982 2024-04-26T21:04:45.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:45.051 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:45.051 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:45.051 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:45.051 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:45.051 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/2-workload/test_rbd_python.yaml").st_ino) 2024-04-26T21:04:45.051 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:45.250 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629989 2024-04-26T21:04:45.255 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:45.265 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:45.265 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:45.265 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:45.265 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:45.265 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/2-workload/blogbench.yaml").st_ino) 2024-04-26T21:04:45.265 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:45.462 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629984 2024-04-26T21:04:45.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:45.477 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:45.478 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:45.478 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:45.478 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:45.478 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/2-workload/rados_api.yaml").st_ino) 2024-04-26T21:04:45.478 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:45.674 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629986 2024-04-26T21:04:45.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:45.690 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:45.690 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:45.690 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:45.690 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:45.690 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/2-workload/rados_loadgenbig.yaml").st_ino) 2024-04-26T21:04:45.690 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:45.888 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629987 2024-04-26T21:04:45.893 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:45.904 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:45.904 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:45.904 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:45.904 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:45.904 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/2-workload/ec-rados-default.yaml").st_ino) 2024-04-26T21:04:45.904 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:46.102 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629985 2024-04-26T21:04:46.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:46.118 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:46.118 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:46.118 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:46.118 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:46.118 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/2-workload/+").st_ino) 2024-04-26T21:04:46.118 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:46.314 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629983 2024-04-26T21:04:46.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:46.330 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:46.330 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:46.330 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:46.330 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:46.330 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/2-workload/test_rbd_api.yaml").st_ino) 2024-04-26T21:04:46.330 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:46.527 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629988 2024-04-26T21:04:46.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:46.543 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:46.543 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:46.543 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:46.544 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:46.544 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/0-cluster").st_ino) 2024-04-26T21:04:46.544 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:46.741 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629976 2024-04-26T21:04:46.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:46.756 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:46.756 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:46.756 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:46.757 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:46.757 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/0-cluster/openstack.yaml").st_ino) 2024-04-26T21:04:46.757 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:46.946 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629978 2024-04-26T21:04:46.951 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:46.961 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:46.961 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:46.961 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:46.961 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:46.961 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/0-cluster/+").st_ino) 2024-04-26T21:04:46.961 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:47.164 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629977 2024-04-26T21:04:47.169 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:47.179 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:47.179 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:47.179 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:47.179 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:47.179 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/0-cluster/start.yaml").st_ino) 2024-04-26T21:04:47.179 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:47.375 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629979 2024-04-26T21:04:47.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:47.390 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:47.390 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:47.390 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:47.390 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:47.391 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/5-final-workload").st_ino) 2024-04-26T21:04:47.391 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:47.583 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629994 2024-04-26T21:04:47.587 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:47.597 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:47.597 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:47.597 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:47.597 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:47.597 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/5-final-workload/blogbench.yaml").st_ino) 2024-04-26T21:04:47.597 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:47.690 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:04:47.695 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:04:47.801 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629996 2024-04-26T21:04:47.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:47.816 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:47.816 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:47.816 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:47.816 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:47.817 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/5-final-workload/rados-snaps-few-objects.yaml").st_ino) 2024-04-26T21:04:47.817 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:48.013 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629997 2024-04-26T21:04:48.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:48.029 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:48.029 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:48.029 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:48.030 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:48.030 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/5-final-workload/+").st_ino) 2024-04-26T21:04:48.030 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:48.227 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629995 2024-04-26T21:04:48.231 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:48.241 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:48.242 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:48.242 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:48.242 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:48.242 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/5-final-workload/rados_loadgenmix.yaml").st_ino) 2024-04-26T21:04:48.242 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:48.441 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629998 2024-04-26T21:04:48.445 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:48.455 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:48.455 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:48.456 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:48.456 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:48.456 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/5-final-workload/rbd_cls.yaml").st_ino) 2024-04-26T21:04:48.456 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:48.655 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630000 2024-04-26T21:04:48.660 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:48.670 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:48.671 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:48.671 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:48.671 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:48.671 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/5-final-workload/rados_mon_thrash.yaml").st_ino) 2024-04-26T21:04:48.671 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:48.867 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629999 2024-04-26T21:04:48.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:48.882 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:48.882 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:48.883 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:48.883 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:48.883 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/5-final-workload/rgw_swift.yaml").st_ino) 2024-04-26T21:04:48.883 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:49.085 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630002 2024-04-26T21:04:49.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:49.100 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:49.100 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:49.100 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:49.100 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:49.100 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/parallel/5-final-workload/rbd_import_export.yaml").st_ino) 2024-04-26T21:04:49.100 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:49.299 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630001 2024-04-26T21:04:49.303 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:49.313 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:49.313 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:49.313 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:49.313 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:49.313 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split").st_ino) 2024-04-26T21:04:49.314 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:49.500 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630041 2024-04-26T21:04:49.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:49.514 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:49.514 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:49.514 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:49.514 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:49.514 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/6-next-mon").st_ino) 2024-04-26T21:04:49.514 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:49.715 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630061 2024-04-26T21:04:49.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:49.730 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:49.730 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:49.730 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:49.730 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:49.730 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/6-next-mon/monb.yaml").st_ino) 2024-04-26T21:04:49.731 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:49.930 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630062 2024-04-26T21:04:49.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:49.944 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:49.945 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:49.945 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:49.945 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:49.945 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/8-next-mon").st_ino) 2024-04-26T21:04:49.945 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:50.139 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630067 2024-04-26T21:04:50.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:50.155 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:50.155 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:50.155 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:50.155 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:50.155 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/8-next-mon/monc.yaml").st_ino) 2024-04-26T21:04:50.155 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:50.351 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630068 2024-04-26T21:04:50.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:50.366 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:50.366 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:50.366 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:50.366 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:50.367 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/1-jewel-install").st_ino) 2024-04-26T21:04:50.367 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:50.563 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630047 2024-04-26T21:04:50.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:50.578 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:50.578 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:50.578 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:50.578 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:50.578 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/1-jewel-install/jewel.yaml").st_ino) 2024-04-26T21:04:50.578 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:50.775 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630048 2024-04-26T21:04:50.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:50.790 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:50.790 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:50.790 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:50.790 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:50.790 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/5-workload").st_ino) 2024-04-26T21:04:50.790 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:50.985 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630055 2024-04-26T21:04:50.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:51.000 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:51.000 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:51.001 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:51.001 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:51.001 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/5-workload/readwrite.yaml").st_ino) 2024-04-26T21:04:51.001 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:51.206 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630059 2024-04-26T21:04:51.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:51.221 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:51.221 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:51.221 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:51.221 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:51.221 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/5-workload/snaps-few-objects.yaml").st_ino) 2024-04-26T21:04:51.221 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:51.418 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630060 2024-04-26T21:04:51.422 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:51.433 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:51.433 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:51.433 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:51.433 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:51.433 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/5-workload/rbd-cls.yaml").st_ino) 2024-04-26T21:04:51.433 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:51.635 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630057 2024-04-26T21:04:51.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:51.649 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:51.650 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:51.650 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:51.650 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:51.650 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/5-workload/rbd-import-export.yaml").st_ino) 2024-04-26T21:04:51.650 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:51.846 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630058 2024-04-26T21:04:51.851 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:51.861 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:51.861 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:51.861 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:51.861 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:51.862 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/5-workload/+").st_ino) 2024-04-26T21:04:51.862 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:52.059 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630056 2024-04-26T21:04:52.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:52.073 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:52.074 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:52.074 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:52.074 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:52.074 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/4-mon").st_ino) 2024-04-26T21:04:52.074 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:52.268 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630053 2024-04-26T21:04:52.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:52.282 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:52.282 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:52.282 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:52.282 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:52.282 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/4-mon/mona.yaml").st_ino) 2024-04-26T21:04:52.282 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:52.481 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630054 2024-04-26T21:04:52.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:52.495 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:52.495 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:52.495 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:52.495 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:52.495 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/distros").st_ino) 2024-04-26T21:04:52.495 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:52.692 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630074 2024-04-26T21:04:52.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:52.707 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:52.707 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:52.707 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:52.707 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:52.707 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/%").st_ino) 2024-04-26T21:04:52.707 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:52.902 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630042 2024-04-26T21:04:52.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:52.917 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:52.917 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:52.918 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:52.918 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:52.918 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/9-workload").st_ino) 2024-04-26T21:04:52.918 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:53.114 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630069 2024-04-26T21:04:53.118 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:53.129 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:53.129 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:53.129 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:53.129 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:53.129 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/9-workload/rbd-python.yaml").st_ino) 2024-04-26T21:04:53.130 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:53.325 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630071 2024-04-26T21:04:53.330 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:53.340 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:53.340 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:53.340 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:53.340 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:53.340 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/9-workload/rgw-swift.yaml").st_ino) 2024-04-26T21:04:53.340 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:53.537 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630072 2024-04-26T21:04:53.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:53.551 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:53.551 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:53.552 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:53.552 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:53.552 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/9-workload/snaps-many-objects.yaml").st_ino) 2024-04-26T21:04:53.552 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:53.750 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630073 2024-04-26T21:04:53.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:53.765 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:53.765 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:53.765 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:53.765 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:53.765 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/9-workload/+").st_ino) 2024-04-26T21:04:53.765 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:53.958 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630070 2024-04-26T21:04:53.963 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:53.973 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:53.973 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:53.973 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:53.974 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:53.974 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/2-partial-upgrade").st_ino) 2024-04-26T21:04:53.974 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:54.173 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630049 2024-04-26T21:04:54.178 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:54.188 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:54.189 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:54.189 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:54.189 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:54.189 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/2-partial-upgrade/firsthalf.yaml").st_ino) 2024-04-26T21:04:54.189 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:54.371 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630050 2024-04-26T21:04:54.375 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:54.384 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:54.385 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:54.385 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:54.385 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:54.385 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/3-thrash").st_ino) 2024-04-26T21:04:54.385 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:54.583 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630051 2024-04-26T21:04:54.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:54.598 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:54.598 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:54.598 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:54.599 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:54.599 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/3-thrash/default.yaml").st_ino) 2024-04-26T21:04:54.599 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:54.798 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630052 2024-04-26T21:04:54.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:54.812 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:54.813 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:54.813 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:54.813 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:54.813 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/7-workload").st_ino) 2024-04-26T21:04:54.813 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:55.012 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630063 2024-04-26T21:04:55.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:55.027 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:55.027 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:55.027 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:55.027 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:55.027 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/7-workload/radosbench.yaml").st_ino) 2024-04-26T21:04:55.027 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:55.224 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630065 2024-04-26T21:04:55.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:55.238 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:55.238 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:55.238 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:55.239 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:55.239 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/7-workload/rbd_api.yaml").st_ino) 2024-04-26T21:04:55.239 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:55.435 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630066 2024-04-26T21:04:55.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:55.449 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:55.449 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:55.450 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:55.450 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:55.450 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/7-workload/+").st_ino) 2024-04-26T21:04:55.450 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:55.650 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630064 2024-04-26T21:04:55.655 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:55.666 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:55.666 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:55.666 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:55.666 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:55.666 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/0-cluster").st_ino) 2024-04-26T21:04:55.666 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:55.862 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630043 2024-04-26T21:04:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:55.877 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:55.877 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:55.878 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:55.878 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:55.878 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/0-cluster/openstack.yaml").st_ino) 2024-04-26T21:04:55.878 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:56.077 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630045 2024-04-26T21:04:56.081 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:56.092 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:56.092 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:56.092 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:56.092 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:56.092 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/0-cluster/+").st_ino) 2024-04-26T21:04:56.092 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:56.289 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630044 2024-04-26T21:04:56.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:56.304 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:56.305 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:56.305 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:56.305 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:56.305 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split/0-cluster/start.yaml").st_ino) 2024-04-26T21:04:56.305 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:56.503 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630046 2024-04-26T21:04:56.507 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:56.519 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:56.519 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:56.519 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:56.519 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:56.519 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64").st_ino) 2024-04-26T21:04:56.519 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:56.715 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630011 2024-04-26T21:04:56.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:56.729 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:56.729 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:56.729 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:56.729 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:56.729 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/6-next-mon").st_ino) 2024-04-26T21:04:56.730 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:56.927 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630022 2024-04-26T21:04:56.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:56.941 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:56.941 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:56.941 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:56.941 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:56.942 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/8-next-mon").st_ino) 2024-04-26T21:04:56.942 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:57.136 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630023 2024-04-26T21:04:57.141 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:57.152 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:57.152 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:57.152 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:57.152 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:57.152 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/1-jewel-install").st_ino) 2024-04-26T21:04:57.152 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:57.350 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630015 2024-04-26T21:04:57.354 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:57.365 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:57.365 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:57.365 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:57.365 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:57.365 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/5-workload").st_ino) 2024-04-26T21:04:57.365 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:57.563 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630020 2024-04-26T21:04:57.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:57.577 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:57.577 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:57.577 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:57.577 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:57.577 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/5-workload/ec-rados-default.yaml").st_ino) 2024-04-26T21:04:57.577 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:57.773 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630021 2024-04-26T21:04:57.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:57.789 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:57.789 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:57.789 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:57.789 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:57.789 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/4-mon").st_ino) 2024-04-26T21:04:57.789 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:57.972 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630019 2024-04-26T21:04:57.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:57.987 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:57.987 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:57.987 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:57.987 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:57.987 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/%").st_ino) 2024-04-26T21:04:57.987 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:58.182 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630012 2024-04-26T21:04:58.187 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:58.197 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:58.197 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:58.197 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:58.197 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:58.197 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/0-x86_64.yaml").st_ino) 2024-04-26T21:04:58.197 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:58.393 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630014 2024-04-26T21:04:58.397 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:58.407 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:58.407 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:58.408 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:58.408 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:58.408 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/9-workload").st_ino) 2024-04-26T21:04:58.408 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:58.607 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630024 2024-04-26T21:04:58.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:58.623 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:58.623 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:58.623 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:58.623 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:58.624 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/9-workload/ec-rados-plugin=jerasure-k=3-m=1.yaml").st_ino) 2024-04-26T21:04:58.624 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:58.818 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630025 2024-04-26T21:04:58.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:58.833 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:58.833 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:58.833 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:58.834 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:58.834 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/2-partial-upgrade").st_ino) 2024-04-26T21:04:58.834 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:59.030 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630016 2024-04-26T21:04:59.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:59.045 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:59.045 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:59.045 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:59.045 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:59.046 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/3-thrash").st_ino) 2024-04-26T21:04:59.046 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:59.242 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630017 2024-04-26T21:04:59.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:59.257 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:59.257 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:59.258 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:59.258 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:59.258 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/3-thrash/default.yaml").st_ino) 2024-04-26T21:04:59.258 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:59.454 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630018 2024-04-26T21:04:59.458 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:59.469 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:59.469 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:59.469 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:59.469 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:59.469 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code-x86_64/0-cluster").st_ino) 2024-04-26T21:04:59.469 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:59.648 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630013 2024-04-26T21:04:59.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:59.662 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:59.662 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:59.663 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:59.663 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:59.663 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code").st_ino) 2024-04-26T21:04:59.663 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:04:59.857 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630026 2024-04-26T21:04:59.862 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:04:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:04:59.873 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:04:59.873 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:04:59.873 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:04:59.873 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:04:59.873 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/6-next-mon").st_ino) 2024-04-26T21:04:59.873 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:00.071 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630036 2024-04-26T21:05:00.076 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:00.087 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:00.087 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:00.087 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:00.087 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:00.087 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/8-next-mon").st_ino) 2024-04-26T21:05:00.087 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:00.286 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630037 2024-04-26T21:05:00.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:00.302 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:00.302 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:00.303 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:00.303 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:00.303 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/1-jewel-install").st_ino) 2024-04-26T21:05:00.303 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:00.499 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630029 2024-04-26T21:05:00.504 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:00.514 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:00.514 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:00.514 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:00.514 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:00.514 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/5-workload").st_ino) 2024-04-26T21:05:00.514 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:00.698 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630034 2024-04-26T21:05:00.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:00.713 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:00.713 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:00.713 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:00.713 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:00.713 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/5-workload/ec-rados-default.yaml").st_ino) 2024-04-26T21:05:00.713 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:00.912 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630035 2024-04-26T21:05:00.916 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:00.927 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:00.927 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:00.927 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:00.927 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:00.927 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/4-mon").st_ino) 2024-04-26T21:05:00.927 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:01.120 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630033 2024-04-26T21:05:01.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:01.137 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:01.137 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:01.137 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:01.137 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:01.137 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/distros").st_ino) 2024-04-26T21:05:01.138 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:01.333 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630040 2024-04-26T21:05:01.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:01.348 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:01.348 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:01.348 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:01.348 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:01.348 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/%").st_ino) 2024-04-26T21:05:01.348 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:01.544 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630027 2024-04-26T21:05:01.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:01.559 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:01.559 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:01.559 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:01.559 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:01.559 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/9-workload").st_ino) 2024-04-26T21:05:01.559 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:01.759 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630038 2024-04-26T21:05:01.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:01.774 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:01.774 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:01.775 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:01.775 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:01.775 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/9-workload/ec-rados-plugin=jerasure-k=3-m=1.yaml").st_ino) 2024-04-26T21:05:01.775 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:01.978 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630039 2024-04-26T21:05:01.983 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:01.993 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:01.993 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:01.993 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:01.993 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:01.993 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/2-partial-upgrade").st_ino) 2024-04-26T21:05:01.993 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:02.190 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630030 2024-04-26T21:05:02.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:02.205 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:02.205 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:02.205 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:02.206 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:02.206 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/3-thrash").st_ino) 2024-04-26T21:05:02.206 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:02.401 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630031 2024-04-26T21:05:02.406 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:02.416 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:02.416 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:02.416 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:02.416 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:02.416 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/3-thrash/default.yaml").st_ino) 2024-04-26T21:05:02.416 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:02.611 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630032 2024-04-26T21:05:02.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:02.627 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:02.627 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:02.627 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:02.627 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:02.627 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/jewel-x/stress-split-erasure-code/0-cluster").st_ino) 2024-04-26T21:05:02.627 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:02.823 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630028 2024-04-26T21:05:02.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:02.839 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:02.839 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:02.839 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:02.839 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:02.839 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x").st_ino) 2024-04-26T21:05:02.839 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:03.033 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629914 2024-04-26T21:05:03.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:03.048 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:03.049 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:03.049 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:03.049 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:03.049 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel").st_ino) 2024-04-26T21:05:03.049 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:03.245 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629915 2024-04-26T21:05:03.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:03.260 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:03.260 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:03.261 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:03.261 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:03.261 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/8-kraken.yaml").st_ino) 2024-04-26T21:05:03.261 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:03.456 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629945 2024-04-26T21:05:03.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:03.470 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:03.471 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:03.471 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:03.471 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:03.471 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/1-hammer-jewel-install").st_ino) 2024-04-26T21:05:03.471 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:03.669 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629919 2024-04-26T21:05:03.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:03.684 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:03.684 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:03.684 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:03.684 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:03.685 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/1-hammer-jewel-install/hammer-jewel.yaml").st_ino) 2024-04-26T21:05:03.685 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:03.883 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629920 2024-04-26T21:05:03.887 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:03.898 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:03.898 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:03.898 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:03.898 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:03.898 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/4-jewel.yaml").st_ino) 2024-04-26T21:05:03.898 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:04.096 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629932 2024-04-26T21:05:04.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:04.110 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:04.111 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:04.111 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:04.111 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:04.111 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/3.5-finish.yaml").st_ino) 2024-04-26T21:05:04.111 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:04.309 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629931 2024-04-26T21:05:04.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:04.324 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:04.324 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:04.324 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:04.325 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:04.325 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/3-upgrade-sequence").st_ino) 2024-04-26T21:05:04.325 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:04.520 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629928 2024-04-26T21:05:04.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:04.535 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:04.535 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:04.535 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:04.535 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:04.535 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/3-upgrade-sequence/upgrade-osd-mds-mon.yaml").st_ino) 2024-04-26T21:05:04.535 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:04.737 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629930 2024-04-26T21:05:04.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:04.752 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:04.752 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:04.752 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:04.752 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:04.752 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/3-upgrade-sequence/upgrade-all.yaml").st_ino) 2024-04-26T21:05:04.752 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:04.952 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629929 2024-04-26T21:05:04.956 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:04.967 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:04.967 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:04.967 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:04.967 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:04.967 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/7-upgrade-sequence").st_ino) 2024-04-26T21:05:04.967 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:05.153 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629942 2024-04-26T21:05:05.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:05.167 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:05.167 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:05.167 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:05.167 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:05.167 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/7-upgrade-sequence/upgrade-by-daemon.yaml").st_ino) 2024-04-26T21:05:05.167 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:05.364 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629944 2024-04-26T21:05:05.369 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:05.379 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:05.379 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:05.379 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:05.379 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:05.379 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/7-upgrade-sequence/upgrade-all.yaml").st_ino) 2024-04-26T21:05:05.379 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:05.578 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629943 2024-04-26T21:05:05.583 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:05.593 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:05.593 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:05.593 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:05.593 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:05.593 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/distros").st_ino) 2024-04-26T21:05:05.593 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:05.791 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629954 2024-04-26T21:05:05.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:05.806 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:05.807 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:05.807 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:05.807 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:05.807 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/distros/centos_7.2.yaml").st_ino) 2024-04-26T21:05:05.807 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:06.006 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629955 2024-04-26T21:05:06.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:06.021 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:06.021 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:06.021 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:06.021 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:06.021 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/distros/ubuntu_14.04.yaml").st_ino) 2024-04-26T21:05:06.021 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:06.222 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629956 2024-04-26T21:05:06.226 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:06.237 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:06.237 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:06.237 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:06.237 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:06.237 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/%").st_ino) 2024-04-26T21:05:06.237 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:06.433 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629916 2024-04-26T21:05:06.438 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:06.448 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:06.448 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:06.448 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:06.448 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:06.448 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/9-final-workload").st_ino) 2024-04-26T21:05:06.448 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:06.648 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629946 2024-04-26T21:05:06.652 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:06.662 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:06.663 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:06.663 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:06.663 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:06.663 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/9-final-workload/rados-snaps-few-objects.yaml").st_ino) 2024-04-26T21:05:06.663 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:06.845 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629948 2024-04-26T21:05:06.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:06.860 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:06.860 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:06.860 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:06.861 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:06.861 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/9-final-workload/+").st_ino) 2024-04-26T21:05:06.861 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:07.043 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629947 2024-04-26T21:05:07.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:07.057 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:07.057 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:07.057 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:07.057 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:07.057 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/9-final-workload/rados_loadgenmix.yaml").st_ino) 2024-04-26T21:05:07.057 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:07.255 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629949 2024-04-26T21:05:07.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:07.269 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:07.269 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:07.269 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:07.269 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:07.269 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/9-final-workload/rbd_cls.yaml").st_ino) 2024-04-26T21:05:07.269 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:07.466 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629951 2024-04-26T21:05:07.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:07.481 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:07.481 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:07.481 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:07.481 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:07.482 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/9-final-workload/rados_mon_thrash.yaml").st_ino) 2024-04-26T21:05:07.482 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:07.681 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629950 2024-04-26T21:05:07.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:07.696 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:07.697 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:07.697 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:07.697 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:07.697 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/9-final-workload/rbd_import_export.yaml").st_ino) 2024-04-26T21:05:07.697 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:07.902 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629952 2024-04-26T21:05:07.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:07.916 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:07.916 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:07.916 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:07.916 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:07.916 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/9-final-workload/rgw_s3tests.yaml").st_ino) 2024-04-26T21:05:07.916 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:08.115 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629953 2024-04-26T21:05:08.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:08.131 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:08.131 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:08.131 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:08.131 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:08.131 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/2-workload").st_ino) 2024-04-26T21:05:08.131 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:08.326 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629921 2024-04-26T21:05:08.331 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:08.341 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:08.341 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:08.341 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:08.341 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:08.341 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/2-workload/test_rbd_python.yaml").st_ino) 2024-04-26T21:05:08.341 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:08.536 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629927 2024-04-26T21:05:08.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:08.552 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:08.552 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:08.552 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:08.552 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:08.552 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/2-workload/rados_api.yaml").st_ino) 2024-04-26T21:05:08.552 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:08.753 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629924 2024-04-26T21:05:08.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:08.768 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:08.769 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:08.769 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:08.769 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:08.769 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/2-workload/rados_loadgenbig.yaml").st_ino) 2024-04-26T21:05:08.769 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:08.966 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629925 2024-04-26T21:05:08.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:08.981 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:08.981 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:08.981 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:08.981 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:08.982 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/2-workload/ec-rados-default.yaml").st_ino) 2024-04-26T21:05:08.982 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:09.181 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629923 2024-04-26T21:05:09.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:09.196 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:09.197 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:09.197 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:09.197 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:09.197 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/2-workload/+").st_ino) 2024-04-26T21:05:09.197 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:09.394 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629922 2024-04-26T21:05:09.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:09.409 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:09.409 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:09.409 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:09.409 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:09.409 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/2-workload/test_rbd_api.yaml").st_ino) 2024-04-26T21:05:09.409 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:09.605 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629926 2024-04-26T21:05:09.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:09.619 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:09.620 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:09.620 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:09.620 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:09.620 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/6-workload").st_ino) 2024-04-26T21:05:09.620 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:09.807 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629935 2024-04-26T21:05:09.812 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:09.822 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:09.822 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:09.822 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:09.822 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:09.822 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/6-workload/test_rbd_python.yaml").st_ino) 2024-04-26T21:05:09.823 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:10.018 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629941 2024-04-26T21:05:10.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:10.034 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:10.034 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:10.034 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:10.034 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:10.034 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/6-workload/rados_api.yaml").st_ino) 2024-04-26T21:05:10.035 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:10.231 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629938 2024-04-26T21:05:10.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:10.247 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:10.247 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:10.247 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:10.247 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:10.247 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/6-workload/rados_loadgenbig.yaml").st_ino) 2024-04-26T21:05:10.247 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:10.435 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629939 2024-04-26T21:05:10.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:10.450 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:10.450 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:10.451 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:10.451 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:10.451 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/6-workload/ec-rados-default.yaml").st_ino) 2024-04-26T21:05:10.451 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:10.649 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629937 2024-04-26T21:05:10.653 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:10.663 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:10.663 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:10.663 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:10.664 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:10.664 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/6-workload/+").st_ino) 2024-04-26T21:05:10.664 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:10.852 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629936 2024-04-26T21:05:10.856 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:10.867 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:10.867 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:10.867 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:10.867 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:10.867 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/6-workload/test_rbd_api.yaml").st_ino) 2024-04-26T21:05:10.867 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:11.063 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629940 2024-04-26T21:05:11.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:11.078 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:11.078 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:11.078 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:11.078 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:11.078 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/5-hammer-jewel-x-upgrade").st_ino) 2024-04-26T21:05:11.078 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:11.274 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629933 2024-04-26T21:05:11.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:11.289 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:11.289 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:11.289 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:11.289 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:11.289 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/5-hammer-jewel-x-upgrade/hammer-jewel-x.yaml").st_ino) 2024-04-26T21:05:11.289 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:11.488 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629934 2024-04-26T21:05:11.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:11.502 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:11.502 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:11.502 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:11.503 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:11.503 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/0-cluster").st_ino) 2024-04-26T21:05:11.503 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:11.688 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629917 2024-04-26T21:05:11.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:11.703 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:11.703 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:11.703 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:11.703 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:11.703 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/parallel/0-cluster/start.yaml").st_ino) 2024-04-26T21:05:11.704 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:11.904 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629918 2024-04-26T21:05:11.909 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:11.919 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:11.919 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:11.919 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:11.920 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:11.920 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split").st_ino) 2024-04-26T21:05:11.920 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:12.115 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629957 2024-04-26T21:05:12.120 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:12.130 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:12.130 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:12.131 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:12.131 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:12.131 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/6-next-mon").st_ino) 2024-04-26T21:05:12.131 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:12.329 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629966 2024-04-26T21:05:12.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:12.344 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:12.344 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:12.344 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:12.344 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:12.344 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/8-next-mon").st_ino) 2024-04-26T21:05:12.344 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:12.543 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629968 2024-04-26T21:05:12.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:12.558 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:12.558 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:12.558 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:12.558 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:12.558 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/5-workload").st_ino) 2024-04-26T21:05:12.558 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:12.755 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629965 2024-04-26T21:05:12.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:12.770 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:12.770 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:12.770 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:12.770 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:12.770 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/4-mon").st_ino) 2024-04-26T21:05:12.770 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:12.968 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629964 2024-04-26T21:05:12.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:12.983 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:12.983 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:12.983 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:12.983 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:12.984 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/distros").st_ino) 2024-04-26T21:05:12.984 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:13.180 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629970 2024-04-26T21:05:13.185 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:13.195 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:13.195 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:13.195 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:13.195 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:13.195 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/distros/centos_7.2.yaml").st_ino) 2024-04-26T21:05:13.195 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:13.397 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629971 2024-04-26T21:05:13.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:13.412 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:13.412 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:13.412 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:13.412 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:13.412 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/distros/ubuntu_14.04.yaml").st_ino) 2024-04-26T21:05:13.412 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:13.605 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629972 2024-04-26T21:05:13.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:13.620 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:13.620 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:13.620 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:13.621 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:13.621 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/%").st_ino) 2024-04-26T21:05:13.621 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:13.816 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629958 2024-04-26T21:05:13.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:13.831 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:13.831 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:13.831 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:13.831 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:13.831 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/9-workload").st_ino) 2024-04-26T21:05:13.832 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:14.028 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629969 2024-04-26T21:05:14.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:14.042 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:14.043 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:14.043 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:14.043 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:14.043 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/1-hammer-install-and-upgrade-to-jewel").st_ino) 2024-04-26T21:05:14.043 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:14.239 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629960 2024-04-26T21:05:14.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:14.254 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:14.255 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:14.255 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:14.255 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:14.255 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/1-hammer-install-and-upgrade-to-jewel/hammer-to-jewel.yaml").st_ino) 2024-04-26T21:05:14.255 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:14.452 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629961 2024-04-26T21:05:14.457 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:14.467 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:14.467 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:14.467 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:14.467 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:14.467 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/2-partial-upgrade").st_ino) 2024-04-26T21:05:14.467 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:14.666 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629962 2024-04-26T21:05:14.670 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:14.681 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:14.681 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:14.681 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:14.681 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:14.681 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/3-thrash").st_ino) 2024-04-26T21:05:14.682 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:14.879 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629963 2024-04-26T21:05:14.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:14.893 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:14.893 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:14.893 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:14.893 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:14.894 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/7-workload").st_ino) 2024-04-26T21:05:14.894 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:15.094 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629967 2024-04-26T21:05:15.099 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:15.110 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:15.110 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:15.111 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:15.111 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:15.111 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/hammer-jewel-x/stress-split/0-cluster").st_ino) 2024-04-26T21:05:15.111 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:15.306 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629959 2024-04-26T21:05:15.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:15.321 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:15.321 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:15.321 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:15.321 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:15.321 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade").st_ino) 2024-04-26T21:05:15.321 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:15.516 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629877 2024-04-26T21:05:15.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:15.531 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:15.531 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:15.531 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:15.531 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:15.532 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x").st_ino) 2024-04-26T21:05:15.532 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:15.730 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629878 2024-04-26T21:05:15.735 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:15.746 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:15.746 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:15.746 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:15.746 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:15.746 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/rbd").st_ino) 2024-04-26T21:05:15.746 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:15.940 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629888 2024-04-26T21:05:15.945 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:15.955 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:15.955 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:15.955 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:15.955 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:15.955 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/rbd/1-install").st_ino) 2024-04-26T21:05:15.955 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:16.155 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629892 2024-04-26T21:05:16.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:16.168 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:16.168 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:16.168 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:16.168 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:16.168 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/rbd/1-install/hammer-client-x.yaml").st_ino) 2024-04-26T21:05:16.168 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:16.356 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629893 2024-04-26T21:05:16.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:16.372 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:16.372 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:16.373 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:16.373 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:16.373 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/rbd/%").st_ino) 2024-04-26T21:05:16.373 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:16.570 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629889 2024-04-26T21:05:16.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:16.584 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:16.584 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:16.584 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:16.584 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:16.584 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/rbd/2-workload").st_ino) 2024-04-26T21:05:16.584 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:16.779 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629894 2024-04-26T21:05:16.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:16.793 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:16.793 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:16.793 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:16.793 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:16.793 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/rbd/2-workload/rbd_notification_tests.yaml").st_ino) 2024-04-26T21:05:16.793 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:16.992 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629895 2024-04-26T21:05:16.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:17.007 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:17.007 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:17.007 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:17.007 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:17.007 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/rbd/0-cluster").st_ino) 2024-04-26T21:05:17.007 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:17.204 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629890 2024-04-26T21:05:17.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:17.219 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:17.219 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:17.219 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:17.219 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:17.219 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/rbd/0-cluster/start.yaml").st_ino) 2024-04-26T21:05:17.219 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:17.413 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629891 2024-04-26T21:05:17.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:17.428 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:17.428 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:17.428 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:17.428 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:17.428 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/basic").st_ino) 2024-04-26T21:05:17.428 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:17.626 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629879 2024-04-26T21:05:17.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:17.640 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:17.640 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:17.640 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:17.640 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:17.640 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/basic/1-install").st_ino) 2024-04-26T21:05:17.640 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:17.831 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:05:17.839 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:05:17.843 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629883 2024-04-26T21:05:17.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:17.857 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:17.858 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:17.858 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:17.858 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:17.858 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/basic/1-install/hammer-client-x.yaml").st_ino) 2024-04-26T21:05:17.858 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:18.046 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629884 2024-04-26T21:05:18.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:18.061 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:18.061 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:18.061 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:18.061 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:18.061 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/basic/%").st_ino) 2024-04-26T21:05:18.061 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:18.206 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629880 2024-04-26T21:05:18.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:18.219 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:18.219 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:18.219 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:18.220 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:18.220 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/basic/2-workload").st_ino) 2024-04-26T21:05:18.220 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:18.416 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629885 2024-04-26T21:05:18.421 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:18.431 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:18.431 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:18.431 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:18.431 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:18.431 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/basic/2-workload/rbd_cli_import_export.yaml").st_ino) 2024-04-26T21:05:18.431 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:18.633 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629887 2024-04-26T21:05:18.637 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:18.647 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:18.647 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:18.647 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:18.647 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:18.647 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/basic/2-workload/rbd_api_tests.yaml").st_ino) 2024-04-26T21:05:18.647 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:18.845 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629886 2024-04-26T21:05:18.850 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:18.861 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:18.862 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:18.862 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:18.862 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:18.862 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/basic/0-cluster").st_ino) 2024-04-26T21:05:18.862 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:19.060 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629881 2024-04-26T21:05:19.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:19.074 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:19.074 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:19.074 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:19.074 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:19.074 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/hammer-client-x/basic/0-cluster/start.yaml").st_ino) 2024-04-26T21:05:19.074 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:19.270 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629882 2024-04-26T21:05:19.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:19.285 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:19.285 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:19.285 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:19.285 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:19.285 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x").st_ino) 2024-04-26T21:05:19.285 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:19.473 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629896 2024-04-26T21:05:19.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:19.487 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:19.488 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:19.488 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:19.488 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:19.488 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/rbd").st_ino) 2024-04-26T21:05:19.488 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:19.685 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629906 2024-04-26T21:05:19.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:19.700 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:19.700 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:19.700 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:19.700 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:19.700 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/rbd/1-install").st_ino) 2024-04-26T21:05:19.700 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:19.901 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629910 2024-04-26T21:05:19.905 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:19.915 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:19.915 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:19.915 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:19.915 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:19.916 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/rbd/1-install/jewel-client-x.yaml").st_ino) 2024-04-26T21:05:19.916 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:20.119 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629911 2024-04-26T21:05:20.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:20.134 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:20.134 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:20.134 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:20.134 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:20.134 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/rbd/%").st_ino) 2024-04-26T21:05:20.134 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:20.329 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629907 2024-04-26T21:05:20.333 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:20.343 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:20.343 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:20.343 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:20.343 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:20.343 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/rbd/2-workload").st_ino) 2024-04-26T21:05:20.344 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:20.540 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629912 2024-04-26T21:05:20.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:20.555 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:20.555 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:20.555 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:20.555 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:20.556 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/rbd/2-workload/rbd_notification_tests.yaml").st_ino) 2024-04-26T21:05:20.556 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:20.753 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629913 2024-04-26T21:05:20.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:20.768 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:20.768 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:20.768 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:20.768 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:20.768 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/rbd/0-cluster").st_ino) 2024-04-26T21:05:20.768 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:20.965 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629908 2024-04-26T21:05:20.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:20.980 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:20.980 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:20.980 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:20.980 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:20.980 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/rbd/0-cluster/start.yaml").st_ino) 2024-04-26T21:05:20.980 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:21.178 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629909 2024-04-26T21:05:21.183 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:21.193 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:21.194 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:21.194 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:21.194 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:21.194 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/basic").st_ino) 2024-04-26T21:05:21.194 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:21.390 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629897 2024-04-26T21:05:21.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:21.405 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:21.405 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:21.405 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:21.405 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:21.405 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/basic/1-install").st_ino) 2024-04-26T21:05:21.405 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:21.603 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629901 2024-04-26T21:05:21.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:21.617 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:21.617 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:21.617 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:21.617 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:21.617 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/basic/1-install/jewel-client-x.yaml").st_ino) 2024-04-26T21:05:21.618 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:21.815 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629902 2024-04-26T21:05:21.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:21.830 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:21.830 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:21.830 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:21.831 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:21.831 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/basic/%").st_ino) 2024-04-26T21:05:21.831 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:22.029 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629898 2024-04-26T21:05:22.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:22.045 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:22.045 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:22.045 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:22.045 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:22.045 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/basic/2-workload").st_ino) 2024-04-26T21:05:22.045 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:22.229 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629903 2024-04-26T21:05:22.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:22.244 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:22.244 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:22.244 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:22.244 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:22.244 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/basic/2-workload/rbd_cli_import_export.yaml").st_ino) 2024-04-26T21:05:22.244 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:22.439 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629905 2024-04-26T21:05:22.444 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:22.454 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:22.454 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:22.454 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:22.454 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:22.455 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/basic/2-workload/rbd_api_tests.yaml").st_ino) 2024-04-26T21:05:22.455 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:22.643 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629904 2024-04-26T21:05:22.648 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:22.659 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:22.659 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:22.659 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:22.659 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:22.659 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/basic/0-cluster").st_ino) 2024-04-26T21:05:22.659 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:22.855 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629899 2024-04-26T21:05:22.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:22.871 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:22.871 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:22.871 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:22.871 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:22.871 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/upgrade/client-upgrade/jewel-client-x/basic/0-cluster/start.yaml").st_ino) 2024-04-26T21:05:22.871 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:23.073 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629900 2024-04-26T21:05:23.078 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:23.089 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:23.089 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:23.090 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:23.090 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:23.090 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/dummy").st_ino) 2024-04-26T21:05:23.090 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:23.284 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628540 2024-04-26T21:05:23.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:23.299 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:23.299 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:23.299 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:23.299 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:23.299 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/dummy/%").st_ino) 2024-04-26T21:05:23.299 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:23.495 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628541 2024-04-26T21:05:23.499 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:23.510 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:23.510 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:23.510 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:23.510 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:23.510 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/dummy/all").st_ino) 2024-04-26T21:05:23.510 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:23.710 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628542 2024-04-26T21:05:23.715 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:23.725 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:23.725 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:23.725 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:23.725 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:23.725 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/dummy/all/nop.yaml").st_ino) 2024-04-26T21:05:23.725 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:23.921 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628543 2024-04-26T21:05:23.925 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:23.935 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:23.935 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:23.935 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:23.935 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:23.935 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/hadoop").st_ino) 2024-04-26T21:05:23.935 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:24.129 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628809 2024-04-26T21:05:24.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:24.143 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:24.144 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:24.144 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:24.144 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:24.144 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/hadoop/basic").st_ino) 2024-04-26T21:05:24.144 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:24.336 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628810 2024-04-26T21:05:24.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:24.351 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:24.352 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:24.352 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:24.352 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:24.352 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/hadoop/basic/clusters").st_ino) 2024-04-26T21:05:24.352 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:24.545 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628812 2024-04-26T21:05:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:24.560 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:24.560 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:24.560 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:24.560 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:24.560 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/hadoop/basic/clusters/fixed-3.yaml").st_ino) 2024-04-26T21:05:24.560 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:24.757 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628813 2024-04-26T21:05:24.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:24.772 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:24.772 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:24.772 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:24.772 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:24.772 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/hadoop/basic/%").st_ino) 2024-04-26T21:05:24.772 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:24.969 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628811 2024-04-26T21:05:24.973 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:24.984 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:24.984 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:24.984 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:24.984 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:24.984 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/hadoop/basic/xfs.yaml").st_ino) 2024-04-26T21:05:24.984 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:25.181 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628818 2024-04-26T21:05:25.186 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:25.196 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:25.196 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:25.196 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:25.196 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:25.197 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/hadoop/basic/tasks").st_ino) 2024-04-26T21:05:25.197 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:25.394 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628814 2024-04-26T21:05:25.398 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:25.409 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:25.410 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:25.410 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:25.410 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:25.410 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/hadoop/basic/tasks/wordcount.yaml").st_ino) 2024-04-26T21:05:25.410 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:25.608 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628817 2024-04-26T21:05:25.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:25.623 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:25.623 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:25.623 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:25.623 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:25.623 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/hadoop/basic/tasks/repl.yaml").st_ino) 2024-04-26T21:05:25.623 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:25.818 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628815 2024-04-26T21:05:25.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:25.832 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:25.832 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:25.832 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:25.832 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:25.832 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/hadoop/basic/tasks/terasort.yaml").st_ino) 2024-04-26T21:05:25.832 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:26.027 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628816 2024-04-26T21:05:26.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:26.041 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:26.042 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:26.042 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:26.042 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:26.042 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology").st_ino) 2024-04-26T21:05:26.042 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:26.240 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629808 2024-04-26T21:05:26.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:26.254 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:26.255 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:26.255 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:26.255 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:26.255 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/integration.yaml").st_ino) 2024-04-26T21:05:26.255 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:26.451 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629823 2024-04-26T21:05:26.456 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:26.466 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:26.466 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:26.466 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:26.466 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:26.466 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/nop").st_ino) 2024-04-26T21:05:26.466 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:26.672 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629839 2024-04-26T21:05:26.676 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:26.687 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:26.687 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:26.687 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:26.687 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:26.687 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/nop/%").st_ino) 2024-04-26T21:05:26.687 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:26.885 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629840 2024-04-26T21:05:26.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:26.900 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:26.900 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:26.900 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:26.900 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:26.901 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/nop/all").st_ino) 2024-04-26T21:05:26.901 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:27.096 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629841 2024-04-26T21:05:27.101 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:27.111 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:27.111 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:27.111 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:27.111 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:27.111 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/nop/all/nop.yaml").st_ino) 2024-04-26T21:05:27.111 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:27.293 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629842 2024-04-26T21:05:27.298 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:27.309 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:27.309 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:27.309 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:27.309 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:27.309 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/rgw").st_ino) 2024-04-26T21:05:27.309 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:27.503 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629843 2024-04-26T21:05:27.508 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:27.519 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:27.519 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:27.519 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:27.519 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:27.519 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/rgw/distros").st_ino) 2024-04-26T21:05:27.519 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:27.720 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629845 2024-04-26T21:05:27.724 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:27.734 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:27.734 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:27.735 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:27.735 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:27.735 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/rgw/%").st_ino) 2024-04-26T21:05:27.735 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:27.931 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629844 2024-04-26T21:05:27.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:27.947 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:27.947 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:27.947 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:27.947 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:27.947 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/rgw/tasks").st_ino) 2024-04-26T21:05:27.947 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:28.144 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629846 2024-04-26T21:05:28.148 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:28.159 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:28.159 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:28.159 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:28.159 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:28.159 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/rgw/tasks/s3tests-fastcgi.yaml").st_ino) 2024-04-26T21:05:28.159 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:28.356 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629848 2024-04-26T21:05:28.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:28.370 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:28.370 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:28.370 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:28.370 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:28.370 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/rgw/tasks/s3tests-fcgi.yaml").st_ino) 2024-04-26T21:05:28.370 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:28.566 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629849 2024-04-26T21:05:28.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:28.582 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:28.582 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:28.582 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:28.582 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:28.582 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/rgw/tasks/s3tests-civetweb.yaml").st_ino) 2024-04-26T21:05:28.582 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:28.776 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629847 2024-04-26T21:05:28.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:28.791 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:28.792 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:28.792 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:28.792 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:28.792 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/no-ceph").st_ino) 2024-04-26T21:05:28.792 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:28.986 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629833 2024-04-26T21:05:28.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:29.001 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:29.001 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:29.001 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:29.001 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:29.001 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/no-ceph/clusters").st_ino) 2024-04-26T21:05:29.001 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:29.196 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629835 2024-04-26T21:05:29.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:29.211 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:29.211 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:29.211 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:29.211 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:29.211 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/no-ceph/clusters/single.yaml").st_ino) 2024-04-26T21:05:29.212 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:29.407 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629836 2024-04-26T21:05:29.411 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:29.422 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:29.422 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:29.422 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:29.422 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:29.422 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/no-ceph/%").st_ino) 2024-04-26T21:05:29.422 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:29.618 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629834 2024-04-26T21:05:29.622 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:29.632 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:29.632 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:29.633 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:29.633 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:29.633 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/no-ceph/tasks").st_ino) 2024-04-26T21:05:29.633 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:29.829 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629837 2024-04-26T21:05:29.834 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:29.844 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:29.845 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:29.845 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:29.845 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:29.845 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/no-ceph/tasks/teuthology.yaml").st_ino) 2024-04-26T21:05:29.845 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:30.044 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629838 2024-04-26T21:05:30.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:30.059 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:30.059 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:30.059 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:30.059 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:30.059 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/ceph").st_ino) 2024-04-26T21:05:30.059 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:30.253 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629816 2024-04-26T21:05:30.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:30.269 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:30.269 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:30.269 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:30.269 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:30.269 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/ceph/clusters").st_ino) 2024-04-26T21:05:30.269 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:30.458 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629818 2024-04-26T21:05:30.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:30.473 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:30.473 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:30.473 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:30.474 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:30.474 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/ceph/clusters/single.yaml").st_ino) 2024-04-26T21:05:30.474 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:30.674 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629819 2024-04-26T21:05:30.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:30.690 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:30.690 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:30.690 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:30.690 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:30.690 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/ceph/distros").st_ino) 2024-04-26T21:05:30.691 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:30.887 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629820 2024-04-26T21:05:30.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:30.902 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:30.902 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:30.902 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:30.902 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:30.902 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/ceph/%").st_ino) 2024-04-26T21:05:30.902 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:31.102 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629817 2024-04-26T21:05:31.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:31.117 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:31.117 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:31.117 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:31.117 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:31.117 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/ceph/tasks").st_ino) 2024-04-26T21:05:31.117 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:31.314 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629821 2024-04-26T21:05:31.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:31.328 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:31.328 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:31.328 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:31.328 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:31.328 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/ceph/tasks/teuthology.yaml").st_ino) 2024-04-26T21:05:31.328 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:31.525 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629822 2024-04-26T21:05:31.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:31.541 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:31.541 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:31.541 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:31.541 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:31.542 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/buildpackages").st_ino) 2024-04-26T21:05:31.542 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:31.735 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629809 2024-04-26T21:05:31.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:31.750 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:31.750 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:31.750 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:31.750 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:31.750 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/buildpackages/distros").st_ino) 2024-04-26T21:05:31.751 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:31.946 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629811 2024-04-26T21:05:31.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:31.960 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:31.960 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:31.961 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:31.961 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:31.961 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/buildpackages/%").st_ino) 2024-04-26T21:05:31.961 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:32.163 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629810 2024-04-26T21:05:32.168 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:32.178 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:32.179 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:32.179 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:32.179 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:32.179 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/buildpackages/tasks").st_ino) 2024-04-26T21:05:32.179 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:32.373 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629812 2024-04-26T21:05:32.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:32.387 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:32.388 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:32.388 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:32.388 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:32.388 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/buildpackages/tasks/default.yaml").st_ino) 2024-04-26T21:05:32.388 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:32.590 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629814 2024-04-26T21:05:32.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:32.605 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:32.605 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:32.605 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:32.605 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:32.606 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/buildpackages/tasks/tag.yaml").st_ino) 2024-04-26T21:05:32.606 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:32.800 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629815 2024-04-26T21:05:32.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:32.814 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:32.814 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:32.815 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:32.815 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:32.815 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/buildpackages/tasks/branch.yaml").st_ino) 2024-04-26T21:05:32.815 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:33.037 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629813 2024-04-26T21:05:33.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:33.057 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:33.057 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:33.057 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:33.057 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:33.057 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/multi-cluster").st_ino) 2024-04-26T21:05:33.057 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:33.258 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629824 2024-04-26T21:05:33.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:33.272 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:33.272 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:33.272 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:33.272 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:33.273 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/multi-cluster/fs").st_ino) 2024-04-26T21:05:33.273 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:33.458 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629831 2024-04-26T21:05:33.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:33.472 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:33.473 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:33.473 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:33.473 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:33.473 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/multi-cluster/fs/xfs.yaml").st_ino) 2024-04-26T21:05:33.473 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:33.670 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629832 2024-04-26T21:05:33.675 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:33.684 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:33.684 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:33.684 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:33.684 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:33.685 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/multi-cluster/%").st_ino) 2024-04-26T21:05:33.685 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:33.887 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629825 2024-04-26T21:05:33.891 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:33.901 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:33.901 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:33.902 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:33.902 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:33.902 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/multi-cluster/all").st_ino) 2024-04-26T21:05:33.902 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:34.098 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629826 2024-04-26T21:05:34.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:34.113 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:34.113 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:34.113 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:34.113 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:34.113 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/multi-cluster/all/thrashosds.yaml").st_ino) 2024-04-26T21:05:34.113 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:34.315 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629828 2024-04-26T21:05:34.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:34.331 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:34.331 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:34.331 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:34.331 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:34.331 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/multi-cluster/all/ceph.yaml").st_ino) 2024-04-26T21:05:34.331 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:34.527 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629827 2024-04-26T21:05:34.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:34.543 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:34.543 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:34.543 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:34.543 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:34.543 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/multi-cluster/all/upgrade.yaml").st_ino) 2024-04-26T21:05:34.543 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:34.740 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629829 2024-04-26T21:05:34.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:34.755 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:34.755 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:34.755 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:34.755 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:34.755 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/multi-cluster/all/workunit.yaml").st_ino) 2024-04-26T21:05:34.756 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:34.960 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629830 2024-04-26T21:05:34.965 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:34.975 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:34.976 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:34.976 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:34.976 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:34.976 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/workunits").st_ino) 2024-04-26T21:05:34.976 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:35.174 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629850 2024-04-26T21:05:35.179 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:35.190 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:35.190 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:35.190 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:35.190 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:35.190 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/teuthology/workunits/yes.yaml").st_ino) 2024-04-26T21:05:35.190 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:35.387 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629851 2024-04-26T21:05:35.391 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:35.402 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:35.402 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:35.402 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:35.402 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:35.402 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds").st_ino) 2024-04-26T21:05:35.402 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:35.600 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629061 2024-04-26T21:05:35.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:35.616 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:35.616 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:35.617 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:35.617 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:35.617 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/basic").st_ino) 2024-04-26T21:05:35.617 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:35.813 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629062 2024-04-26T21:05:35.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:35.828 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:35.828 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:35.828 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:35.828 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:35.829 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/basic/overrides").st_ino) 2024-04-26T21:05:35.829 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:36.024 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629073 2024-04-26T21:05:36.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:36.039 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:36.039 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:36.039 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:36.039 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:36.039 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/basic/overrides/fuse-default-perm-no.yaml").st_ino) 2024-04-26T21:05:36.039 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:36.239 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629076 2024-04-26T21:05:36.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:36.255 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:36.255 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:36.255 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:36.255 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:36.255 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/basic/overrides/basic").st_ino) 2024-04-26T21:05:36.255 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:36.456 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629075 2024-04-26T21:05:36.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:36.472 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:36.472 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:36.472 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:36.472 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:36.472 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/basic/overrides/%").st_ino) 2024-04-26T21:05:36.472 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:36.669 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629074 2024-04-26T21:05:36.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:36.683 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:36.683 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:36.684 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:36.684 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:36.684 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/basic/clusters").st_ino) 2024-04-26T21:05:36.684 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:36.881 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629065 2024-04-26T21:05:36.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:36.895 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:36.895 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:36.895 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:36.895 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:36.896 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/basic/clusters/9-mds.yaml").st_ino) 2024-04-26T21:05:36.896 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:37.092 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629067 2024-04-26T21:05:37.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:37.106 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:37.106 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:37.106 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:37.107 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:37.107 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/basic/clusters/3-mds.yaml").st_ino) 2024-04-26T21:05:37.107 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:37.303 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629066 2024-04-26T21:05:37.307 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:37.317 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:37.318 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:37.318 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:37.318 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:37.318 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/basic/fs").st_ino) 2024-04-26T21:05:37.318 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:37.508 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629068 2024-04-26T21:05:37.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:37.523 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:37.523 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:37.523 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:37.524 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:37.524 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/basic/mount").st_ino) 2024-04-26T21:05:37.524 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:37.705 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629070 2024-04-26T21:05:37.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:37.719 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:37.719 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:37.719 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:37.719 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:37.719 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/basic/mount/kclient.yaml").st_ino) 2024-04-26T21:05:37.719 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:37.906 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629072 2024-04-26T21:05:37.910 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:37.920 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:37.920 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:37.921 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:37.921 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:37.921 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/basic/mount/fuse.yaml").st_ino) 2024-04-26T21:05:37.921 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:38.118 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629071 2024-04-26T21:05:38.123 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:38.133 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:38.133 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:38.133 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:38.133 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:38.133 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/basic/inline").st_ino) 2024-04-26T21:05:38.134 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:38.331 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629069 2024-04-26T21:05:38.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:38.346 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:38.346 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:38.346 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:38.346 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:38.346 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/basic/begin.yaml").st_ino) 2024-04-26T21:05:38.346 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:38.539 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629064 2024-04-26T21:05:38.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:38.553 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:38.553 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:38.553 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:38.553 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:38.554 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/basic/%").st_ino) 2024-04-26T21:05:38.554 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:38.750 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629063 2024-04-26T21:05:38.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:38.765 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:38.765 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:38.765 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:38.765 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:38.765 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/basic/tasks").st_ino) 2024-04-26T21:05:38.765 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:38.962 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629077 2024-04-26T21:05:38.967 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:38.978 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:38.978 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:38.978 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:38.978 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:38.978 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/verify").st_ino) 2024-04-26T21:05:38.978 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:39.185 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629078 2024-04-26T21:05:39.190 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:39.201 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:39.201 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:39.201 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:39.201 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:39.201 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/verify/validater").st_ino) 2024-04-26T21:05:39.201 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:39.396 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629093 2024-04-26T21:05:39.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:39.411 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:39.411 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:39.411 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:39.412 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:39.412 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/verify/overrides").st_ino) 2024-04-26T21:05:39.412 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:39.611 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629088 2024-04-26T21:05:39.616 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:39.627 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:39.627 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:39.627 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:39.627 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:39.627 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/verify/overrides/fuse-default-perm-no.yaml").st_ino) 2024-04-26T21:05:39.627 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:39.823 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629090 2024-04-26T21:05:39.828 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:39.839 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:39.839 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:39.839 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:39.839 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:39.839 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/verify/overrides/%").st_ino) 2024-04-26T21:05:39.839 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:40.034 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629089 2024-04-26T21:05:40.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:40.050 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:40.050 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:40.050 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:40.050 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:40.050 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/verify/overrides/verify").st_ino) 2024-04-26T21:05:40.050 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:40.247 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629091 2024-04-26T21:05:40.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:40.262 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:40.262 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:40.262 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:40.263 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:40.263 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/verify/clusters").st_ino) 2024-04-26T21:05:40.263 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:40.461 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629081 2024-04-26T21:05:40.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:40.477 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:40.477 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:40.477 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:40.477 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:40.478 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/verify/clusters/9-mds.yaml").st_ino) 2024-04-26T21:05:40.478 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:40.675 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629083 2024-04-26T21:05:40.679 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:40.689 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:40.689 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:40.689 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:40.689 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:40.689 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/verify/clusters/3-mds.yaml").st_ino) 2024-04-26T21:05:40.689 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:40.885 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629082 2024-04-26T21:05:40.890 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:40.900 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:40.900 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:40.900 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:40.900 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:40.900 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/verify/fs").st_ino) 2024-04-26T21:05:40.900 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:41.097 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629084 2024-04-26T21:05:41.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:41.112 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:41.113 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:41.113 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:41.113 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:41.113 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/verify/mount").st_ino) 2024-04-26T21:05:41.113 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:41.314 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629085 2024-04-26T21:05:41.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:41.329 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:41.329 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:41.329 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:41.329 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:41.329 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/verify/mount/kclient.yaml").st_ino) 2024-04-26T21:05:41.329 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:41.528 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629087 2024-04-26T21:05:41.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:41.543 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:41.543 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:41.543 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:41.543 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:41.543 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/verify/mount/fuse.yaml").st_ino) 2024-04-26T21:05:41.543 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:41.726 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629086 2024-04-26T21:05:41.731 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:41.741 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:41.741 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:41.741 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:41.741 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:41.741 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/verify/begin.yaml").st_ino) 2024-04-26T21:05:41.741 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:41.936 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629080 2024-04-26T21:05:41.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:41.951 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:41.951 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:41.951 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:41.952 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:41.952 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/verify/%").st_ino) 2024-04-26T21:05:41.952 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:42.148 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629079 2024-04-26T21:05:42.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:42.163 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:42.163 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:42.163 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:42.163 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:42.164 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/multimds/verify/tasks").st_ino) 2024-04-26T21:05:42.164 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:42.359 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629092 2024-04-26T21:05:42.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:42.373 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:42.373 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:42.373 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:42.373 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:42.373 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke").st_ino) 2024-04-26T21:05:42.374 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:42.567 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629733 2024-04-26T21:05:42.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:42.581 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:42.581 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:42.582 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:42.582 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:42.582 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/1node").st_ino) 2024-04-26T21:05:42.582 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:42.768 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629734 2024-04-26T21:05:42.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:42.783 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:42.783 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:42.783 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:42.783 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:42.783 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/1node/clusters").st_ino) 2024-04-26T21:05:42.783 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:42.983 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629736 2024-04-26T21:05:42.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:42.997 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:42.997 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:42.997 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:42.997 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:42.998 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/1node/clusters/openstack.yaml").st_ino) 2024-04-26T21:05:42.998 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:43.192 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629739 2024-04-26T21:05:43.197 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:43.208 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:43.208 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:43.208 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:43.208 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:43.208 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/1node/clusters/+").st_ino) 2024-04-26T21:05:43.208 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:43.413 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629737 2024-04-26T21:05:43.418 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:43.428 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:43.428 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:43.428 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:43.428 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:43.428 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/1node/clusters/fixed-1.yaml").st_ino) 2024-04-26T21:05:43.428 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:43.626 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629738 2024-04-26T21:05:43.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:43.642 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:43.642 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:43.642 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:43.642 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:43.642 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/1node/%").st_ino) 2024-04-26T21:05:43.642 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:43.841 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629735 2024-04-26T21:05:43.846 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:43.856 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:43.856 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:43.856 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:43.856 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:43.856 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/1node/tasks").st_ino) 2024-04-26T21:05:43.856 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:44.053 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629740 2024-04-26T21:05:44.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:44.067 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:44.067 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:44.067 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:44.067 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:44.067 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/1node/tasks/ceph-deploy.yaml").st_ino) 2024-04-26T21:05:44.067 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:44.264 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629741 2024-04-26T21:05:44.268 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:44.279 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:44.279 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:44.279 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:44.279 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:44.279 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/systemd").st_ino) 2024-04-26T21:05:44.280 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:44.476 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629774 2024-04-26T21:05:44.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:44.492 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:44.492 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:44.492 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:44.492 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:44.493 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/systemd/distro").st_ino) 2024-04-26T21:05:44.493 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:44.688 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629776 2024-04-26T21:05:44.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:44.702 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:44.702 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:44.702 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:44.702 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:44.702 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/systemd/distro/ubuntu_16.04.yaml").st_ino) 2024-04-26T21:05:44.702 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:44.887 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629778 2024-04-26T21:05:44.892 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:44.902 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:44.902 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:44.902 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:44.902 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:44.902 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/systemd/distro/centos_7.2.yaml").st_ino) 2024-04-26T21:05:44.902 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:45.098 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629777 2024-04-26T21:05:45.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:45.113 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:45.113 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:45.114 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:45.114 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:45.114 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/systemd/%").st_ino) 2024-04-26T21:05:45.114 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:45.301 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629775 2024-04-26T21:05:45.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:45.316 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:45.316 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:45.316 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:45.316 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:45.316 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/systemd/tasks").st_ino) 2024-04-26T21:05:45.316 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:45.516 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629779 2024-04-26T21:05:45.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:45.530 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:45.530 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:45.531 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:45.531 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:45.531 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/systemd/tasks/systemd.yaml").st_ino) 2024-04-26T21:05:45.531 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:45.729 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629780 2024-04-26T21:05:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:45.743 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:45.743 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:45.743 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:45.743 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:45.743 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic").st_ino) 2024-04-26T21:05:45.744 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:45.938 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629742 2024-04-26T21:05:45.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:45.953 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:45.954 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:45.954 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:45.954 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:45.954 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/clusters").st_ino) 2024-04-26T21:05:45.954 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:46.151 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629744 2024-04-26T21:05:46.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:46.167 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:46.167 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:46.167 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:46.167 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:46.167 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/clusters/fixed-3-cephfs.yaml").st_ino) 2024-04-26T21:05:46.167 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:46.364 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629745 2024-04-26T21:05:46.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:46.379 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:46.379 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:46.379 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:46.379 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:46.379 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/fs").st_ino) 2024-04-26T21:05:46.379 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:46.575 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629746 2024-04-26T21:05:46.579 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:46.590 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:46.590 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:46.590 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:46.590 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:46.590 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/fs/btrfs.yaml").st_ino) 2024-04-26T21:05:46.590 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:46.789 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629747 2024-04-26T21:05:46.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:46.804 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:46.804 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:46.805 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:46.805 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:46.805 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/%").st_ino) 2024-04-26T21:05:46.805 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:47.003 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629743 2024-04-26T21:05:47.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:47.018 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:47.018 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:47.018 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:47.018 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:47.018 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks").st_ino) 2024-04-26T21:05:47.018 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:47.218 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629748 2024-04-26T21:05:47.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:47.232 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:47.232 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:47.232 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:47.232 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:47.232 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/libcephfs_interface_tests.yaml").st_ino) 2024-04-26T21:05:47.233 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:47.431 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629757 2024-04-26T21:05:47.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:47.446 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:47.446 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:47.446 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:47.446 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:47.446 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/rados_cache_snaps.yaml").st_ino) 2024-04-26T21:05:47.446 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:47.644 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629761 2024-04-26T21:05:47.649 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:47.659 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:47.659 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:47.659 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:47.659 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:47.659 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/rados_cls_all.yaml").st_ino) 2024-04-26T21:05:47.659 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:47.858 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629762 2024-04-26T21:05:47.863 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:47.874 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:47.874 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:47.874 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:47.874 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:47.874 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/rbd_cli_import_export.yaml").st_ino) 2024-04-26T21:05:47.875 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:47.961 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:05:47.969 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:05:48.077 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629767 2024-04-26T21:05:48.082 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:48.093 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:48.093 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:48.093 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:48.093 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:48.093 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/kclient_workunit_suites_fsstress.yaml").st_ino) 2024-04-26T21:05:48.093 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:48.287 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629755 2024-04-26T21:05:48.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:48.303 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:48.304 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:48.304 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:48.304 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:48.304 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/cfuse_workunit_suites_pjd.yaml").st_ino) 2024-04-26T21:05:48.304 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:48.505 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629752 2024-04-26T21:05:48.509 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:48.520 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:48.520 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:48.520 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:48.520 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:48.520 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/mon_thrash.yaml").st_ino) 2024-04-26T21:05:48.520 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:48.714 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629758 2024-04-26T21:05:48.719 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:48.729 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:48.730 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:48.730 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:48.730 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:48.730 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/kclient_workunit_direct_io.yaml").st_ino) 2024-04-26T21:05:48.730 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:48.934 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629753 2024-04-26T21:05:48.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:48.950 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:48.950 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:48.950 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:48.950 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:48.950 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/rbd_fsx.yaml").st_ino) 2024-04-26T21:05:48.950 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:49.146 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629768 2024-04-26T21:05:49.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:49.160 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:49.160 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:49.160 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:49.161 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:49.161 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/rbd_python_api_tests.yaml").st_ino) 2024-04-26T21:05:49.161 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:49.357 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629769 2024-04-26T21:05:49.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:49.373 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:49.373 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:49.373 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:49.373 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:49.373 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/rbd_workunit_suites_iozone.yaml").st_ino) 2024-04-26T21:05:49.373 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:49.567 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629770 2024-04-26T21:05:49.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:49.581 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:49.582 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:49.582 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:49.582 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:49.582 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/kclient_workunit_suites_dbench.yaml").st_ino) 2024-04-26T21:05:49.582 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:49.778 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629754 2024-04-26T21:05:49.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:49.793 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:49.793 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:49.793 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:49.793 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:49.793 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/rbd_api_tests.yaml").st_ino) 2024-04-26T21:05:49.793 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:49.990 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629766 2024-04-26T21:05:49.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:50.005 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:50.005 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:50.005 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:50.006 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:50.006 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/cfuse_workunit_suites_blogbench.yaml").st_ino) 2024-04-26T21:05:50.006 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:50.203 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629749 2024-04-26T21:05:50.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:50.218 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:50.218 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:50.218 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:50.218 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:50.219 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/rados_ec_snaps.yaml").st_ino) 2024-04-26T21:05:50.219 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:50.415 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629763 2024-04-26T21:05:50.419 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:50.430 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:50.430 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:50.431 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:50.431 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:50.431 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/rgw_ec_s3tests.yaml").st_ino) 2024-04-26T21:05:50.431 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:50.634 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629771 2024-04-26T21:05:50.639 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:50.650 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:50.651 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:50.651 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:50.651 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:50.651 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/kclient_workunit_suites_pjd.yaml").st_ino) 2024-04-26T21:05:50.651 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:50.832 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629756 2024-04-26T21:05:50.837 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:50.847 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:50.847 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:50.847 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:50.847 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:50.848 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/rados_workunit_loadgen_mix.yaml").st_ino) 2024-04-26T21:05:50.848 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:51.049 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629765 2024-04-26T21:05:51.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:51.064 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:51.064 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:51.065 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:51.065 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:51.065 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/rados_python.yaml").st_ino) 2024-04-26T21:05:51.065 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:51.261 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629764 2024-04-26T21:05:51.266 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:51.277 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:51.278 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:51.278 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:51.278 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:51.278 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/rgw_swift.yaml").st_ino) 2024-04-26T21:05:51.278 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:51.473 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629773 2024-04-26T21:05:51.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:51.490 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:51.490 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:51.490 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:51.490 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:51.490 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/rados_bench.yaml").st_ino) 2024-04-26T21:05:51.490 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:51.688 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629760 2024-04-26T21:05:51.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:51.703 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:51.703 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:51.703 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:51.703 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:51.703 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/rados_api_tests.yaml").st_ino) 2024-04-26T21:05:51.703 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:51.900 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629759 2024-04-26T21:05:51.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:51.915 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:51.915 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:51.915 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:51.915 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:51.915 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/cfuse_workunit_suites_fsstress.yaml").st_ino) 2024-04-26T21:05:51.915 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:52.111 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629750 2024-04-26T21:05:52.116 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:52.126 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:52.126 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:52.126 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:52.126 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:52.126 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/rgw_s3tests.yaml").st_ino) 2024-04-26T21:05:52.126 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:52.324 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629772 2024-04-26T21:05:52.328 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:52.338 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:52.338 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:52.338 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:52.338 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:52.338 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/suites/smoke/basic/tasks/cfuse_workunit_suites_iozone.yaml").st_ino) 2024-04-26T21:05:52.339 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:52.534 INFO:teuthology.orchestra.run.smithi028.stdout:1099511629751 2024-04-26T21:05:52.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:52.548 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:52.548 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:52.548 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:52.548 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:52.549 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/machine_types").st_ino) 2024-04-26T21:05:52.549 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:52.745 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628439 2024-04-26T21:05:52.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:52.760 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:52.760 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:52.760 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:52.760 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:52.760 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/machine_types/vps.yaml").st_ino) 2024-04-26T21:05:52.760 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:52.954 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628442 2024-04-26T21:05:52.959 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:52.970 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:52.970 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:52.970 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:52.971 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:52.971 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/machine_types/schedule_rados_ovh.sh").st_ino) 2024-04-26T21:05:52.971 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:53.166 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628441 2024-04-26T21:05:53.170 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:53.181 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:53.181 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:53.181 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:53.181 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:53.181 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/machine_types/schedule_rados.sh").st_ino) 2024-04-26T21:05:53.181 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:53.377 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628440 2024-04-26T21:05:53.382 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:53.392 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:53.392 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:53.392 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:53.392 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:53.393 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/mon_kv_backend").st_ino) 2024-04-26T21:05:53.393 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:53.587 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628443 2024-04-26T21:05:53.591 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:53.602 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:53.602 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:53.602 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:53.602 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:53.602 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/mon_kv_backend/leveldb.yaml").st_ino) 2024-04-26T21:05:53.602 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:53.802 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628444 2024-04-26T21:05:53.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:53.818 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:53.819 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:53.819 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:53.819 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:53.819 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/mon_kv_backend/rocksdb.yaml").st_ino) 2024-04-26T21:05:53.819 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:54.015 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628445 2024-04-26T21:05:54.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:54.031 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:54.031 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:54.032 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:54.032 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:54.032 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/README").st_ino) 2024-04-26T21:05:54.032 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:54.229 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628339 2024-04-26T21:05:54.232 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:54.243 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:54.243 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:54.243 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:54.244 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:54.244 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks").st_ino) 2024-04-26T21:05:54.244 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:54.439 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630075 2024-04-26T21:05:54.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:54.454 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:54.454 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:54.454 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:54.454 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:54.454 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/rebuild_mondb.py").st_ino) 2024-04-26T21:05:54.454 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:54.663 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630189 2024-04-26T21:05:54.667 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:54.678 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:54.678 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:54.678 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:54.678 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:54.678 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/workunit.py").st_ino) 2024-04-26T21:05:54.678 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:54.871 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630227 2024-04-26T21:05:54.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:54.887 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:54.887 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:54.887 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:54.887 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:54.887 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/osd_backfill.py").st_ino) 2024-04-26T21:05:54.887 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:55.084 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630172 2024-04-26T21:05:55.090 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:55.101 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:55.101 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:55.101 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:55.101 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:55.102 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/calamari_setup.py").st_ino) 2024-04-26T21:05:55.102 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:55.296 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630100 2024-04-26T21:05:55.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:55.312 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:55.312 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:55.312 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:55.312 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:55.312 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/object_source_down.py").st_ino) 2024-04-26T21:05:55.312 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:55.507 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630170 2024-04-26T21:05:55.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:55.522 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:55.522 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:55.522 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:55.522 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:55.522 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/populate_rbd_pool.py").st_ino) 2024-04-26T21:05:55.522 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:55.719 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630177 2024-04-26T21:05:55.723 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:55.733 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:55.733 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:55.733 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:55.734 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:55.734 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/util").st_ino) 2024-04-26T21:05:55.734 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:55.931 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630217 2024-04-26T21:05:55.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:55.945 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:55.945 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:55.945 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:55.946 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:55.946 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/util/rgw.py").st_ino) 2024-04-26T21:05:55.946 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:56.145 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630220 2024-04-26T21:05:56.149 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:56.159 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:56.159 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:56.159 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:56.159 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:56.159 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/util/__init__.py").st_ino) 2024-04-26T21:05:56.159 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:56.356 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630218 2024-04-26T21:05:56.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:56.370 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:56.370 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:56.370 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:56.370 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:56.371 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/util/rados.py").st_ino) 2024-04-26T21:05:56.371 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:56.565 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630219 2024-04-26T21:05:56.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:56.580 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:56.580 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:56.581 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:56.581 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:56.581 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/util/test").st_ino) 2024-04-26T21:05:56.581 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:56.780 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630221 2024-04-26T21:05:56.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:56.795 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:56.795 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:56.795 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:56.796 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:56.796 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/util/test/__init__.py").st_ino) 2024-04-26T21:05:56.796 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:56.991 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630222 2024-04-26T21:05:56.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:57.007 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:57.007 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:57.007 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:57.007 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:57.007 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/util/test/test_rados.py").st_ino) 2024-04-26T21:05:57.007 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:57.195 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630223 2024-04-26T21:05:57.200 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:57.210 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:57.210 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:57.211 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:57.211 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:57.211 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/osd_failsafe_enospc.py").st_ino) 2024-04-26T21:05:57.211 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:57.406 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630173 2024-04-26T21:05:57.410 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:57.420 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:57.421 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:57.421 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:57.421 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:57.421 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/osd_recovery.py").st_ino) 2024-04-26T21:05:57.421 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:57.613 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630174 2024-04-26T21:05:57.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:57.628 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:57.628 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:57.628 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:57.628 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:57.628 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/die_on_err.py").st_ino) 2024-04-26T21:05:57.629 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:57.826 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630144 2024-04-26T21:05:57.831 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:57.842 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:57.842 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:57.842 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:57.842 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:57.842 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/calamari_nosetests.py").st_ino) 2024-04-26T21:05:57.842 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:58.037 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630099 2024-04-26T21:05:58.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:58.051 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:58.052 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:58.052 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:58.052 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:58.052 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/admin_socket.py").st_ino) 2024-04-26T21:05:58.052 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:58.248 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630077 2024-04-26T21:05:58.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:58.262 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:58.262 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:58.263 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:58.263 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:58.263 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/mgr").st_ino) 2024-04-26T21:05:58.263 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:58.459 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630158 2024-04-26T21:05:58.463 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:58.474 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:58.474 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:58.474 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:58.474 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:58.474 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/mgr/__init__.py").st_ino) 2024-04-26T21:05:58.474 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:58.669 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630159 2024-04-26T21:05:58.674 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:58.684 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:58.684 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:58.684 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:58.684 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:58.684 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/mgr/test_failover.py").st_ino) 2024-04-26T21:05:58.684 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:58.878 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630161 2024-04-26T21:05:58.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:58.894 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:58.894 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:58.894 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:58.895 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:58.895 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/mgr/mgr_test_case.py").st_ino) 2024-04-26T21:05:58.895 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:59.100 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630160 2024-04-26T21:05:59.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:59.115 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:59.116 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:59.116 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:59.116 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:59.116 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/thrash_pool_snaps.py").st_ino) 2024-04-26T21:05:59.116 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:59.315 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630213 2024-04-26T21:05:59.319 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:59.330 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:59.330 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:59.330 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:59.330 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:59.330 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/userdata_teardown.yaml").st_ino) 2024-04-26T21:05:59.330 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:59.526 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630216 2024-04-26T21:05:59.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:59.540 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:59.540 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:59.540 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:59.540 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:59.540 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/radosbenchsweep.py").st_ino) 2024-04-26T21:05:59.540 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:59.735 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630181 2024-04-26T21:05:59.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:59.750 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:59.750 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:59.750 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:59.750 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:59.750 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/mds_thrash.py").st_ino) 2024-04-26T21:05:59.750 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:05:59.946 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630156 2024-04-26T21:05:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:05:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:05:59.963 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:05:59.963 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:05:59.963 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:05:59.963 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:05:59.963 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/rbd.py").st_ino) 2024-04-26T21:05:59.963 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:00.158 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630185 2024-04-26T21:06:00.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:00.173 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:00.173 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:00.173 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:00.173 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:00.173 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/rgw.py").st_ino) 2024-04-26T21:06:00.173 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:00.368 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630197 2024-04-26T21:06:00.374 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:00.382 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:00.382 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:00.383 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:00.383 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:00.383 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/teuthology_integration.py").st_ino) 2024-04-26T21:06:00.383 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:00.581 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630211 2024-04-26T21:06:00.585 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:00.596 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:00.596 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:00.596 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:00.596 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:00.596 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/mon_seesaw.py").st_ino) 2024-04-26T21:06:00.596 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:00.791 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630167 2024-04-26T21:06:00.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:00.806 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:00.806 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:00.806 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:00.806 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:00.806 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/repair_test.py").st_ino) 2024-04-26T21:06:00.806 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:01.003 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630193 2024-04-26T21:06:01.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:01.017 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:01.018 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:01.018 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:01.018 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:01.018 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/watch_notify_same_primary.py").st_ino) 2024-04-26T21:06:01.018 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:01.198 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630225 2024-04-26T21:06:01.203 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:01.214 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:01.214 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:01.214 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:01.214 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:01.214 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/peer.py").st_ino) 2024-04-26T21:06:01.214 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:01.408 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630175 2024-04-26T21:06:01.413 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:01.424 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:01.424 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:01.424 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:01.424 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:01.424 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/restart.py").st_ino) 2024-04-26T21:06:01.424 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:01.619 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630196 2024-04-26T21:06:01.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:01.633 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:01.633 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:01.633 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:01.633 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:01.633 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/ceph.py").st_ino) 2024-04-26T21:06:01.633 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:01.829 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630101 2024-04-26T21:06:01.833 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:01.843 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:01.843 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:01.843 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:01.844 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:01.844 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/tests").st_ino) 2024-04-26T21:06:01.844 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:02.042 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630206 2024-04-26T21:06:02.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:02.057 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:02.057 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:02.057 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:02.057 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:02.057 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/tests/test_buildpackages.py").st_ino) 2024-04-26T21:06:02.057 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:02.255 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630208 2024-04-26T21:06:02.259 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:02.269 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:02.269 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:02.269 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:02.269 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:02.269 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/tests/test_radosgw_admin.py").st_ino) 2024-04-26T21:06:02.269 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:02.468 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630210 2024-04-26T21:06:02.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:02.484 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:02.484 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:02.484 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:02.484 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:02.485 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/tests/test_devstack.py").st_ino) 2024-04-26T21:06:02.485 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:02.682 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630209 2024-04-26T21:06:02.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:02.697 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:02.697 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:02.697 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:02.697 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:02.697 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/tests/__init__.py").st_ino) 2024-04-26T21:06:02.697 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:02.892 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630207 2024-04-26T21:06:02.897 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:02.908 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:02.908 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:02.908 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:02.908 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:02.908 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/rgw_logsocket.py").st_ino) 2024-04-26T21:06:02.908 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:03.105 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630198 2024-04-26T21:06:03.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:03.122 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:03.122 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:03.122 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:03.122 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:03.122 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/radosgw_admin.py").st_ino) 2024-04-26T21:06:03.122 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:03.301 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630182 2024-04-26T21:06:03.306 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:03.315 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:03.315 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:03.315 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:03.315 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:03.315 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/ceph_client.py").st_ino) 2024-04-26T21:06:03.315 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:03.500 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630102 2024-04-26T21:06:03.505 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:03.515 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:03.515 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:03.515 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:03.515 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:03.515 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/__init__.py").st_ino) 2024-04-26T21:06:03.515 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:03.710 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630076 2024-04-26T21:06:03.714 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:03.725 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:03.725 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:03.725 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:03.725 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:03.725 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/aver.py").st_ino) 2024-04-26T21:06:03.725 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:03.907 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630080 2024-04-26T21:06:03.911 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:03.922 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:03.922 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:03.922 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:03.922 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:03.922 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cram.py").st_ino) 2024-04-26T21:06:03.922 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:04.121 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630141 2024-04-26T21:06:04.126 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:04.137 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:04.137 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:04.137 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:04.137 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:04.137 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs").st_ino) 2024-04-26T21:06:04.137 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:04.331 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630108 2024-04-26T21:06:04.335 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:04.345 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:04.346 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:04.346 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:04.346 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:04.346 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_misc.py").st_ino) 2024-04-26T21:06:04.346 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:04.544 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630132 2024-04-26T21:06:04.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:04.560 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:04.560 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:04.560 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:04.560 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:04.560 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_client_limits.py").st_ino) 2024-04-26T21:06:04.561 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:04.755 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630118 2024-04-26T21:06:04.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:04.770 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:04.770 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:04.770 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:04.770 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:04.770 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_pool_perm.py").st_ino) 2024-04-26T21:06:04.770 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:04.964 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630133 2024-04-26T21:06:04.969 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:04.979 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:04.979 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:04.979 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:04.979 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:04.979 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_data_scan.py").st_ino) 2024-04-26T21:06:04.979 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:05.177 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630122 2024-04-26T21:06:05.182 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:05.192 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:05.192 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:05.192 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:05.192 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:05.192 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_client_recovery.py").st_ino) 2024-04-26T21:06:05.192 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:05.388 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630119 2024-04-26T21:06:05.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:05.403 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:05.403 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:05.403 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:05.404 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:05.404 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_journal_migration.py").st_ino) 2024-04-26T21:06:05.404 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:05.600 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630129 2024-04-26T21:06:05.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:05.614 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:05.615 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:05.615 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:05.615 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:05.615 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_dump_tree.py").st_ino) 2024-04-26T21:06:05.615 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:05.814 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630123 2024-04-26T21:06:05.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:05.828 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:05.828 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:05.828 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:05.828 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:05.828 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_mantle.py").st_ino) 2024-04-26T21:06:05.828 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:06.023 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630131 2024-04-26T21:06:06.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:06.039 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:06.039 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:06.039 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:06.039 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:06.039 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_sessionmap.py").st_ino) 2024-04-26T21:06:06.039 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:06.234 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630136 2024-04-26T21:06:06.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:06.249 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:06.249 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:06.249 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:06.250 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:06.250 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/__init__.py").st_ino) 2024-04-26T21:06:06.250 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:06.446 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630109 2024-04-26T21:06:06.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:06.461 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:06.461 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:06.461 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:06.461 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:06.461 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_journal_repair.py").st_ino) 2024-04-26T21:06:06.461 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:06.657 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630130 2024-04-26T21:06:06.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:06.672 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:06.672 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:06.672 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:06.672 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:06.672 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/kernel_mount.py").st_ino) 2024-04-26T21:06:06.672 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:06.868 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630113 2024-04-26T21:06:06.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:06.883 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:06.884 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:06.884 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:06.884 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:06.884 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_cap_flush.py").st_ino) 2024-04-26T21:06:06.884 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:07.081 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630117 2024-04-26T21:06:07.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:07.096 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:07.096 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:07.096 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:07.096 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:07.096 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_readahead.py").st_ino) 2024-04-26T21:06:07.096 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:07.292 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630134 2024-04-26T21:06:07.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:07.307 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:07.307 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:07.307 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:07.307 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:07.307 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_backtrace.py").st_ino) 2024-04-26T21:06:07.307 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:07.509 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630116 2024-04-26T21:06:07.514 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:07.524 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:07.525 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:07.525 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:07.525 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:07.525 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_auto_repair.py").st_ino) 2024-04-26T21:06:07.525 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:07.704 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630115 2024-04-26T21:06:07.708 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:07.718 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:07.718 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:07.718 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:07.718 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:07.718 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_strays.py").st_ino) 2024-04-26T21:06:07.718 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:07.913 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630137 2024-04-26T21:06:07.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:07.928 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:07.928 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:07.928 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:07.928 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:07.928 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_volume_client.py").st_ino) 2024-04-26T21:06:07.928 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:08.123 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630138 2024-04-26T21:06:08.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:08.139 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:08.139 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:08.139 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:08.139 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:08.139 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/cephfs_test_case.py").st_ino) 2024-04-26T21:06:08.139 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:08.333 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630110 2024-04-26T21:06:08.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:08.348 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:08.348 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:08.348 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:08.348 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:08.348 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_fragment.py").st_ino) 2024-04-26T21:06:08.348 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:08.545 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630127 2024-04-26T21:06:08.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:08.563 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:08.563 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:08.563 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:08.563 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:08.563 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_failover.py").st_ino) 2024-04-26T21:06:08.563 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:08.759 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630124 2024-04-26T21:06:08.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:08.774 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:08.774 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:08.774 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:08.774 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:08.774 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_full.py").st_ino) 2024-04-26T21:06:08.774 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:08.971 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630128 2024-04-26T21:06:08.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:08.986 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:08.986 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:08.986 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:08.986 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:08.986 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/filesystem.py").st_ino) 2024-04-26T21:06:08.986 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:09.184 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630111 2024-04-26T21:06:09.189 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:09.200 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:09.200 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:09.200 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:09.201 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:09.201 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/mount.py").st_ino) 2024-04-26T21:06:09.201 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:09.396 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630114 2024-04-26T21:06:09.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:09.411 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:09.411 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:09.411 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:09.411 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:09.411 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_damage.py").st_ino) 2024-04-26T21:06:09.412 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:09.609 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630121 2024-04-26T21:06:09.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:09.624 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:09.624 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:09.624 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:09.625 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:09.625 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_scrub_checks.py").st_ino) 2024-04-26T21:06:09.625 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:09.820 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630135 2024-04-26T21:06:09.825 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:09.836 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:09.836 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:09.836 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:09.836 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:09.836 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_config_commands.py").st_ino) 2024-04-26T21:06:09.836 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:10.033 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630120 2024-04-26T21:06:10.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:10.048 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:10.048 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:10.048 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:10.048 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:10.048 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_flush.py").st_ino) 2024-04-26T21:06:10.048 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:10.245 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630125 2024-04-26T21:06:10.250 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:10.260 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:10.260 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:10.260 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:10.260 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:10.260 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/fuse_mount.py").st_ino) 2024-04-26T21:06:10.260 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:10.457 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630112 2024-04-26T21:06:10.461 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:10.472 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:10.473 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:10.473 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:10.473 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:10.473 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs/test_forward_scrub.py").st_ino) 2024-04-26T21:06:10.473 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:10.668 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630126 2024-04-26T21:06:10.673 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:10.682 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:10.683 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:10.683 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:10.683 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:10.683 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/recovery_bench.py").st_ino) 2024-04-26T21:06:10.683 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:10.879 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630190 2024-04-26T21:06:10.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:10.895 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:10.895 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:10.895 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:10.895 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:10.896 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/multibench.py").st_ino) 2024-04-26T21:06:10.896 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:11.086 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630169 2024-04-26T21:06:11.091 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:11.101 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:11.101 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:11.101 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:11.101 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:11.101 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/devstack.py").st_ino) 2024-04-26T21:06:11.101 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:11.300 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630143 2024-04-26T21:06:11.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:11.314 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:11.314 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:11.314 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:11.314 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:11.314 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cephfs_test_runner.py").st_ino) 2024-04-26T21:06:11.314 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:11.510 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630139 2024-04-26T21:06:11.515 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:11.525 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:11.525 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:11.525 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:11.525 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:11.525 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/filestore_idempotent.py").st_ino) 2024-04-26T21:06:11.525 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:11.721 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630149 2024-04-26T21:06:11.726 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:11.736 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:11.737 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:11.737 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:11.737 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:11.737 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/mod_proxy_fcgi.tcp.conf.template").st_ino) 2024-04-26T21:06:11.737 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:11.935 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630163 2024-04-26T21:06:11.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:11.950 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:11.950 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:11.950 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:11.950 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:11.950 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/scrub.py").st_ino) 2024-04-26T21:06:11.950 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:12.150 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630203 2024-04-26T21:06:12.154 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:12.164 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:12.165 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:12.165 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:12.165 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:12.165 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/divergent_priors.py").st_ino) 2024-04-26T21:06:12.165 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:12.358 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630145 2024-04-26T21:06:12.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:12.373 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:12.373 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:12.373 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:12.373 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:12.373 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/samba.py").st_ino) 2024-04-26T21:06:12.373 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:12.562 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630202 2024-04-26T21:06:12.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:12.576 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:12.576 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:12.576 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:12.576 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:12.576 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/watch_notify_stress.py").st_ino) 2024-04-26T21:06:12.576 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:12.771 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630226 2024-04-26T21:06:12.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:12.786 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:12.786 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:12.786 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:12.787 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:12.787 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/boto.cfg.template").st_ino) 2024-04-26T21:06:12.787 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:12.980 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630082 2024-04-26T21:06:12.985 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:12.995 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:12.995 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:12.995 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:12.995 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:12.995 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/buildpackages.py").st_ino) 2024-04-26T21:06:12.995 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:13.190 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630083 2024-04-26T21:06:13.195 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:13.205 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:13.205 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:13.205 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:13.205 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:13.205 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/ceph_test_case.py").st_ino) 2024-04-26T21:06:13.205 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:13.396 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630107 2024-04-26T21:06:13.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:13.411 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:13.411 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:13.412 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:13.412 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:13.412 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/s3roundtrip.py").st_ino) 2024-04-26T21:06:13.412 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:13.607 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630200 2024-04-26T21:06:13.611 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:13.622 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:13.622 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:13.622 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:13.622 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:13.622 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/qemu.py").st_ino) 2024-04-26T21:06:13.622 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:13.817 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630178 2024-04-26T21:06:13.822 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:13.832 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:13.832 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:13.832 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:13.832 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:13.833 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/locktest.py").st_ino) 2024-04-26T21:06:13.833 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:14.027 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630151 2024-04-26T21:06:14.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:14.042 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:14.042 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:14.042 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:14.042 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:14.042 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/manypools.py").st_ino) 2024-04-26T21:06:14.042 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:14.236 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630154 2024-04-26T21:06:14.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:14.251 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:14.252 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:14.252 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:14.252 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:14.252 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/mds_creation_failure.py").st_ino) 2024-04-26T21:06:14.252 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:14.446 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630155 2024-04-26T21:06:14.451 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:14.461 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:14.461 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:14.461 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:14.461 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:14.462 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/apache.conf.template").st_ino) 2024-04-26T21:06:14.462 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:14.660 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630078 2024-04-26T21:06:14.664 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:14.674 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:14.674 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:14.675 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:14.675 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:14.675 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/ec_lost_unfound.py").st_ino) 2024-04-26T21:06:14.675 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:14.878 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630148 2024-04-26T21:06:14.883 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:14.894 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:14.894 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:14.894 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:14.894 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:14.894 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/buildpackages").st_ino) 2024-04-26T21:06:14.894 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:15.089 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630084 2024-04-26T21:06:15.093 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:15.103 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:15.103 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:15.103 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:15.103 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:15.103 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/buildpackages/ubuntu-16.04-user-data.txt").st_ino) 2024-04-26T21:06:15.103 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:15.303 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630097 2024-04-26T21:06:15.309 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:15.319 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:15.319 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:15.319 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:15.319 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:15.319 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/buildpackages/ubuntu-14.04-user-data.txt").st_ino) 2024-04-26T21:06:15.319 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:15.516 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630096 2024-04-26T21:06:15.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:15.531 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:15.532 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:15.532 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:15.532 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:15.532 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/buildpackages/centos-6.5-user-data.txt").st_ino) 2024-04-26T21:06:15.532 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:15.721 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630086 2024-04-26T21:06:15.725 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:15.735 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:15.735 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:15.735 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:15.735 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:15.735 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/buildpackages/centos-7.1-user-data.txt").st_ino) 2024-04-26T21:06:15.736 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:15.932 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630088 2024-04-26T21:06:15.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:15.947 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:15.947 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:15.947 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:15.947 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:15.947 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/buildpackages/make-deb.sh").st_ino) 2024-04-26T21:06:15.947 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:16.142 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630092 2024-04-26T21:06:16.150 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:16.157 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:16.158 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:16.158 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:16.158 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:16.158 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/buildpackages/centos-7.0-user-data.txt").st_ino) 2024-04-26T21:06:16.158 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:16.357 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630087 2024-04-26T21:06:16.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:16.371 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:16.371 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:16.371 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:16.371 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:16.371 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/buildpackages/make-rpm.sh").st_ino) 2024-04-26T21:06:16.371 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:16.566 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630093 2024-04-26T21:06:16.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:16.581 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:16.581 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:16.581 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:16.581 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:16.581 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/buildpackages/opensuse-42.1-user-data.txt").st_ino) 2024-04-26T21:06:16.581 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:16.777 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630094 2024-04-26T21:06:16.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:16.792 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:16.792 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:16.792 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:16.792 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:16.792 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/buildpackages/Makefile").st_ino) 2024-04-26T21:06:16.792 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:16.983 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630085 2024-04-26T21:06:16.987 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:16.997 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:16.997 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:16.997 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:16.997 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:16.997 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/buildpackages/ubuntu-12.04-user-data.txt").st_ino) 2024-04-26T21:06:16.998 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:17.193 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630095 2024-04-26T21:06:17.198 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:17.208 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:17.208 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:17.208 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:17.208 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:17.208 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/buildpackages/common.sh").st_ino) 2024-04-26T21:06:17.208 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:17.405 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630090 2024-04-26T21:06:17.409 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:17.419 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:17.420 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:17.420 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:17.420 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:17.420 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/buildpackages/user-data.txt").st_ino) 2024-04-26T21:06:17.420 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:17.604 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630098 2024-04-26T21:06:17.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:17.619 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:17.619 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:17.619 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:17.619 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:17.619 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/buildpackages/debian-8.0-user-data.txt").st_ino) 2024-04-26T21:06:17.619 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:17.815 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630091 2024-04-26T21:06:17.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:17.829 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:17.829 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:17.829 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:17.830 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:17.830 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/buildpackages/centos-7.2-user-data.txt").st_ino) 2024-04-26T21:06:17.830 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:18.024 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630089 2024-04-26T21:06:18.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:18.039 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:18.039 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:18.039 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:18.039 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:18.039 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/rbd_mirror.py").st_ino) 2024-04-26T21:06:18.039 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:18.109 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:06:18.116 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:06:18.215 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630188 2024-04-26T21:06:18.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:18.228 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:18.228 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:18.229 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:18.229 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:18.229 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/dump_stuck.py").st_ino) 2024-04-26T21:06:18.229 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:18.421 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630147 2024-04-26T21:06:18.427 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:18.438 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:18.438 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:18.438 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:18.439 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:18.439 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/ceph_objectstore_tool.py").st_ino) 2024-04-26T21:06:18.439 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:18.636 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630106 2024-04-26T21:06:18.640 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:18.651 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:18.651 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:18.651 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:18.651 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:18.651 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/rbd_fsx.py").st_ino) 2024-04-26T21:06:18.651 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:18.843 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630187 2024-04-26T21:06:18.848 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:18.859 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:18.859 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:18.859 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:18.859 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:18.859 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/radosgw_agent.py").st_ino) 2024-04-26T21:06:18.859 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:19.055 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630184 2024-04-26T21:06:19.059 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:19.070 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:19.070 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:19.070 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:19.070 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:19.070 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/tgt.py").st_ino) 2024-04-26T21:06:19.070 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:19.269 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630212 2024-04-26T21:06:19.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:19.283 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:19.284 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:19.284 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:19.284 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:19.284 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/vstart_runner.py").st_ino) 2024-04-26T21:06:19.284 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:19.479 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630224 2024-04-26T21:06:19.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:19.495 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:19.495 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:19.495 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:19.495 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:19.495 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/resolve_stuck_peering.py").st_ino) 2024-04-26T21:06:19.495 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:19.693 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630194 2024-04-26T21:06:19.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:19.707 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:19.707 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:19.707 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:19.707 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:19.707 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/rados.py").st_ino) 2024-04-26T21:06:19.707 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:19.900 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630179 2024-04-26T21:06:19.904 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:19.914 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:19.914 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:19.914 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:19.914 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:19.914 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/s3tests.py").st_ino) 2024-04-26T21:06:19.914 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:20.112 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630201 2024-04-26T21:06:20.117 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:20.127 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:20.127 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:20.127 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:20.127 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:20.127 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/userdata_setup.yaml").st_ino) 2024-04-26T21:06:20.127 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:20.322 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630215 2024-04-26T21:06:20.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:20.337 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:20.337 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:20.337 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:20.337 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:20.337 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/lost_unfound.py").st_ino) 2024-04-26T21:06:20.337 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:20.519 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630153 2024-04-26T21:06:20.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:20.533 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:20.534 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:20.534 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:20.534 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:20.534 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/s3readwrite.py").st_ino) 2024-04-26T21:06:20.534 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:20.730 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630199 2024-04-26T21:06:20.734 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:20.744 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:20.744 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:20.744 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:20.745 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:20.745 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/ceph_fuse.py").st_ino) 2024-04-26T21:06:20.745 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:20.938 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630104 2024-04-26T21:06:20.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:20.953 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:20.953 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:20.954 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:20.954 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:20.954 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/rest_api.py").st_ino) 2024-04-26T21:06:20.954 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:21.152 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630195 2024-04-26T21:06:21.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:21.166 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:21.167 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:21.167 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:21.167 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:21.167 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/reg11184.py").st_ino) 2024-04-26T21:06:21.167 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:21.363 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630191 2024-04-26T21:06:21.368 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:21.378 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:21.379 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:21.379 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:21.379 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:21.379 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/rep_lost_unfound_delete.py").st_ino) 2024-04-26T21:06:21.379 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:21.573 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630192 2024-04-26T21:06:21.577 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:21.587 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:21.587 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:21.587 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:21.587 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:21.588 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/mod_proxy_fcgi.uds.conf.template").st_ino) 2024-04-26T21:06:21.588 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:21.783 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630164 2024-04-26T21:06:21.787 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:21.798 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:21.798 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:21.798 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:21.798 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:21.798 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/ceph_deploy.py").st_ino) 2024-04-26T21:06:21.798 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:21.992 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630103 2024-04-26T21:06:21.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:22.006 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:22.006 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:22.006 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:22.007 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:22.007 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/ceph_manager.py").st_ino) 2024-04-26T21:06:22.007 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:22.203 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630105 2024-04-26T21:06:22.207 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:22.218 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:22.218 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:22.218 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:22.218 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:22.218 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/radosbench.py").st_ino) 2024-04-26T21:06:22.218 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:22.412 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630180 2024-04-26T21:06:22.417 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:22.427 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:22.427 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:22.427 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:22.427 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:22.427 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/scrub_test.py").st_ino) 2024-04-26T21:06:22.427 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:22.623 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630204 2024-04-26T21:06:22.628 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:22.638 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:22.638 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:22.639 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:22.639 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:22.639 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/mon_clock_skew_check.py").st_ino) 2024-04-26T21:06:22.639 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:22.836 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630165 2024-04-26T21:06:22.841 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:22.851 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:22.851 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:22.852 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:22.852 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:22.852 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/kclient.py").st_ino) 2024-04-26T21:06:22.852 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:23.051 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630150 2024-04-26T21:06:23.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:23.066 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:23.066 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:23.067 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:23.067 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:23.067 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/omapbench.py").st_ino) 2024-04-26T21:06:23.067 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:23.261 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630171 2024-04-26T21:06:23.265 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:23.277 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:23.277 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:23.277 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:23.277 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:23.277 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/autotest.py").st_ino) 2024-04-26T21:06:23.277 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:23.474 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630079 2024-04-26T21:06:23.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:23.489 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:23.489 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:23.489 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:23.489 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:23.489 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/mod_fastcgi.conf.template").st_ino) 2024-04-26T21:06:23.490 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:23.684 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630162 2024-04-26T21:06:23.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:23.699 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:23.699 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:23.699 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:23.699 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:23.699 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/peering_speed_test.py").st_ino) 2024-04-26T21:06:23.699 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:23.894 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630176 2024-04-26T21:06:23.898 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:23.908 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:23.908 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:23.908 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:23.908 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:23.908 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/mon_recovery.py").st_ino) 2024-04-26T21:06:23.908 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:24.104 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630166 2024-04-26T21:06:24.109 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:24.120 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:24.120 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:24.120 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:24.120 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:24.120 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/cifs_mount.py").st_ino) 2024-04-26T21:06:24.120 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:24.315 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630140 2024-04-26T21:06:24.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:24.334 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:24.335 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:24.335 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:24.335 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:24.335 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/rbd_fio.py").st_ino) 2024-04-26T21:06:24.335 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:24.525 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630186 2024-04-26T21:06:24.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:24.543 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:24.543 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:24.543 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:24.543 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:24.543 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/logrotate.conf").st_ino) 2024-04-26T21:06:24.543 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:24.738 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630152 2024-04-26T21:06:24.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:24.752 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:24.752 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:24.752 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:24.752 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:24.752 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/mon_thrash.py").st_ino) 2024-04-26T21:06:24.752 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:24.950 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630168 2024-04-26T21:06:24.952 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:24.962 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:24.962 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:24.962 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:24.962 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:24.962 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/metadata.yaml").st_ino) 2024-04-26T21:06:24.962 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:25.159 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630157 2024-04-26T21:06:25.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:25.175 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:25.175 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:25.175 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:25.175 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:25.175 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/divergent_priors2.py").st_ino) 2024-04-26T21:06:25.175 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:25.369 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630146 2024-04-26T21:06:25.373 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:25.384 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:25.384 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:25.384 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:25.384 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:25.384 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/blktrace.py").st_ino) 2024-04-26T21:06:25.384 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:25.578 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630081 2024-04-26T21:06:25.582 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:25.593 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:25.594 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:25.594 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:25.594 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:25.594 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/thrashosds.py").st_ino) 2024-04-26T21:06:25.594 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:25.788 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630214 2024-04-26T21:06:25.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:25.802 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:25.802 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:25.802 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:25.802 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:25.802 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/systemd.py").st_ino) 2024-04-26T21:06:25.802 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:25.996 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630205 2024-04-26T21:06:26.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:26.010 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:26.010 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:26.010 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:26.011 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:26.011 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/radosgw_admin_rest.py").st_ino) 2024-04-26T21:06:26.011 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:26.206 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630183 2024-04-26T21:06:26.210 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:26.220 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:26.220 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:26.220 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:26.220 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:26.220 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/tasks/create_verify_lfn_objects.py").st_ino) 2024-04-26T21:06:26.220 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:26.416 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630142 2024-04-26T21:06:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:26.430 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:26.430 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:26.430 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:26.430 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:26.430 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/timezone").st_ino) 2024-04-26T21:06:26.430 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:26.625 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630228 2024-04-26T21:06:26.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:26.640 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:26.640 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:26.640 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:26.640 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:26.640 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/timezone/random.yaml").st_ino) 2024-04-26T21:06:26.640 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:26.823 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630231 2024-04-26T21:06:26.827 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:26.837 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:26.837 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:26.837 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:26.837 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:26.837 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/timezone/eastern.yaml").st_ino) 2024-04-26T21:06:26.837 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:27.031 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630229 2024-04-26T21:06:27.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:27.045 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:27.045 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:27.046 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:27.046 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:27.046 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/timezone/pacific.yaml").st_ino) 2024-04-26T21:06:27.046 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:27.238 INFO:teuthology.orchestra.run.smithi028.stdout:1099511630230 2024-04-26T21:06:27.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:27.253 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:27.253 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:27.254 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:27.254 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:27.254 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/.gitignore").st_ino) 2024-04-26T21:06:27.254 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:27.448 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628338 2024-04-26T21:06:27.452 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:27.463 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:27.463 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:27.463 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:27.464 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:27.464 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/debug").st_ino) 2024-04-26T21:06:27.464 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:27.645 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628386 2024-04-26T21:06:27.650 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:27.659 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:27.659 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:27.659 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:27.659 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:27.659 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/debug/openstack-30G.yaml").st_ino) 2024-04-26T21:06:27.660 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:27.856 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628390 2024-04-26T21:06:27.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:27.871 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:27.871 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:27.871 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:27.871 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:27.871 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/debug/buildpackages.yaml").st_ino) 2024-04-26T21:06:27.872 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:28.067 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628387 2024-04-26T21:06:28.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:28.082 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:28.083 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:28.083 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:28.083 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:28.083 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/debug/mds_client.yaml").st_ino) 2024-04-26T21:06:28.083 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:28.279 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628388 2024-04-26T21:06:28.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:28.294 DEBUG:teuthology.orchestra.run.smithi028:> stdin-killer -- python3 -c ' 2024-04-26T21:06:28.294 DEBUG:teuthology.orchestra.run.smithi028:> import os 2024-04-26T21:06:28.294 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2024-04-26T21:06:28.294 DEBUG:teuthology.orchestra.run.smithi028:> 2024-04-26T21:06:28.294 DEBUG:teuthology.orchestra.run.smithi028:> print(os.lstat("/home/ubuntu/cephtest/mnt.0/./ceph-qa-suite/debug/openstack-15G.yaml").st_ino) 2024-04-26T21:06:28.294 DEBUG:teuthology.orchestra.run.smithi028:> ' 2024-04-26T21:06:28.488 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628389 2024-04-26T21:06:28.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T21:06:28.504 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:06:28.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.754+0000 7f6724800640 1 Processor -- start 2024-04-26T21:06:28.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.754+0000 7f6724800640 1 -- start start 2024-04-26T21:06:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.756+0000 7f6724800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f671c0b6810 0x7f671c15d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:28.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.756+0000 7f6724800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f671c0d0a30 con 0x7f671c146980 2024-04-26T21:06:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.756+0000 7f6724800640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f671c15e230 con 0x7f671c0740e0 2024-04-26T21:06:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.756+0000 7f6724800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f671c15e510 con 0x7f671c0b6810 2024-04-26T21:06:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.757+0000 7f6721e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f671c146980 0x7f671c0b6100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:42322/0 (socket says 172.21.15.28:42322) 2024-04-26T21:06:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.757+0000 7f6721e00640 1 -- 172.21.15.28:0/2063916933 learned_addr learned my addr 172.21.15.28:0/2063916933 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.757+0000 7f6721400640 1 --2- 172.21.15.28:0/2063916933 >> v2:172.21.15.28:3300/0 conn(0x7f671c0b6810 0x7f671c15d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:28.758 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.757+0000 7f6721400640 1 -- 172.21.15.28:0/2063916933 >> v1:172.21.15.103:6789/0 conn(0x7f671c0740e0 legacy=0x7f671c071ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:28.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.757+0000 7f6721400640 1 -- 172.21.15.28:0/2063916933 >> v1:172.21.15.28:6789/0 conn(0x7f671c146980 legacy=0x7f671c0b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:28.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.757+0000 7f6721400640 1 -- 172.21.15.28:0/2063916933 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671c073600 con 0x7f671c0b6810 2024-04-26T21:06:28.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.757+0000 7f6720a00640 1 -- 172.21.15.28:0/2063916933 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1177580358 0 0) 0x7f6710001110 con 0x7f671c146980 2024-04-26T21:06:28.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.758+0000 7f6721400640 1 --2- 172.21.15.28:0/2063916933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f671c0b6810 0x7f671c15d5b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f670c0097a0 tx=0x7f670c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5755438b60374901 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:28.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.759+0000 7f6720a00640 1 -- 172.21.15.28:0/2063916933 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c037070 con 0x7f671c0b6810 2024-04-26T21:06:28.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.759+0000 7f6720a00640 1 -- 172.21.15.28:0/2063916933 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f670c02ae50 con 0x7f671c0b6810 2024-04-26T21:06:28.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.759+0000 7f6720a00640 1 -- 172.21.15.28:0/2063916933 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f670c032660 con 0x7f671c0b6810 2024-04-26T21:06:28.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.759+0000 7f6724800640 1 -- 172.21.15.28:0/2063916933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f671c0b6810 msgr2=0x7f671c15d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:28.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.759+0000 7f6724800640 1 --2- 172.21.15.28:0/2063916933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f671c0b6810 0x7f671c15d5b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f670c0097a0 tx=0x7f670c02a140 comp rx=0 tx=0).stop 2024-04-26T21:06:28.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.759+0000 7f6724800640 1 -- 172.21.15.28:0/2063916933 shutdown_connections 2024-04-26T21:06:28.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.759+0000 7f6724800640 1 --2- 172.21.15.28:0/2063916933 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f671c0b6810 0x7f671c15d5b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:28.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.759+0000 7f6724800640 1 -- 172.21.15.28:0/2063916933 >> 172.21.15.28:0/2063916933 conn(0x7f671c06e220 msgr2=0x7f671c06e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:28.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.759+0000 7f6724800640 1 -- 172.21.15.28:0/2063916933 shutdown_connections 2024-04-26T21:06:28.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.759+0000 7f6724800640 1 -- 172.21.15.28:0/2063916933 wait complete. 2024-04-26T21:06:28.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.761+0000 7f6724800640 1 Processor -- start 2024-04-26T21:06:28.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.761+0000 7f6724800640 1 -- start start 2024-04-26T21:06:28.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.762+0000 7f6724800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f671c0740e0 0x7f671c15a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:28.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.762+0000 7f6724800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f671c0b6810 0x7f671c13e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:28.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.762+0000 7f6724800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f671c146980 0x7f671c13eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:28.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.762+0000 7f6724800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f671c15daf0 con 0x7f671c146980 2024-04-26T21:06:28.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.762+0000 7f6724800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f671c073bc0 con 0x7f671c0740e0 2024-04-26T21:06:28.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.762+0000 7f6724800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f671c15dc70 con 0x7f671c0b6810 2024-04-26T21:06:28.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.763+0000 7f6722800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f671c146980 0x7f671c13eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:28.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.763+0000 7f6722800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f671c146980 0x7f671c13eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40088/0 (socket says 172.21.15.28:40088) 2024-04-26T21:06:28.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.763+0000 7f6722800640 1 -- 172.21.15.28:0/1212845718 learned_addr learned my addr 172.21.15.28:0/1212845718 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:28.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.763+0000 7f6721400640 1 --2- 172.21.15.28:0/1212845718 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f671c0b6810 0x7f671c13e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:28.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.763+0000 7f6721e00640 1 --2- 172.21.15.28:0/1212845718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f671c0740e0 0x7f671c15a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:28.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.763+0000 7f6722800640 1 -- 172.21.15.28:0/1212845718 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f671c0b6810 msgr2=0x7f671c13e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:28.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.763+0000 7f6722800640 1 --2- 172.21.15.28:0/1212845718 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f671c0b6810 0x7f671c13e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:28.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.763+0000 7f6722800640 1 -- 172.21.15.28:0/1212845718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f671c0740e0 msgr2=0x7f671c15a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:28.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.763+0000 7f6722800640 1 --2- 172.21.15.28:0/1212845718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f671c0740e0 0x7f671c15a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:28.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.763+0000 7f6722800640 1 -- 172.21.15.28:0/1212845718 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671c13f1b0 con 0x7f671c146980 2024-04-26T21:06:28.765 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.763+0000 7f6721400640 1 --2- 172.21.15.28:0/1212845718 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f671c0b6810 0x7f671c13e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:28.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.764+0000 7f6722800640 1 --2- 172.21.15.28:0/1212845718 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f671c146980 0x7f671c13eb30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f671800c780 tx=0x7f671800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:28.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.764+0000 7f6716000640 1 -- 172.21.15.28:0/1212845718 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718004b60 con 0x7f671c146980 2024-04-26T21:06:28.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.764+0000 7f6716000640 1 -- 172.21.15.28:0/1212845718 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6718004d00 con 0x7f671c146980 2024-04-26T21:06:28.766 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.764+0000 7f6716000640 1 -- 172.21.15.28:0/1212845718 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671800aa90 con 0x7f671c146980 2024-04-26T21:06:28.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.764+0000 7f6724800640 1 -- 172.21.15.28:0/1212845718 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671c13f490 con 0x7f671c146980 2024-04-26T21:06:28.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.764+0000 7f6724800640 1 -- 172.21.15.28:0/1212845718 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f671c13fa50 con 0x7f671c146980 2024-04-26T21:06:28.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.765+0000 7f6724800640 1 -- 172.21.15.28:0/1212845718 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f671c06a400 con 0x7f671c146980 2024-04-26T21:06:28.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.766+0000 7f6716000640 1 -- 172.21.15.28:0/1212845718 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f671800ac30 con 0x7f671c146980 2024-04-26T21:06:28.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.768+0000 7f6716000640 1 --2- 172.21.15.28:0/1212845718 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f670006c8f0 0x7f670006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:28.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.768+0000 7f6716000640 1 -- 172.21.15.28:0/1212845718 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f671808bf50 con 0x7f671c146980 2024-04-26T21:06:28.769 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.768+0000 7f6721e00640 1 --2- 172.21.15.28:0/1212845718 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f670006c8f0 0x7f670006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:28.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.769+0000 7f6721e00640 1 --2- 172.21.15.28:0/1212845718 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f670006c8f0 0x7f670006edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6710000f60 tx=0x7f6710007c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:28.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:28.774+0000 7f6716000640 1 -- 172.21.15.28:0/1212845718 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6718020070 con 0x7f671c146980 2024-04-26T21:06:29.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.020+0000 7f6724800640 1 -- 172.21.15.28:0/1212845718 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f671c06ea60 con 0x7f671c146980 2024-04-26T21:06:29.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.021+0000 7f6716000640 1 -- 172.21.15.28:0/1212845718 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5184 (secure 0 0 0) 0x7f671805a2a0 con 0x7f671c146980 2024-04-26T21:06:29.022 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T21:06:29.022 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/356870645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":356870645},{"type":"v1","addr":"172.21.15.28:6835","nonce":356870645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4829,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2957511478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":2957511478},{"type":"v1","addr":"172.21.15.103:6835","nonce":2957511478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/2060632176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2060632176},{"type":"v1","addr":"172.21.15.103:6834","nonce":2060632176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:58:43.534341+0000","modified":"2024-04-26T20:58:46.581816+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5571},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6837/2997047727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":2997047727},{"type":"v1","addr":"172.21.15.28:6837","nonce":2997047727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5571,"qdb_cluster":[5571]},"id":3}]} 2024-04-26T21:06:29.022 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 26 2024-04-26T21:06:29.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.023+0000 7f6724800640 1 -- 172.21.15.28:0/1212845718 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f670006c8f0 msgr2=0x7f670006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:29.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.023+0000 7f6724800640 1 --2- 172.21.15.28:0/1212845718 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f670006c8f0 0x7f670006edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6710000f60 tx=0x7f6710007c00 comp rx=0 tx=0).stop 2024-04-26T21:06:29.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.023+0000 7f6724800640 1 -- 172.21.15.28:0/1212845718 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f671c146980 msgr2=0x7f671c13eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:29.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.023+0000 7f6724800640 1 --2- 172.21.15.28:0/1212845718 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f671c146980 0x7f671c13eb30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f671800c780 tx=0x7f671800cc40 comp rx=0 tx=0).stop 2024-04-26T21:06:29.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.023+0000 7f6724800640 1 -- 172.21.15.28:0/1212845718 shutdown_connections 2024-04-26T21:06:29.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.023+0000 7f6724800640 1 --2- 172.21.15.28:0/1212845718 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f670006c8f0 0x7f670006edb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:29.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.024+0000 7f6724800640 1 --2- 172.21.15.28:0/1212845718 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f671c146980 0x7f671c13eb30 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:29.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.024+0000 7f6724800640 1 --2- 172.21.15.28:0/1212845718 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f671c0b6810 0x7f671c13e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:29.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.024+0000 7f6724800640 1 --2- 172.21.15.28:0/1212845718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f671c0740e0 0x7f671c15a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:29.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.024+0000 7f6724800640 1 -- 172.21.15.28:0/1212845718 >> 172.21.15.28:0/1212845718 conn(0x7f671c06e220 msgr2=0x7f671c15bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:29.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.024+0000 7f6724800640 1 -- 172.21.15.28:0/1212845718 shutdown_connections 2024-04-26T21:06:29.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.024+0000 7f6724800640 1 -- 172.21.15.28:0/1212845718 wait complete. 2024-04-26T21:06:29.046 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump tree /ceph-qa-suite/suites/smoke/1node 2024-04-26T21:06:29.262 INFO:teuthology.orchestra.run.smithi028.stdout:[{"ino":1099511629739,"rdev":0,"ctime":"2024-04-26T20:59:27.176487+0000","btime":"2024-04-26T20:59:27.172421+0000","mode":33188,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":0,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":7,"pool_ns":""},"old_pools":[],"size":155,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T20:59:27.176487+0000","atime":"2024-04-26T20:59:27.172421+0000","time_warp_seq":0,"change_attr":1,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":155,"rfiles":1,"rsubdirs":0,"rsnaps":0,"rctime":"2024-04-26T20:59:27.176487+0000"},"accounted_rstat":{"version":0,"rbytes":155,"rfiles":1,"rsubdirs":0,"rsnaps":0,"rctime":"2024-04-26T20:59:27.176487+0000"},"version":10,"file_data_version":0,"xattr_version":1,"backtrace_version":6,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"lock":0,"caps":1,"dirtyparent":1,"dirty":1,"authpin":0},"nref":3,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":5751,"pending":"pAsLsXsFscr","issued":"pAsLsXsFscr","wanted":"-","last_sent":3}],"loner":5751,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[],"is_subvolume":false},{"ino":1099511629737,"rdev":0,"ctime":"2024-04-26T20:59:27.147340+0000","btime":"2024-04-26T20:59:27.147340+0000","mode":33188,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":0,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":7,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T20:59:27.147340+0000","atime":"2024-04-26T20:59:27.147340+0000","time_warp_seq":0,"change_attr":0,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":1,"rsubdirs":0,"rsnaps":0,"rctime":"0.000000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":1,"rsubdirs":0,"rsnaps":0,"rctime":"0.000000"},"version":8,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"lock":0,"caps":1,"dirtyparent":1,"dirty":1,"authpin":0},"nref":3,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":5751,"pending":"pAsLsXsFscr","issued":"pAsLsXsFscr","wanted":"-","last_sent":3}],"loner":5751,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[],"is_subvolume":false},{"ino":1099511629738,"rdev":0,"ctime":"2024-04-26T20:59:27.160401+0000","btime":"2024-04-26T20:59:27.160401+0000","mode":41471,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":0,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":7,"pool_ns":""},"old_pools":[],"size":33,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T20:59:27.160401+0000","atime":"2024-04-26T20:59:27.160401+0000","time_warp_seq":0,"change_attr":0,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":33,"rfiles":1,"rsubdirs":0,"rsnaps":0,"rctime":"0.000000"},"accounted_rstat":{"version":0,"rbytes":33,"rfiles":1,"rsubdirs":0,"rsnaps":0,"rctime":"0.000000"},"version":4,"file_data_version":0,"xattr_version":1,"backtrace_version":4,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"../../../../clusters/fixed-1.yaml","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"caps":1,"dirtyparent":1,"dirty":1},"nref":3,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":5751,"pending":"pAsLsXsFscr","issued":"pAsLsXsFscr","wanted":"-","last_sent":2}],"loner":-1,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[],"is_subvolume":false},{"ino":1099511629736,"rdev":0,"ctime":"2024-04-26T20:59:27.172421+0000","btime":"2024-04-26T20:59:27.138794+0000","mode":16877,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T20:59:27.172421+0000","atime":"2024-04-26T20:59:27.138794+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-26T20:59:27.172421+0000","num_files":3,"num_subdirs":0,"change_attr":3},"rstat":{"version":0,"rbytes":188,"rfiles":3,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T20:59:27.176487+0000"},"accounted_rstat":{"version":0,"rbytes":188,"rfiles":3,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T20:59:27.176487+0000"},"version":20,"file_data_version":0,"xattr_version":1,"backtrace_version":4,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1,"waiter":0,"authpin":0},"nref":4,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","type":"ifile","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":5751,"pending":"pAsLsXsFsx","issued":"pAsLsXsFsx","wanted":"AsLsXsFsx","last_sent":10}],"loner":5751,"want_loner":5751,"mds_caps_wanted":[],"dirfrags":[{"path":"/ceph-qa-suite/suites/smoke/1node/clusters","dirfrag":"0x100000007a8","snapid_first":2,"projected_version":"11","version":"11","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"dirty":1,"authpin":0},"nref":2,"dentries":[{"path":"ceph-qa-suite/suites/smoke/1node/clusters/openstack.yaml","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511629739,"remote_type":"","version":10,"projected_version":10,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"ceph-qa-suite/suites/smoke/1node/clusters/+","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511629737,"remote_type":"","version":8,"projected_version":8,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"ceph-qa-suite/suites/smoke/1node/clusters/fixed-1.yaml","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511629738,"remote_type":"","version":4,"projected_version":4,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false},{"ino":1099511629735,"rdev":0,"ctime":"2024-04-26T20:59:27.122643+0000","btime":"2024-04-26T20:59:27.122643+0000","mode":33188,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":0,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":7,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T20:59:27.122643+0000","atime":"2024-04-26T20:59:27.122643+0000","time_warp_seq":0,"change_attr":0,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":1,"rsubdirs":0,"rsnaps":0,"rctime":"0.000000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":1,"rsubdirs":0,"rsnaps":0,"rctime":"0.000000"},"version":16,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"lock":0,"caps":1,"dirtyparent":1,"dirty":1,"authpin":0},"nref":3,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":5751,"pending":"pAsLsXsFscr","issued":"pAsLsXsFscr","wanted":"-","last_sent":3}],"loner":5751,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[],"is_subvolume":false},{"ino":1099511629741,"rdev":0,"ctime":"2024-04-26T20:59:27.199637+0000","btime":"2024-04-26T20:59:27.195227+0000","mode":33188,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":0,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":7,"pool_ns":""},"old_pools":[],"size":216,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T20:59:27.199637+0000","atime":"2024-04-26T20:59:27.195227+0000","time_warp_seq":0,"change_attr":1,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":216,"rfiles":1,"rsubdirs":0,"rsnaps":0,"rctime":"2024-04-26T20:59:27.199637+0000"},"accounted_rstat":{"version":0,"rbytes":216,"rfiles":1,"rsubdirs":0,"rsnaps":0,"rctime":"2024-04-26T20:59:27.199637+0000"},"version":4,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"lock":0,"caps":1,"dirtyparent":1,"dirty":1,"authpin":0},"nref":3,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":5751,"pending":"pAsLsXsFscr","issued":"pAsLsXsFscr","wanted":"-","last_sent":3}],"loner":5751,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[],"is_subvolume":false},{"ino":1099511629740,"rdev":0,"ctime":"2024-04-26T20:59:27.195227+0000","btime":"2024-04-26T20:59:27.185736+0000","mode":16877,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T20:59:27.195227+0000","atime":"2024-04-26T20:59:27.185736+0000","time_warp_seq":0,"change_attr":1,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-26T20:59:27.195227+0000","num_files":1,"num_subdirs":0,"change_attr":1},"rstat":{"version":0,"rbytes":216,"rfiles":1,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T20:59:27.199637+0000"},"accounted_rstat":{"version":0,"rbytes":216,"rfiles":1,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T20:59:27.199637+0000"},"version":22,"file_data_version":0,"xattr_version":1,"backtrace_version":12,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1,"waiter":0,"authpin":0},"nref":4,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","type":"ifile","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":5751,"pending":"pAsLsXsFsx","issued":"pAsLsXsFsx","wanted":"AsLsXsFsx","last_sent":8}],"loner":5751,"want_loner":5751,"mds_caps_wanted":[],"dirfrags":[{"path":"/ceph-qa-suite/suites/smoke/1node/tasks","dirfrag":"0x100000007ac","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"dirty":1,"authpin":0},"nref":2,"dentries":[{"path":"ceph-qa-suite/suites/smoke/1node/tasks/ceph-deploy.yaml","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511629741,"remote_type":"","version":4,"projected_version":4,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false},{"ino":1099511629734,"rdev":0,"ctime":"2024-04-26T20:59:27.185736+0000","btime":"2024-04-26T20:59:27.112401+0000","mode":16877,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T20:59:27.185736+0000","atime":"2024-04-26T20:59:27.112401+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-26T20:59:27.185736+0000","num_files":1,"num_subdirs":2,"change_attr":3},"rstat":{"version":1,"rbytes":404,"rfiles":5,"rsubdirs":3,"rsnaps":0,"rctime":"2024-04-26T20:59:27.199637+0000"},"accounted_rstat":{"version":1,"rbytes":404,"rfiles":5,"rsubdirs":3,"rsnaps":0,"rctime":"2024-04-26T20:59:27.199637+0000"},"version":41,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirtyscattered":0,"request":0,"lock":0,"dirfrag":1,"caps":1,"dirtyrstat":0,"dirtyparent":1,"dirty":1,"waiter":0,"authpin":0},"nref":4,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","type":"ifile","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":5751,"pending":"pAsLsXsFsx","issued":"pAsLsXsFsx","wanted":"AsLsXsFsx","last_sent":12}],"loner":5751,"want_loner":5751,"mds_caps_wanted":[],"dirfrags":[{"path":"/ceph-qa-suite/suites/smoke/1node","dirfrag":"0x100000007a6","snapid_first":2,"projected_version":"24","version":"24","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"dirty":1,"authpin":0},"nref":2,"dentries":[{"path":"ceph-qa-suite/suites/smoke/1node/clusters","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511629736,"remote_type":"","version":20,"projected_version":20,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"ceph-qa-suite/suites/smoke/1node/%","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511629735,"remote_type":"","version":16,"projected_version":16,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"ceph-qa-suite/suites/smoke/1node/tasks","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511629740,"remote_type":"","version":22,"projected_version":22,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false}] 2024-04-26T21:06:29.266 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 155, "rctime": "2024-04-26T20:59:27.176487+0000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 0, "version": 0 }, "atime": "2024-04-26T20:59:27.172421+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 6, "btime": "2024-04-26T20:59:27.172421+0000", "change_attr": 1, "client_caps": [ { "client_id": 5751, "issued": "pAsLsXsFscr", "last_sent": 3, "pending": "pAsLsXsFscr", "wanted": "-" } ], "client_ranges": [], "ctime": "2024-04-26T20:59:27.176487+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 0, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1316, "ino": 1099511629739, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 7, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": 5751, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 33188, "mtime": "2024-04-26T20:59:27.176487+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 3, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 1, "dirty": 1, "dirtyparent": 1, "lock": 0, "ptrwaiter": 0, "request": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 155, "rctime": "2024-04-26T20:59:27.176487+0000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 0, "version": 0 }, "size": 155, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 10, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] }, { "accounted_rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 0, "version": 0 }, "atime": "2024-04-26T20:59:27.147340+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-26T20:59:27.147340+0000", "change_attr": 0, "client_caps": [ { "client_id": 5751, "issued": "pAsLsXsFscr", "last_sent": 3, "pending": "pAsLsXsFscr", "wanted": "-" } ], "client_ranges": [], "ctime": "2024-04-26T20:59:27.147340+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 0, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1316, "ino": 1099511629737, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 7, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": 5751, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 33188, "mtime": "2024-04-26T20:59:27.147340+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 3, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 1, "dirty": 1, "dirtyparent": 1, "lock": 0, "ptrwaiter": 0, "request": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 0, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 8, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] }, { "accounted_rstat": { "rbytes": 33, "rctime": "0.000000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 0, "version": 0 }, "atime": "2024-04-26T20:59:27.160401+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 4, "btime": "2024-04-26T20:59:27.160401+0000", "change_attr": 0, "client_caps": [ { "client_id": 5751, "issued": "pAsLsXsFscr", "last_sent": 2, "pending": "pAsLsXsFscr", "wanted": "-" } ], "client_ranges": [], "ctime": "2024-04-26T20:59:27.160401+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 0, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1316, "ino": 1099511629738, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 7, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 41471, "mtime": "2024-04-26T20:59:27.160401+0000", "nestlock": {}, "nlink": 1, "nref": 3, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "caps": 1, "dirty": 1, "dirtyparent": 1, "request": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 33, "rctime": "0.000000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 0, "version": 0 }, "size": 33, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "../../../../clusters/fixed-1.yaml", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] }, { "accounted_rstat": { "rbytes": 188, "rctime": "2024-04-26T20:59:27.176487+0000", "rfiles": 3, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2024-04-26T20:59:27.138794+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 4, "btime": "2024-04-26T20:59:27.138794+0000", "change_attr": 3, "client_caps": [ { "client_id": 5751, "issued": "pAsLsXsFsx", "last_sent": 10, "pending": "pAsLsXsFsx", "wanted": "AsLsXsFsx" } ], "client_ranges": [], "ctime": "2024-04-26T20:59:27.172421+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dentries": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511629739, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "ceph-qa-suite/suites/smoke/1node/clusters/openstack.yaml", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 10, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 10, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511629737, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "ceph-qa-suite/suites/smoke/1node/clusters/+", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 8, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 8, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511629738, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "ceph-qa-suite/suites/smoke/1node/clusters/fixed-1.yaml", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 4, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x100000007a8", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/ceph-qa-suite/suites/smoke/1node/clusters", "pins": { "authpin": 0, "child": 1, "dirty": 1 }, "projected_version": "11", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "11" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 3, "mtime": "2024-04-26T20:59:27.172421+0000", "num_files": 3, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "type": "ifile", "xlock_by": {} }, "flocklock": {}, "gid": 1316, "ino": 1099511629736, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 5751, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-04-26T20:59:27.172421+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "lock": 0, "request": 0, "waiter": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 188, "rctime": "2024-04-26T20:59:27.176487+0000", "rfiles": 3, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 20, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": 5751, "xattr_version": 1, "xattrlock": {}, "xattrs": [] }, { "accounted_rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 0, "version": 0 }, "atime": "2024-04-26T20:59:27.122643+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-26T20:59:27.122643+0000", "change_attr": 0, "client_caps": [ { "client_id": 5751, "issued": "pAsLsXsFscr", "last_sent": 3, "pending": "pAsLsXsFscr", "wanted": "-" } ], "client_ranges": [], "ctime": "2024-04-26T20:59:27.122643+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 0, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1316, "ino": 1099511629735, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 7, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": 5751, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 33188, "mtime": "2024-04-26T20:59:27.122643+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 3, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 1, "dirty": 1, "dirtyparent": 1, "lock": 0, "ptrwaiter": 0, "request": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 0, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 16, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] }, { "accounted_rstat": { "rbytes": 216, "rctime": "2024-04-26T20:59:27.199637+0000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 0, "version": 0 }, "atime": "2024-04-26T20:59:27.195227+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-26T20:59:27.195227+0000", "change_attr": 1, "client_caps": [ { "client_id": 5751, "issued": "pAsLsXsFscr", "last_sent": 3, "pending": "pAsLsXsFscr", "wanted": "-" } ], "client_ranges": [], "ctime": "2024-04-26T20:59:27.199637+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 0, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1316, "ino": 1099511629741, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 7, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": 5751, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 33188, "mtime": "2024-04-26T20:59:27.199637+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 3, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 1, "dirty": 1, "dirtyparent": 1, "lock": 0, "ptrwaiter": 0, "request": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 216, "rctime": "2024-04-26T20:59:27.199637+0000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 0, "version": 0 }, "size": 216, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] }, { "accounted_rstat": { "rbytes": 216, "rctime": "2024-04-26T20:59:27.199637+0000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2024-04-26T20:59:27.185736+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 12, "btime": "2024-04-26T20:59:27.185736+0000", "change_attr": 1, "client_caps": [ { "client_id": 5751, "issued": "pAsLsXsFsx", "last_sent": 8, "pending": "pAsLsXsFsx", "wanted": "AsLsXsFsx" } ], "client_ranges": [], "ctime": "2024-04-26T20:59:27.195227+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dentries": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511629741, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "ceph-qa-suite/suites/smoke/1node/tasks/ceph-deploy.yaml", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 4, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x100000007ac", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/ceph-qa-suite/suites/smoke/1node/tasks", "pins": { "authpin": 0, "child": 1, "dirty": 1 }, "projected_version": "5", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 1, "mtime": "2024-04-26T20:59:27.195227+0000", "num_files": 1, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "type": "ifile", "xlock_by": {} }, "flocklock": {}, "gid": 1316, "ino": 1099511629740, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 5751, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-04-26T20:59:27.195227+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "lock": 0, "request": 0, "waiter": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 216, "rctime": "2024-04-26T20:59:27.199637+0000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 22, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": 5751, "xattr_version": 1, "xattrlock": {}, "xattrs": [] }, { "accounted_rstat": { "rbytes": 404, "rctime": "2024-04-26T20:59:27.199637+0000", "rfiles": 5, "rsnaps": 0, "rsubdirs": 3, "version": 1 }, "atime": "2024-04-26T20:59:27.112401+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-26T20:59:27.112401+0000", "change_attr": 3, "client_caps": [ { "client_id": 5751, "issued": "pAsLsXsFsx", "last_sent": 12, "pending": "pAsLsXsFsx", "wanted": "AsLsXsFsx" } ], "client_ranges": [], "ctime": "2024-04-26T20:59:27.185736+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dentries": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511629736, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "ceph-qa-suite/suites/smoke/1node/clusters", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 20, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 20, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511629735, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "ceph-qa-suite/suites/smoke/1node/%", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 16, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 16, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511629740, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "ceph-qa-suite/suites/smoke/1node/tasks", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 22, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 22, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x100000007a6", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/ceph-qa-suite/suites/smoke/1node", "pins": { "authpin": 0, "child": 1, "dirty": 1 }, "projected_version": "24", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "24" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 3, "mtime": "2024-04-26T20:59:27.185736+0000", "num_files": 1, "num_subdirs": 2, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "type": "ifile", "xlock_by": {} }, "flocklock": {}, "gid": 1316, "ino": 1099511629734, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 5751, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-04-26T20:59:27.185736+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "dirtyrstat": 0, "dirtyscattered": 0, "lock": 0, "request": 0, "waiter": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 404, "rctime": "2024-04-26T20:59:27.199637+0000", "rfiles": 5, "rsnaps": 0, "rsubdirs": 3, "version": 1 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 41, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": 5751, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } ] 2024-04-26T21:06:29.266 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:06:29.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.476+0000 7fa316200640 1 Processor -- start 2024-04-26T21:06:29.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.476+0000 7fa316200640 1 -- start start 2024-04-26T21:06:29.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.477+0000 7fa316200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa3100cead0 0x7fa3101632b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:29.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.477+0000 7fa316200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa31006d220 con 0x7fa3100b2430 2024-04-26T21:06:29.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.477+0000 7fa316200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa31006d500 con 0x7fa310162e70 2024-04-26T21:06:29.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.477+0000 7fa316200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fa3101637f0 con 0x7fa3100cead0 2024-04-26T21:06:29.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.477+0000 7fa30f400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa3100b2430 0x7fa31006c6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:42330/0 (socket says 172.21.15.28:42330) 2024-04-26T21:06:29.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.477+0000 7fa30f400640 1 -- 172.21.15.28:0/675383127 learned_addr learned my addr 172.21.15.28:0/675383127 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:29.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.478+0000 7fa30fe00640 1 --2- 172.21.15.28:0/675383127 >> v2:172.21.15.103:3300/0 conn(0x7fa3100cead0 0x7fa3101632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:29.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.478+0000 7fa30e000640 1 -- 172.21.15.28:0/675383127 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 341476208 0 0) 0x7fa31006d220 con 0x7fa3100b2430 2024-04-26T21:06:29.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.478+0000 7fa30e000640 1 -- 172.21.15.28:0/675383127 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa2f4003520 con 0x7fa3100b2430 2024-04-26T21:06:29.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.478+0000 7fa30e000640 1 -- 172.21.15.28:0/675383127 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3467094794 0 0) 0x7fa2f4003520 con 0x7fa3100b2430 2024-04-26T21:06:29.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.478+0000 7fa30e000640 1 -- 172.21.15.28:0/675383127 >> v2:172.21.15.103:3300/0 conn(0x7fa3100cead0 msgr2=0x7fa3101632b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:29.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.478+0000 7fa30e000640 1 --2- 172.21.15.28:0/675383127 >> v2:172.21.15.103:3300/0 conn(0x7fa3100cead0 0x7fa3101632b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:29.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.478+0000 7fa30e000640 1 -- 172.21.15.28:0/675383127 >> v1:172.21.15.103:6789/0 conn(0x7fa310162e70 legacy=0x7fa31015ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:29.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.478+0000 7fa30e000640 1 -- 172.21.15.28:0/675383127 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31006d7e0 con 0x7fa3100b2430 2024-04-26T21:06:29.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.478+0000 7fa30e000640 1 -- 172.21.15.28:0/675383127 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fa300001670 con 0x7fa3100b2430 2024-04-26T21:06:29.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.478+0000 7fa30e000640 1 -- 172.21.15.28:0/675383127 >> v1:172.21.15.28:6789/0 conn(0x7fa3100b2430 legacy=0x7fa31006c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:29.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.478+0000 7fa30e000640 1 --2- 172.21.15.28:0/675383127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2f40039b0 0x7fa2f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:29.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.478+0000 7fa30e000640 1 --2- 172.21.15.28:0/675383127 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2f4004800 0x7fa2f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:29.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.478+0000 7fa30e000640 1 --2- 172.21.15.28:0/675383127 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa2f40051c0 0x7fa2f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:29.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.478+0000 7fa30e000640 1 -- 172.21.15.28:0/675383127 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa3101637f0 con 0x7fa2f4004800 2024-04-26T21:06:29.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.478+0000 7fa30e000640 1 -- 172.21.15.28:0/675383127 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa2f400c060 con 0x7fa2f40039b0 2024-04-26T21:06:29.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.478+0000 7fa30e000640 1 -- 172.21.15.28:0/675383127 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa2f400c260 con 0x7fa2f40051c0 2024-04-26T21:06:29.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.479+0000 7fa30ea00640 1 --2- 172.21.15.28:0/675383127 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2f4004800 0x7fa2f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:29.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.479+0000 7fa30f400640 1 --2- 172.21.15.28:0/675383127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2f40039b0 0x7fa2f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:29.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.479+0000 7fa30ea00640 1 -- 172.21.15.28:0/675383127 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa2f40051c0 msgr2=0x7fa2f400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:29.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.479+0000 7fa30ea00640 1 --2- 172.21.15.28:0/675383127 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa2f40051c0 0x7fa2f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:29.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.479+0000 7fa30ea00640 1 -- 172.21.15.28:0/675383127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2f40039b0 msgr2=0x7fa2f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:29.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.479+0000 7fa30ea00640 1 --2- 172.21.15.28:0/675383127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2f40039b0 0x7fa2f4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:29.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.479+0000 7fa30ea00640 1 -- 172.21.15.28:0/675383127 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2f4003520 con 0x7fa2f4004800 2024-04-26T21:06:29.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.479+0000 7fa30f400640 1 --2- 172.21.15.28:0/675383127 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2f40039b0 0x7fa2f4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:29.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.479+0000 7fa30fe00640 1 -- 172.21.15.28:0/675383127 reap_dead start 2024-04-26T21:06:29.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.480+0000 7fa30ea00640 1 --2- 172.21.15.28:0/675383127 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2f4004800 0x7fa2f4004c80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8001750 tx=0x7fa2f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337bafc3acb2911e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:29.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.480+0000 7fa30e000640 1 -- 172.21.15.28:0/675383127 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8001370 con 0x7fa2f4004800 2024-04-26T21:06:29.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.481+0000 7fa30e000640 1 -- 172.21.15.28:0/675383127 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2f80052d0 con 0x7fa2f4004800 2024-04-26T21:06:29.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.481+0000 7fa30e000640 1 -- 172.21.15.28:0/675383127 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f802abe0 con 0x7fa2f4004800 2024-04-26T21:06:29.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.481+0000 7fa316200640 1 -- 172.21.15.28:0/675383127 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2f4004800 msgr2=0x7fa2f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:29.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.481+0000 7fa316200640 1 --2- 172.21.15.28:0/675383127 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2f4004800 0x7fa2f4004c80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8001750 tx=0x7fa2f4005830 comp rx=0 tx=0).stop 2024-04-26T21:06:29.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.481+0000 7fa316200640 1 -- 172.21.15.28:0/675383127 shutdown_connections 2024-04-26T21:06:29.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.481+0000 7fa316200640 1 --2- 172.21.15.28:0/675383127 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2f4004800 0x7fa2f4004c80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:29.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.481+0000 7fa316200640 1 -- 172.21.15.28:0/675383127 >> 172.21.15.28:0/675383127 conn(0x7fa31009ba50 msgr2=0x7fa31009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:29.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.481+0000 7fa316200640 1 -- 172.21.15.28:0/675383127 shutdown_connections 2024-04-26T21:06:29.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.481+0000 7fa316200640 1 -- 172.21.15.28:0/675383127 wait complete. 2024-04-26T21:06:29.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.483+0000 7fa316200640 1 Processor -- start 2024-04-26T21:06:29.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.483+0000 7fa316200640 1 -- start start 2024-04-26T21:06:29.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.484+0000 7fa316200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2f4004800 0x7fa310156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:29.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.484+0000 7fa316200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3101571d0 0x7fa3100d2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:29.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.484+0000 7fa316200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3100b0130 0x7fa3100b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:29.486 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.484+0000 7fa316200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa3101632b0 con 0x7fa3100b0130 2024-04-26T21:06:29.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.484+0000 7fa316200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa310163c30 con 0x7fa2f4004800 2024-04-26T21:06:29.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.484+0000 7fa316200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa310163430 con 0x7fa3101571d0 2024-04-26T21:06:29.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.485+0000 7fa30fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3100b0130 0x7fa3100b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:29.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.485+0000 7fa30fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3100b0130 0x7fa3100b05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:40110/0 (socket says 172.21.15.28:40110) 2024-04-26T21:06:29.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.485+0000 7fa30fe00640 1 -- 172.21.15.28:0/32444357 learned_addr learned my addr 172.21.15.28:0/32444357 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:29.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.485+0000 7fa30f400640 1 --2- 172.21.15.28:0/32444357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2f4004800 0x7fa310156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:29.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.485+0000 7fa30ea00640 1 --2- 172.21.15.28:0/32444357 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3101571d0 0x7fa3100d2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:29.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.485+0000 7fa30fe00640 1 -- 172.21.15.28:0/32444357 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3101571d0 msgr2=0x7fa3100d2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:29.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.485+0000 7fa30fe00640 1 --2- 172.21.15.28:0/32444357 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3101571d0 0x7fa3100d2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:29.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.485+0000 7fa30fe00640 1 -- 172.21.15.28:0/32444357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2f4004800 msgr2=0x7fa310156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:29.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.485+0000 7fa30fe00640 1 --2- 172.21.15.28:0/32444357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2f4004800 0x7fa310156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:29.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.485+0000 7fa30fe00640 1 -- 172.21.15.28:0/32444357 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3100b5bd0 con 0x7fa3100b0130 2024-04-26T21:06:29.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.485+0000 7fa30f400640 1 --2- 172.21.15.28:0/32444357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2f4004800 0x7fa310156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:06:29.488 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.485+0000 7fa30fe00640 1 --2- 172.21.15.28:0/32444357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3100b0130 0x7fa3100b05e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa3100c6890 tx=0x7fa30800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:29.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.486+0000 7fa307400640 1 -- 172.21.15.28:0/32444357 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308018020 con 0x7fa3100b0130 2024-04-26T21:06:29.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.486+0000 7fa307400640 1 -- 172.21.15.28:0/32444357 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa30800bab0 con 0x7fa3100b0130 2024-04-26T21:06:29.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.486+0000 7fa316200640 1 -- 172.21.15.28:0/32444357 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3100b7dc0 con 0x7fa3100b0130 2024-04-26T21:06:29.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.486+0000 7fa307400640 1 -- 172.21.15.28:0/32444357 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308014400 con 0x7fa3100b0130 2024-04-26T21:06:29.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.486+0000 7fa316200640 1 -- 172.21.15.28:0/32444357 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3100b8380 con 0x7fa3100b0130 2024-04-26T21:06:29.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.487+0000 7fa316200640 1 -- 172.21.15.28:0/32444357 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa31006a400 con 0x7fa3100b0130 2024-04-26T21:06:29.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.488+0000 7fa307400640 1 -- 172.21.15.28:0/32444357 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa308005d10 con 0x7fa3100b0130 2024-04-26T21:06:29.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.489+0000 7fa307400640 1 --2- 172.21.15.28:0/32444357 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa2ec06c920 0x7fa2ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:29.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.489+0000 7fa307400640 1 -- 172.21.15.28:0/32444357 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fa30808c0f0 con 0x7fa3100b0130 2024-04-26T21:06:29.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.489+0000 7fa30f400640 1 --2- 172.21.15.28:0/32444357 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa2ec06c920 0x7fa2ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:29.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.490+0000 7fa30f400640 1 --2- 172.21.15.28:0/32444357 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa2ec06c920 0x7fa2ec06ede0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa300001eb0 tx=0x7fa300001d00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:29.494 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.493+0000 7fa307400640 1 -- 172.21.15.28:0/32444357 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa308022030 con 0x7fa3100b0130 2024-04-26T21:06:29.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.740+0000 7fa316200640 1 -- 172.21.15.28:0/32444357 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa310070a00 con 0x7fa3100b0130 2024-04-26T21:06:29.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.741+0000 7fa307400640 1 -- 172.21.15.28:0/32444357 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5184 (secure 0 0 0) 0x7fa30805a4d0 con 0x7fa3100b0130 2024-04-26T21:06:29.742 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T21:06:29.743 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/356870645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":356870645},{"type":"v1","addr":"172.21.15.28:6835","nonce":356870645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4829,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2957511478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":2957511478},{"type":"v1","addr":"172.21.15.103:6835","nonce":2957511478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/2060632176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2060632176},{"type":"v1","addr":"172.21.15.103:6834","nonce":2060632176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:58:43.534341+0000","modified":"2024-04-26T20:58:46.581816+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5571},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6837/2997047727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":2997047727},{"type":"v1","addr":"172.21.15.28:6837","nonce":2997047727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5571,"qdb_cluster":[5571]},"id":3}]} 2024-04-26T21:06:29.743 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 26 2024-04-26T21:06:29.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.744+0000 7fa316200640 1 -- 172.21.15.28:0/32444357 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa2ec06c920 msgr2=0x7fa2ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:29.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.744+0000 7fa316200640 1 --2- 172.21.15.28:0/32444357 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa2ec06c920 0x7fa2ec06ede0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa300001eb0 tx=0x7fa300001d00 comp rx=0 tx=0).stop 2024-04-26T21:06:29.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.744+0000 7fa316200640 1 -- 172.21.15.28:0/32444357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3100b0130 msgr2=0x7fa3100b05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:29.745 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.744+0000 7fa316200640 1 --2- 172.21.15.28:0/32444357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3100b0130 0x7fa3100b05e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa3100c6890 tx=0x7fa30800eea0 comp rx=0 tx=0).stop 2024-04-26T21:06:29.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.744+0000 7fa316200640 1 -- 172.21.15.28:0/32444357 shutdown_connections 2024-04-26T21:06:29.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.744+0000 7fa316200640 1 --2- 172.21.15.28:0/32444357 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa2ec06c920 0x7fa2ec06ede0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:29.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.744+0000 7fa316200640 1 --2- 172.21.15.28:0/32444357 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3100b0130 0x7fa3100b05e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:29.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.744+0000 7fa316200640 1 --2- 172.21.15.28:0/32444357 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3101571d0 0x7fa3100d2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:29.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.744+0000 7fa316200640 1 --2- 172.21.15.28:0/32444357 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa2f4004800 0x7fa310156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:29.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.744+0000 7fa316200640 1 -- 172.21.15.28:0/32444357 >> 172.21.15.28:0/32444357 conn(0x7fa31009ba50 msgr2=0x7fa3100d1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:29.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.744+0000 7fa316200640 1 -- 172.21.15.28:0/32444357 shutdown_connections 2024-04-26T21:06:29.746 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:29.744+0000 7fa316200640 1 -- 172.21.15.28:0/32444357 wait complete. 2024-04-26T21:06:29.766 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump tree /ceph-qa-suite/suites/smoke/1node 1 2024-04-26T21:06:29.996 INFO:teuthology.orchestra.run.smithi028.stdout:[{"ino":1099511629736,"rdev":0,"ctime":"2024-04-26T20:59:27.172421+0000","btime":"2024-04-26T20:59:27.138794+0000","mode":16877,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T20:59:27.172421+0000","atime":"2024-04-26T20:59:27.138794+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-26T20:59:27.172421+0000","num_files":3,"num_subdirs":0,"change_attr":3},"rstat":{"version":0,"rbytes":188,"rfiles":3,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T20:59:27.176487+0000"},"accounted_rstat":{"version":0,"rbytes":188,"rfiles":3,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T20:59:27.176487+0000"},"version":20,"file_data_version":0,"xattr_version":1,"backtrace_version":4,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1,"waiter":0,"authpin":0},"nref":4,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","type":"ifile","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":5751,"pending":"pAsLsXsFsx","issued":"pAsLsXsFsx","wanted":"AsLsXsFsx","last_sent":10}],"loner":5751,"want_loner":5751,"mds_caps_wanted":[],"dirfrags":[{"path":"/ceph-qa-suite/suites/smoke/1node/clusters","dirfrag":"0x100000007a8","snapid_first":2,"projected_version":"11","version":"11","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"dirty":1,"authpin":0},"nref":2,"dentries":[{"path":"ceph-qa-suite/suites/smoke/1node/clusters/openstack.yaml","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511629739,"remote_type":"","version":10,"projected_version":10,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"ceph-qa-suite/suites/smoke/1node/clusters/+","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511629737,"remote_type":"","version":8,"projected_version":8,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"ceph-qa-suite/suites/smoke/1node/clusters/fixed-1.yaml","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511629738,"remote_type":"","version":4,"projected_version":4,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false},{"ino":1099511629735,"rdev":0,"ctime":"2024-04-26T20:59:27.122643+0000","btime":"2024-04-26T20:59:27.122643+0000","mode":33188,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":0,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":7,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T20:59:27.122643+0000","atime":"2024-04-26T20:59:27.122643+0000","time_warp_seq":0,"change_attr":0,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":1,"rsubdirs":0,"rsnaps":0,"rctime":"0.000000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":1,"rsubdirs":0,"rsnaps":0,"rctime":"0.000000"},"version":16,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"lock":0,"caps":1,"dirtyparent":1,"dirty":1,"authpin":0},"nref":3,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":5751,"pending":"pAsLsXsFscr","issued":"pAsLsXsFscr","wanted":"-","last_sent":3}],"loner":5751,"want_loner":-1,"mds_caps_wanted":[],"dirfrags":[],"is_subvolume":false},{"ino":1099511629740,"rdev":0,"ctime":"2024-04-26T20:59:27.195227+0000","btime":"2024-04-26T20:59:27.185736+0000","mode":16877,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T20:59:27.195227+0000","atime":"2024-04-26T20:59:27.185736+0000","time_warp_seq":0,"change_attr":1,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-26T20:59:27.195227+0000","num_files":1,"num_subdirs":0,"change_attr":1},"rstat":{"version":0,"rbytes":216,"rfiles":1,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T20:59:27.199637+0000"},"accounted_rstat":{"version":0,"rbytes":216,"rfiles":1,"rsubdirs":1,"rsnaps":0,"rctime":"2024-04-26T20:59:27.199637+0000"},"version":22,"file_data_version":0,"xattr_version":1,"backtrace_version":12,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1,"waiter":0,"authpin":0},"nref":4,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","type":"ifile","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":5751,"pending":"pAsLsXsFsx","issued":"pAsLsXsFsx","wanted":"AsLsXsFsx","last_sent":8}],"loner":5751,"want_loner":5751,"mds_caps_wanted":[],"dirfrags":[{"path":"/ceph-qa-suite/suites/smoke/1node/tasks","dirfrag":"0x100000007ac","snapid_first":2,"projected_version":"5","version":"5","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"dirty":1,"authpin":0},"nref":2,"dentries":[{"path":"ceph-qa-suite/suites/smoke/1node/tasks/ceph-deploy.yaml","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511629741,"remote_type":"","version":4,"projected_version":4,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false},{"ino":1099511629734,"rdev":0,"ctime":"2024-04-26T20:59:27.185736+0000","btime":"2024-04-26T20:59:27.112401+0000","mode":16877,"uid":1000,"gid":1316,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2024-04-26T20:59:27.185736+0000","atime":"2024-04-26T20:59:27.112401+0000","time_warp_seq":0,"change_attr":3,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"quiesce_block":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"2024-04-26T20:59:27.185736+0000","num_files":1,"num_subdirs":2,"change_attr":3},"rstat":{"version":1,"rbytes":404,"rfiles":5,"rsubdirs":3,"rsnaps":0,"rctime":"2024-04-26T20:59:27.199637+0000"},"accounted_rstat":{"version":1,"rbytes":404,"rfiles":5,"rsubdirs":3,"rsnaps":0,"rctime":"2024-04-26T20:59:27.199637+0000"},"version":41,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dirtyscattered":0,"request":0,"lock":0,"dirfrag":1,"caps":1,"dirtyrstat":0,"dirtyparent":1,"dirty":1,"waiter":0,"authpin":0},"nref":4,"versionlock":{"gather_set":[],"state":"lock","type":"iversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","type":"ifile","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","type":"inest","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"quiescelock":{"gather_set":[],"state":"lock","type":"iquiesce","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":5751,"pending":"pAsLsXsFsx","issued":"pAsLsXsFsx","wanted":"AsLsXsFsx","last_sent":12}],"loner":5751,"want_loner":5751,"mds_caps_wanted":[],"dirfrags":[{"path":"/ceph-qa-suite/suites/smoke/1node","dirfrag":"0x100000007a6","snapid_first":2,"projected_version":"24","version":"24","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"dirty":1,"authpin":0},"nref":2,"dentries":[{"path":"ceph-qa-suite/suites/smoke/1node/clusters","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511629736,"remote_type":"","version":20,"projected_version":20,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"ceph-qa-suite/suites/smoke/1node/%","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511629735,"remote_type":"","version":16,"projected_version":16,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]},{"path":"ceph-qa-suite/suites/smoke/1node/tasks","path_ino":1,"snap_first":2,"snap_last":18446744073709551614,"is_primary":true,"is_remote":false,"is_null":false,"is_new":false,"inode":1099511629740,"remote_type":"","version":22,"projected_version":22,"auth_pins":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"lock":0,"inodepin":1,"dirty":1,"authpin":0},"nref":2,"lock":{},"versionlock":{"gather_set":[],"state":"lock","type":"dversion","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"states":["auth","dirty"]}]}],"is_subvolume":false}] 2024-04-26T21:06:29.998 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "accounted_rstat": { "rbytes": 188, "rctime": "2024-04-26T20:59:27.176487+0000", "rfiles": 3, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2024-04-26T20:59:27.138794+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 4, "btime": "2024-04-26T20:59:27.138794+0000", "change_attr": 3, "client_caps": [ { "client_id": 5751, "issued": "pAsLsXsFsx", "last_sent": 10, "pending": "pAsLsXsFsx", "wanted": "AsLsXsFsx" } ], "client_ranges": [], "ctime": "2024-04-26T20:59:27.172421+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dentries": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511629739, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "ceph-qa-suite/suites/smoke/1node/clusters/openstack.yaml", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 10, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 10, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511629737, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "ceph-qa-suite/suites/smoke/1node/clusters/+", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 8, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 8, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511629738, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "ceph-qa-suite/suites/smoke/1node/clusters/fixed-1.yaml", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 4, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x100000007a8", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/ceph-qa-suite/suites/smoke/1node/clusters", "pins": { "authpin": 0, "child": 1, "dirty": 1 }, "projected_version": "11", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "11" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 3, "mtime": "2024-04-26T20:59:27.172421+0000", "num_files": 3, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "type": "ifile", "xlock_by": {} }, "flocklock": {}, "gid": 1316, "ino": 1099511629736, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 5751, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-04-26T20:59:27.172421+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "lock": 0, "request": 0, "waiter": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 188, "rctime": "2024-04-26T20:59:27.176487+0000", "rfiles": 3, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 20, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": 5751, "xattr_version": 1, "xattrlock": {}, "xattrs": [] }, { "accounted_rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 0, "version": 0 }, "atime": "2024-04-26T20:59:27.122643+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-26T20:59:27.122643+0000", "change_attr": 0, "client_caps": [ { "client_id": 5751, "issued": "pAsLsXsFscr", "last_sent": 3, "pending": "pAsLsXsFscr", "wanted": "-" } ], "client_ranges": [], "ctime": "2024-04-26T20:59:27.122643+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 0, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 1316, "ino": 1099511629735, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 7, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": 5751, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 33188, "mtime": "2024-04-26T20:59:27.122643+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 3, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 1, "dirty": 1, "dirtyparent": 1, "lock": 0, "ptrwaiter": 0, "request": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 0, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 16, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] }, { "accounted_rstat": { "rbytes": 216, "rctime": "2024-04-26T20:59:27.199637+0000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2024-04-26T20:59:27.185736+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 12, "btime": "2024-04-26T20:59:27.185736+0000", "change_attr": 1, "client_caps": [ { "client_id": 5751, "issued": "pAsLsXsFsx", "last_sent": 8, "pending": "pAsLsXsFsx", "wanted": "AsLsXsFsx" } ], "client_ranges": [], "ctime": "2024-04-26T20:59:27.195227+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dentries": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511629741, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "ceph-qa-suite/suites/smoke/1node/tasks/ceph-deploy.yaml", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 4, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 4, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x100000007ac", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/ceph-qa-suite/suites/smoke/1node/tasks", "pins": { "authpin": 0, "child": 1, "dirty": 1 }, "projected_version": "5", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 1, "mtime": "2024-04-26T20:59:27.195227+0000", "num_files": 1, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "type": "ifile", "xlock_by": {} }, "flocklock": {}, "gid": 1316, "ino": 1099511629740, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 5751, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-04-26T20:59:27.195227+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "lock": 0, "request": 0, "waiter": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 216, "rctime": "2024-04-26T20:59:27.199637+0000", "rfiles": 1, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 22, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": 5751, "xattr_version": 1, "xattrlock": {}, "xattrs": [] }, { "accounted_rstat": { "rbytes": 404, "rctime": "2024-04-26T20:59:27.199637+0000", "rfiles": 5, "rsnaps": 0, "rsubdirs": 3, "version": 1 }, "atime": "2024-04-26T20:59:27.112401+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": {}, "backtrace_version": 2, "btime": "2024-04-26T20:59:27.112401+0000", "change_attr": 3, "client_caps": [ { "client_id": 5751, "issued": "pAsLsXsFsx", "last_sent": 12, "pending": "pAsLsXsFsx", "wanted": "AsLsXsFsx" } ], "client_ranges": [], "ctime": "2024-04-26T20:59:27.185736+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfrags": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dentries": [ { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511629736, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "ceph-qa-suite/suites/smoke/1node/clusters", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 20, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 20, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511629735, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "ceph-qa-suite/suites/smoke/1node/%", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 16, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 16, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } }, { "auth_pins": 0, "auth_state": { "replicas": {} }, "inode": 1099511629740, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_new": false, "is_null": false, "is_primary": true, "is_remote": false, "lock": {}, "nref": 2, "path": "ceph-qa-suite/suites/smoke/1node/tasks", "path_ino": 1, "pins": { "authpin": 0, "dirty": 1, "inodepin": 1, "lock": 0, "request": 0 }, "projected_version": 22, "remote_type": "", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snap_first": 2, "snap_last": 18446744073709551614, "states": [ "auth", "dirty" ], "version": 22, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "dversion", "xlock_by": {} } } ], "dir_auth": "", "dirfrag": "0x100000007a6", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/ceph-qa-suite/suites/smoke/1node", "pins": { "authpin": 0, "child": 1, "dirty": 1 }, "projected_version": "24", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "24" } ], "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 3, "mtime": "2024-04-26T20:59:27.185736+0000", "num_files": 1, "num_subdirs": 2, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "type": "ifile", "xlock_by": {} }, "flocklock": {}, "gid": 1316, "ino": 1099511629734, "is_auth": true, "is_freezing": false, "is_frozen": false, "is_subvolume": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 5751, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2024-04-26T20:59:27.185736+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "inest", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "pins": { "authpin": 0, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "dirtyrstat": 0, "dirtyscattered": 0, "lock": 0, "request": 0, "waiter": 0 }, "policylock": {}, "quiesce_block": false, "quiescelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iquiesce", "xlock_by": {} }, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 404, "rctime": "2024-04-26T20:59:27.199637+0000", "rfiles": 5, "rsnaps": 0, "rsubdirs": 3, "version": 1 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 1000, "version": 41, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "type": "iversion", "xlock_by": {} }, "want_loner": 5751, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } ] 2024-04-26T21:06:29.999 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T21:06:29.999 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:06:30.101 DEBUG:teuthology.orchestra.run.smithi103:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T21:06:30.212 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T21:06:30.212 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:06:30.212 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T21:06:30.212 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:06:30.274 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T21:06:30.274 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:06:30.326 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:06:30.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.538+0000 7f3d69800640 1 Processor -- start 2024-04-26T21:06:30.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.538+0000 7f3d69800640 1 -- start start 2024-04-26T21:06:30.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.539+0000 7f3d69800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3d64071e20 0x7f3d641610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:30.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.539+0000 7f3d69800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d64146440 con 0x7f3d64146670 2024-04-26T21:06:30.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.539+0000 7f3d69800640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d6406f010 con 0x7f3d64146060 2024-04-26T21:06:30.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.539+0000 7f3d69800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f3d64072220 con 0x7f3d64071e20 2024-04-26T21:06:30.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.539+0000 7f3d68e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3d64071e20 0x7f3d641610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:30.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.539+0000 7f3d5be00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3d64146670 0x7f3d640cb0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45044/0 (socket says 172.21.15.28:45044) 2024-04-26T21:06:30.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.539+0000 7f3d5be00640 1 -- 172.21.15.28:0/3771339102 learned_addr learned my addr 172.21.15.28:0/3771339102 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:30.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.539+0000 7f3d68e00640 1 -- 172.21.15.28:0/3771339102 >> v1:172.21.15.103:6789/0 conn(0x7f3d64146060 legacy=0x7f3d641609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:30.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.540+0000 7f3d68e00640 1 -- 172.21.15.28:0/3771339102 >> v1:172.21.15.28:6789/0 conn(0x7f3d64146670 legacy=0x7f3d640cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:30.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.539+0000 7f3d63400640 1 -- 172.21.15.28:0/3771339102 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 300406660 0 0) 0x7f3d64146440 con 0x7f3d64146670 2024-04-26T21:06:30.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.540+0000 7f3d68e00640 1 -- 172.21.15.28:0/3771339102 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d641617a0 con 0x7f3d64071e20 2024-04-26T21:06:30.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.541+0000 7f3d68e00640 1 --2- 172.21.15.28:0/3771339102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d64071e20 0x7f3d641610c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009d90 tx=0x7f3d5c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1993e13e91027374 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:30.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.541+0000 7f3d63400640 1 -- 172.21.15.28:0/3771339102 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c031600 con 0x7f3d64071e20 2024-04-26T21:06:30.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.541+0000 7f3d63400640 1 -- 172.21.15.28:0/3771339102 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d5c0317a0 con 0x7f3d64071e20 2024-04-26T21:06:30.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.541+0000 7f3d63400640 1 -- 172.21.15.28:0/3771339102 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c031b70 con 0x7f3d64071e20 2024-04-26T21:06:30.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.542+0000 7f3d69800640 1 -- 172.21.15.28:0/3771339102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d64071e20 msgr2=0x7f3d641610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:30.544 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.542+0000 7f3d69800640 1 --2- 172.21.15.28:0/3771339102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d64071e20 0x7f3d641610c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009d90 tx=0x7f3d5c02a610 comp rx=0 tx=0).stop 2024-04-26T21:06:30.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.542+0000 7f3d69800640 1 -- 172.21.15.28:0/3771339102 shutdown_connections 2024-04-26T21:06:30.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.542+0000 7f3d69800640 1 --2- 172.21.15.28:0/3771339102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d64071e20 0x7f3d641610c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:30.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.542+0000 7f3d69800640 1 -- 172.21.15.28:0/3771339102 >> 172.21.15.28:0/3771339102 conn(0x7f3d640d0a50 msgr2=0x7f3d640d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:30.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.542+0000 7f3d69800640 1 -- 172.21.15.28:0/3771339102 shutdown_connections 2024-04-26T21:06:30.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.542+0000 7f3d69800640 1 -- 172.21.15.28:0/3771339102 wait complete. 2024-04-26T21:06:30.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.543+0000 7f3d69800640 1 Processor -- start 2024-04-26T21:06:30.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.544+0000 7f3d69800640 1 -- start start 2024-04-26T21:06:30.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.545+0000 7f3d69800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d64071e20 0x7f3d640ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:30.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.545+0000 7f3d69800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d64146060 0x7f3d640aa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:30.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.545+0000 7f3d69800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d64146670 0x7f3d640d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:30.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.545+0000 7f3d69800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6406ff60 con 0x7f3d64146670 2024-04-26T21:06:30.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.545+0000 7f3d69800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6406f510 con 0x7f3d64071e20 2024-04-26T21:06:30.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.545+0000 7f3d69800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f3d640cb7b0 con 0x7f3d64146060 2024-04-26T21:06:30.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.545+0000 7f3d68e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d64146670 0x7f3d640d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:30.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.545+0000 7f3d68e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d64146670 0x7f3d640d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37270/0 (socket says 172.21.15.28:37270) 2024-04-26T21:06:30.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.545+0000 7f3d68e00640 1 -- 172.21.15.28:0/2590237692 learned_addr learned my addr 172.21.15.28:0/2590237692 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:30.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.545+0000 7f3d63e00640 1 --2- 172.21.15.28:0/2590237692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d64071e20 0x7f3d640ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:30.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.545+0000 7f3d5be00640 1 --2- 172.21.15.28:0/2590237692 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d64146060 0x7f3d640aa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:30.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.546+0000 7f3d68e00640 1 -- 172.21.15.28:0/2590237692 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d64146060 msgr2=0x7f3d640aa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:30.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.546+0000 7f3d68e00640 1 --2- 172.21.15.28:0/2590237692 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d64146060 0x7f3d640aa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:30.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.546+0000 7f3d68e00640 1 -- 172.21.15.28:0/2590237692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d64071e20 msgr2=0x7f3d640ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:30.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.546+0000 7f3d68e00640 1 --2- 172.21.15.28:0/2590237692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d64071e20 0x7f3d640ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:30.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.546+0000 7f3d68e00640 1 -- 172.21.15.28:0/2590237692 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d640aaf60 con 0x7f3d64146670 2024-04-26T21:06:30.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.546+0000 7f3d5be00640 1 --2- 172.21.15.28:0/2590237692 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d64146060 0x7f3d640aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:30.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.546+0000 7f3d68e00640 1 --2- 172.21.15.28:0/2590237692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d64146670 0x7f3d640d10a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009eb0 tx=0x7f3d5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:30.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.546+0000 7f3d60c00640 1 -- 172.21.15.28:0/2590237692 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c040070 con 0x7f3d64146670 2024-04-26T21:06:30.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.546+0000 7f3d60c00640 1 -- 172.21.15.28:0/2590237692 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d5c030b60 con 0x7f3d64146670 2024-04-26T21:06:30.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.546+0000 7f3d69800640 1 -- 172.21.15.28:0/2590237692 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d640ab200 con 0x7f3d64146670 2024-04-26T21:06:30.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.546+0000 7f3d60c00640 1 -- 172.21.15.28:0/2590237692 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c045460 con 0x7f3d64146670 2024-04-26T21:06:30.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.546+0000 7f3d69800640 1 -- 172.21.15.28:0/2590237692 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d640ab7c0 con 0x7f3d64146670 2024-04-26T21:06:30.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.548+0000 7f3d69800640 1 -- 172.21.15.28:0/2590237692 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6406a360 con 0x7f3d64146670 2024-04-26T21:06:30.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.549+0000 7f3d60c00640 1 -- 172.21.15.28:0/2590237692 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d5c039040 con 0x7f3d64146670 2024-04-26T21:06:30.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.550+0000 7f3d60c00640 1 --2- 172.21.15.28:0/2590237692 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3d4006c8f0 0x7f3d4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:30.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.550+0000 7f3d60c00640 1 -- 172.21.15.28:0/2590237692 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f3d5c0ad470 con 0x7f3d64146670 2024-04-26T21:06:30.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.550+0000 7f3d63e00640 1 --2- 172.21.15.28:0/2590237692 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3d4006c8f0 0x7f3d4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:30.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.551+0000 7f3d63e00640 1 --2- 172.21.15.28:0/2590237692 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3d4006c8f0 0x7f3d4006edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3d54004ea0 tx=0x7f3d54009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:30.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.554+0000 7f3d60c00640 1 -- 172.21.15.28:0/2590237692 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3d5c07b880 con 0x7f3d64146670 2024-04-26T21:06:30.761 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.760+0000 7f3d69800640 1 -- 172.21.15.28:0/2590237692 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d640b16b0 con 0x7f3d64146670 2024-04-26T21:06:30.764 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.763+0000 7f3d60c00640 1 -- 172.21.15.28:0/2590237692 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16685 (secure 0 0 0) 0x7f3d5c07b180 con 0x7f3d64146670 2024-04-26T21:06:30.765 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T21:06:30.765 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":47,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:59:38.275251+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:58:41.306286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T20:58:42.497340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.16","mappings":[{"from":0,"to":5}]},{"pgid":"7.30","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:06:30.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.766+0000 7f3d69800640 1 -- 172.21.15.28:0/2590237692 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3d4006c8f0 msgr2=0x7f3d4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.766+0000 7f3d69800640 1 --2- 172.21.15.28:0/2590237692 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3d4006c8f0 0x7f3d4006edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3d54004ea0 tx=0x7f3d54009040 comp rx=0 tx=0).stop 2024-04-26T21:06:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.766+0000 7f3d69800640 1 -- 172.21.15.28:0/2590237692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d64146670 msgr2=0x7f3d640d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.766+0000 7f3d69800640 1 --2- 172.21.15.28:0/2590237692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d64146670 0x7f3d640d10a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c009eb0 tx=0x7f3d5c004a40 comp rx=0 tx=0).stop 2024-04-26T21:06:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.766+0000 7f3d69800640 1 -- 172.21.15.28:0/2590237692 shutdown_connections 2024-04-26T21:06:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.766+0000 7f3d69800640 1 --2- 172.21.15.28:0/2590237692 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3d4006c8f0 0x7f3d4006edb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.766+0000 7f3d69800640 1 --2- 172.21.15.28:0/2590237692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3d64146670 0x7f3d640d10a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.766+0000 7f3d69800640 1 --2- 172.21.15.28:0/2590237692 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3d64146060 0x7f3d640aa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.766+0000 7f3d69800640 1 --2- 172.21.15.28:0/2590237692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3d64071e20 0x7f3d640ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.766+0000 7f3d69800640 1 -- 172.21.15.28:0/2590237692 >> 172.21.15.28:0/2590237692 conn(0x7f3d640d0a50 msgr2=0x7f3d64160e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.766+0000 7f3d69800640 1 -- 172.21.15.28:0/2590237692 shutdown_connections 2024-04-26T21:06:30.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:30.766+0000 7f3d69800640 1 -- 172.21.15.28:0/2590237692 wait complete. 2024-04-26T21:06:30.788 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:58:41.306286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:58:42.497340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:06:30.788 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:06:31.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.007+0000 7f6f65200640 1 Processor -- start 2024-04-26T21:06:31.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.007+0000 7f6f65200640 1 -- start start 2024-04-26T21:06:31.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.008+0000 7f6f65200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f6f6015f1d0 0x7f6f600d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:31.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.008+0000 7f6f65200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6f600d0310 0x7f6f600b7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:31.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.008+0000 7f6f65200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f6015f5b0 con 0x7f6f600d34b0 2024-04-26T21:06:31.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.008+0000 7f6f65200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f6f600d3a30 con 0x7f6f6015f1d0 2024-04-26T21:06:31.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.008+0000 7f6f65200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f6f600d3d10 con 0x7f6f600d0310 2024-04-26T21:06:31.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.009+0000 7f6f5fe00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6f600d0310 0x7f6f600b7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:31.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.009+0000 7f6f5fe00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6f600d0310 0x7f6f600b7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.28:49782/0 (socket says 172.21.15.28:49782) 2024-04-26T21:06:31.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.009+0000 7f6f5fe00640 1 -- 172.21.15.28:0/3281145249 learned_addr learned my addr 172.21.15.28:0/3281145249 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:31.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.009+0000 7f6f5f400640 1 --2- 172.21.15.28:0/3281145249 >> v2:172.21.15.103:3300/0 conn(0x7f6f6015f1d0 0x7f6f600d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:31.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.010+0000 7f6f5fe00640 1 -- 172.21.15.28:0/3281145249 >> v2:172.21.15.103:3300/0 conn(0x7f6f6015f1d0 msgr2=0x7f6f600d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:31.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.010+0000 7f6f5fe00640 1 --2- 172.21.15.28:0/3281145249 >> v2:172.21.15.103:3300/0 conn(0x7f6f6015f1d0 0x7f6f600d2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.010+0000 7f6f5fe00640 1 -- 172.21.15.28:0/3281145249 >> v1:172.21.15.103:6789/0 conn(0x7f6f600d34b0 legacy=0x7f6f600cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:31.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.010+0000 7f6f5fe00640 1 -- 172.21.15.28:0/3281145249 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f600d3ff0 con 0x7f6f600d0310 2024-04-26T21:06:31.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.012+0000 7f6f5fe00640 1 --2- 172.21.15.28:0/3281145249 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6f600d0310 0x7f6f600b7f40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009e50 tx=0x7f6f5802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85ce70fb517f6e59 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:31.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.012+0000 7f6f5e000640 1 -- 172.21.15.28:0/3281145249 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58036070 con 0x7f6f600d0310 2024-04-26T21:06:31.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.012+0000 7f6f5e000640 1 -- 172.21.15.28:0/3281145249 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f58031600 con 0x7f6f600d0310 2024-04-26T21:06:31.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.012+0000 7f6f5e000640 1 -- 172.21.15.28:0/3281145249 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f580319d0 con 0x7f6f600d0310 2024-04-26T21:06:31.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.013+0000 7f6f65200640 1 -- 172.21.15.28:0/3281145249 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6f600d0310 msgr2=0x7f6f600b7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:31.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.013+0000 7f6f65200640 1 --2- 172.21.15.28:0/3281145249 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6f600d0310 0x7f6f600b7f40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009e50 tx=0x7f6f5802a710 comp rx=0 tx=0).stop 2024-04-26T21:06:31.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.013+0000 7f6f65200640 1 -- 172.21.15.28:0/3281145249 shutdown_connections 2024-04-26T21:06:31.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.013+0000 7f6f65200640 1 --2- 172.21.15.28:0/3281145249 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6f600d0310 0x7f6f600b7f40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.013+0000 7f6f65200640 1 --2- 172.21.15.28:0/3281145249 >> v2:172.21.15.103:3300/0 conn(0x7f6f6015f1d0 0x7f6f600d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.013+0000 7f6f65200640 1 -- 172.21.15.28:0/3281145249 >> 172.21.15.28:0/3281145249 conn(0x7f6f600d5030 msgr2=0x7f6f600d5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:31.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.013+0000 7f6f65200640 1 -- 172.21.15.28:0/3281145249 shutdown_connections 2024-04-26T21:06:31.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.013+0000 7f6f65200640 1 -- 172.21.15.28:0/3281145249 wait complete. 2024-04-26T21:06:31.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.016+0000 7f6f65200640 1 Processor -- start 2024-04-26T21:06:31.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.016+0000 7f6f65200640 1 -- start start 2024-04-26T21:06:31.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.016+0000 7f6f65200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6f600d0310 0x7f6f600c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:31.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.016+0000 7f6f65200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f600d34b0 0x7f6f601f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:31.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.016+0000 7f6f65200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f6015f1d0 0x7f6f60160f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:31.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.016+0000 7f6f65200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f6f600bacf0 con 0x7f6f600d34b0 2024-04-26T21:06:31.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.016+0000 7f6f65200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6f600d45b0 con 0x7f6f6015f1d0 2024-04-26T21:06:31.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.016+0000 7f6f65200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f6f600bae70 con 0x7f6f600d0310 2024-04-26T21:06:31.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.016+0000 7f6f5ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f600d34b0 0x7f6f601f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:31.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.016+0000 7f6f5ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f600d34b0 0x7f6f601f46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37284/0 (socket says 172.21.15.28:37284) 2024-04-26T21:06:31.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.017+0000 7f6f5ea00640 1 -- 172.21.15.28:0/1610849540 learned_addr learned my addr 172.21.15.28:0/1610849540 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:31.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.017+0000 7f6f5fe00640 1 --2- 172.21.15.28:0/1610849540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f6015f1d0 0x7f6f60160f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:31.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.017+0000 7f6f5f400640 1 --2- 172.21.15.28:0/1610849540 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6f600d0310 0x7f6f600c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:31.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.017+0000 7f6f5ea00640 1 -- 172.21.15.28:0/1610849540 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6f600d0310 msgr2=0x7f6f600c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:31.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.017+0000 7f6f5ea00640 1 --2- 172.21.15.28:0/1610849540 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6f600d0310 0x7f6f600c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.017+0000 7f6f5ea00640 1 -- 172.21.15.28:0/1610849540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f6015f1d0 msgr2=0x7f6f60160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:31.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.017+0000 7f6f5ea00640 1 --2- 172.21.15.28:0/1610849540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f6015f1d0 0x7f6f60160f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.017+0000 7f6f5ea00640 1 -- 172.21.15.28:0/1610849540 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f600cb350 con 0x7f6f600d34b0 2024-04-26T21:06:31.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.017+0000 7f6f5ea00640 1 --2- 172.21.15.28:0/1610849540 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f600d34b0 0x7f6f601f46d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00c430 tx=0x7f6f4c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:31.023 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.018+0000 7f6f57400640 1 -- 172.21.15.28:0/1610849540 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c0052d0 con 0x7f6f600d34b0 2024-04-26T21:06:31.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.018+0000 7f6f57400640 1 -- 172.21.15.28:0/1610849540 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f4c005490 con 0x7f6f600d34b0 2024-04-26T21:06:31.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.018+0000 7f6f65200640 1 -- 172.21.15.28:0/1610849540 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f600cb630 con 0x7f6f600d34b0 2024-04-26T21:06:31.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.018+0000 7f6f57400640 1 -- 172.21.15.28:0/1610849540 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c007260 con 0x7f6f600d34b0 2024-04-26T21:06:31.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.018+0000 7f6f65200640 1 -- 172.21.15.28:0/1610849540 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f601f4c10 con 0x7f6f600d34b0 2024-04-26T21:06:31.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.019+0000 7f6f65200640 1 -- 172.21.15.28:0/1610849540 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f6006a250 con 0x7f6f600d34b0 2024-04-26T21:06:31.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.020+0000 7f6f57400640 1 -- 172.21.15.28:0/1610849540 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6f4c007400 con 0x7f6f600d34b0 2024-04-26T21:06:31.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.022+0000 7f6f57400640 1 --2- 172.21.15.28:0/1610849540 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6f4006c8f0 0x7f6f4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:31.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.022+0000 7f6f57400640 1 -- 172.21.15.28:0/1610849540 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f6f4c08b190 con 0x7f6f600d34b0 2024-04-26T21:06:31.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.022+0000 7f6f5f400640 1 --2- 172.21.15.28:0/1610849540 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6f4006c8f0 0x7f6f4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:31.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.023+0000 7f6f5f400640 1 --2- 172.21.15.28:0/1610849540 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6f4006c8f0 0x7f6f4006edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6f50009ad0 tx=0x7f6f50008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:31.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.028+0000 7f6f57400640 1 -- 172.21.15.28:0/1610849540 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6f4c0594e0 con 0x7f6f600d34b0 2024-04-26T21:06:31.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.233+0000 7f6f65200640 1 -- 172.21.15.28:0/1610849540 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6f600b9c70 con 0x7f6f600d34b0 2024-04-26T21:06:31.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.234+0000 7f6f57400640 1 -- 172.21.15.28:0/1610849540 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) ==== 93+0+10 (secure 0 0 0) 0x7f6f4c058de0 con 0x7f6f600d34b0 2024-04-26T21:06:31.235 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T21:06:31.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.237+0000 7f6f65200640 1 -- 172.21.15.28:0/1610849540 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6f4006c8f0 msgr2=0x7f6f4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:31.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.237+0000 7f6f65200640 1 --2- 172.21.15.28:0/1610849540 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6f4006c8f0 0x7f6f4006edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6f50009ad0 tx=0x7f6f50008040 comp rx=0 tx=0).stop 2024-04-26T21:06:31.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.237+0000 7f6f65200640 1 -- 172.21.15.28:0/1610849540 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f600d34b0 msgr2=0x7f6f601f46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:31.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.237+0000 7f6f65200640 1 --2- 172.21.15.28:0/1610849540 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f600d34b0 0x7f6f601f46d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00c430 tx=0x7f6f4c00c8f0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.237+0000 7f6f65200640 1 -- 172.21.15.28:0/1610849540 shutdown_connections 2024-04-26T21:06:31.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.237+0000 7f6f65200640 1 --2- 172.21.15.28:0/1610849540 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f6f4006c8f0 0x7f6f4006edb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.237+0000 7f6f65200640 1 --2- 172.21.15.28:0/1610849540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6f6015f1d0 0x7f6f60160f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.237+0000 7f6f65200640 1 --2- 172.21.15.28:0/1610849540 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f600d34b0 0x7f6f601f46d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.237+0000 7f6f65200640 1 --2- 172.21.15.28:0/1610849540 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f6f600d0310 0x7f6f600c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.237+0000 7f6f65200640 1 -- 172.21.15.28:0/1610849540 >> 172.21.15.28:0/1610849540 conn(0x7f6f600d5030 msgr2=0x7f6f600b6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:31.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.238+0000 7f6f65200640 1 -- 172.21.15.28:0/1610849540 shutdown_connections 2024-04-26T21:06:31.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.238+0000 7f6f65200640 1 -- 172.21.15.28:0/1610849540 wait complete. 2024-04-26T21:06:31.258 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:06:31.466 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.466+0000 7f0e00600640 1 Processor -- start 2024-04-26T21:06:31.467 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.466+0000 7f0e00600640 1 -- start start 2024-04-26T21:06:31.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.468+0000 7f0e00600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0df8146730 0x7f0df80b7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:31.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.468+0000 7f0e00600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0df8146b10 0x7f0df8070430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:31.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.468+0000 7f0e00600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0df80d14a0 con 0x7f0df815e3a0 2024-04-26T21:06:31.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.468+0000 7f0e00600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f0df8071a10 con 0x7f0df8146730 2024-04-26T21:06:31.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.468+0000 7f0e00600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f0df8071cf0 con 0x7f0df8146b10 2024-04-26T21:06:31.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.469+0000 7f0dfdc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0df8146730 0x7f0df80b7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:31.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.469+0000 7f0dfe600640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f0df815e3a0 0x7f0df8160c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45058/0 (socket says 172.21.15.28:45058) 2024-04-26T21:06:31.469 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.469+0000 7f0dfe600640 1 -- 172.21.15.28:0/3449822686 learned_addr learned my addr 172.21.15.28:0/3449822686 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:31.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.469+0000 7f0dfd200640 1 --2- 172.21.15.28:0/3449822686 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0df8146b10 0x7f0df8070430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:31.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.469+0000 7f0dfdc00640 1 -- 172.21.15.28:0/3449822686 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0df8146b10 msgr2=0x7f0df8070430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:31.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.469+0000 7f0dfdc00640 1 --2- 172.21.15.28:0/3449822686 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0df8146b10 0x7f0df8070430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.469+0000 7f0dfdc00640 1 -- 172.21.15.28:0/3449822686 >> v1:172.21.15.28:6789/0 conn(0x7f0df815e3a0 legacy=0x7f0df8160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:31.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.469+0000 7f0dfdc00640 1 -- 172.21.15.28:0/3449822686 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df8071fd0 con 0x7f0df8146730 2024-04-26T21:06:31.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.469+0000 7f0df7e00640 1 -- 172.21.15.28:0/3449822686 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 716909971 0 0) 0x7f0df0001490 con 0x7f0df815e3a0 2024-04-26T21:06:31.470 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.469+0000 7f0dfd200640 1 --2- 172.21.15.28:0/3449822686 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0df8146b10 0x7f0df8070430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:31.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.470+0000 7f0dfdc00640 1 --2- 172.21.15.28:0/3449822686 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0df8146730 0x7f0df80b7cb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0dec0097a0 tx=0x7f0dec02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a162658c806af2d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:31.471 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.471+0000 7f0df7e00640 1 -- 172.21.15.28:0/3449822686 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec037070 con 0x7f0df8146730 2024-04-26T21:06:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.471+0000 7f0df7e00640 1 -- 172.21.15.28:0/3449822686 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dec032440 con 0x7f0df8146730 2024-04-26T21:06:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.471+0000 7f0df7e00640 1 -- 172.21.15.28:0/3449822686 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec032810 con 0x7f0df8146730 2024-04-26T21:06:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.471+0000 7f0e00600640 1 -- 172.21.15.28:0/3449822686 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0df8146730 msgr2=0x7f0df80b7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.471+0000 7f0e00600640 1 --2- 172.21.15.28:0/3449822686 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0df8146730 0x7f0df80b7cb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0dec0097a0 tx=0x7f0dec02a180 comp rx=0 tx=0).stop 2024-04-26T21:06:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.471+0000 7f0e00600640 1 -- 172.21.15.28:0/3449822686 shutdown_connections 2024-04-26T21:06:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.471+0000 7f0e00600640 1 --2- 172.21.15.28:0/3449822686 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0df8146b10 0x7f0df8070430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.471+0000 7f0e00600640 1 --2- 172.21.15.28:0/3449822686 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0df8146730 0x7f0df80b7cb0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.471+0000 7f0e00600640 1 -- 172.21.15.28:0/3449822686 >> 172.21.15.28:0/3449822686 conn(0x7f0df80b4b10 msgr2=0x7f0df80b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.471+0000 7f0e00600640 1 -- 172.21.15.28:0/3449822686 shutdown_connections 2024-04-26T21:06:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.472+0000 7f0e00600640 1 -- 172.21.15.28:0/3449822686 wait complete. 2024-04-26T21:06:31.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.473+0000 7f0e00600640 1 Processor -- start 2024-04-26T21:06:31.473 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.473+0000 7f0e00600640 1 -- start start 2024-04-26T21:06:31.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.474+0000 7f0e00600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0df8146730 0x7f0df81f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:31.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.474+0000 7f0e00600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0df8146b10 0x7f0df8157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:31.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.474+0000 7f0e00600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0df815e3a0 0x7f0df80b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:31.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.474+0000 7f0e00600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0df80b73f0 con 0x7f0df815e3a0 2024-04-26T21:06:31.475 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.474+0000 7f0e00600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0df8072590 con 0x7f0df8146730 2024-04-26T21:06:31.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.474+0000 7f0e00600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f0df80b7570 con 0x7f0df8146b10 2024-04-26T21:06:31.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.475+0000 7f0dfe600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0df815e3a0 0x7f0df80b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:31.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.475+0000 7f0dfe600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0df815e3a0 0x7f0df80b2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37292/0 (socket says 172.21.15.28:37292) 2024-04-26T21:06:31.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.475+0000 7f0dfe600640 1 -- 172.21.15.28:0/2466049227 learned_addr learned my addr 172.21.15.28:0/2466049227 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:31.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.475+0000 7f0dfdc00640 1 --2- 172.21.15.28:0/2466049227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0df8146730 0x7f0df81f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:31.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.475+0000 7f0dfd200640 1 --2- 172.21.15.28:0/2466049227 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0df8146b10 0x7f0df8157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:31.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.475+0000 7f0dfe600640 1 -- 172.21.15.28:0/2466049227 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0df8146b10 msgr2=0x7f0df8157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:31.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.475+0000 7f0dfe600640 1 --2- 172.21.15.28:0/2466049227 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0df8146b10 0x7f0df8157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.475+0000 7f0dfe600640 1 -- 172.21.15.28:0/2466049227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0df8146730 msgr2=0x7f0df81f6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:31.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.475+0000 7f0dfe600640 1 --2- 172.21.15.28:0/2466049227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0df8146730 0x7f0df81f6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.476 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.475+0000 7f0dfe600640 1 -- 172.21.15.28:0/2466049227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df8158710 con 0x7f0df815e3a0 2024-04-26T21:06:31.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.476+0000 7f0dfe600640 1 --2- 172.21.15.28:0/2466049227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0df815e3a0 0x7f0df80b2b80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0df000c780 tx=0x7f0df000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:31.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.476+0000 7f0df5600640 1 -- 172.21.15.28:0/2466049227 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df0018070 con 0x7f0df815e3a0 2024-04-26T21:06:31.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.476+0000 7f0df5600640 1 -- 172.21.15.28:0/2466049227 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df000ce30 con 0x7f0df815e3a0 2024-04-26T21:06:31.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.476+0000 7f0df5600640 1 -- 172.21.15.28:0/2466049227 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df000a840 con 0x7f0df815e3a0 2024-04-26T21:06:31.477 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.476+0000 7f0e00600640 1 -- 172.21.15.28:0/2466049227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df81589f0 con 0x7f0df815e3a0 2024-04-26T21:06:31.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.476+0000 7f0e00600640 1 -- 172.21.15.28:0/2466049227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df81f4270 con 0x7f0df815e3a0 2024-04-26T21:06:31.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.477+0000 7f0e00600640 1 -- 172.21.15.28:0/2466049227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df806a360 con 0x7f0df815e3a0 2024-04-26T21:06:31.478 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.479+0000 7f0df5600640 1 -- 172.21.15.28:0/2466049227 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0df001b070 con 0x7f0df815e3a0 2024-04-26T21:06:31.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.480+0000 7f0df5600640 1 --2- 172.21.15.28:0/2466049227 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0dd806c8f0 0x7f0dd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:31.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.480+0000 7f0df5600640 1 -- 172.21.15.28:0/2466049227 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f0df008b040 con 0x7f0df815e3a0 2024-04-26T21:06:31.480 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.480+0000 7f0dfdc00640 1 --2- 172.21.15.28:0/2466049227 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0dd806c8f0 0x7f0dd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:31.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.481+0000 7f0dfdc00640 1 --2- 172.21.15.28:0/2466049227 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0dd806c8f0 0x7f0dd806edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0dec034000 tx=0x7f0dec02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:31.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.484+0000 7f0df5600640 1 -- 172.21.15.28:0/2466049227 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0df80d14a0 con 0x7f0df815e3a0 2024-04-26T21:06:31.686 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.686+0000 7f0e00600640 1 -- 172.21.15.28:0/2466049227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0df8145ce0 con 0x7f0df815e3a0 2024-04-26T21:06:31.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.687+0000 7f0df5600640 1 -- 172.21.15.28:0/2466049227 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) ==== 104+0+11 (secure 0 0 0) 0x7f0df0059430 con 0x7f0df815e3a0 2024-04-26T21:06:31.687 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-04-26T21:06:31.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.690+0000 7f0e00600640 1 -- 172.21.15.28:0/2466049227 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0dd806c8f0 msgr2=0x7f0dd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:31.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.690+0000 7f0e00600640 1 --2- 172.21.15.28:0/2466049227 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0dd806c8f0 0x7f0dd806edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0dec034000 tx=0x7f0dec02e000 comp rx=0 tx=0).stop 2024-04-26T21:06:31.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.690+0000 7f0e00600640 1 -- 172.21.15.28:0/2466049227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0df815e3a0 msgr2=0x7f0df80b2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:31.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.690+0000 7f0e00600640 1 --2- 172.21.15.28:0/2466049227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0df815e3a0 0x7f0df80b2b80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0df000c780 tx=0x7f0df000cc40 comp rx=0 tx=0).stop 2024-04-26T21:06:31.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.690+0000 7f0e00600640 1 -- 172.21.15.28:0/2466049227 shutdown_connections 2024-04-26T21:06:31.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.690+0000 7f0e00600640 1 --2- 172.21.15.28:0/2466049227 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0dd806c8f0 0x7f0dd806edb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.690+0000 7f0e00600640 1 --2- 172.21.15.28:0/2466049227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0df815e3a0 0x7f0df80b2b80 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.690+0000 7f0e00600640 1 --2- 172.21.15.28:0/2466049227 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0df8146b10 0x7f0df8157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.690+0000 7f0e00600640 1 --2- 172.21.15.28:0/2466049227 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0df8146730 0x7f0df81f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.690+0000 7f0e00600640 1 -- 172.21.15.28:0/2466049227 >> 172.21.15.28:0/2466049227 conn(0x7f0df80b4b10 msgr2=0x7f0df8070430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:31.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.690+0000 7f0e00600640 1 -- 172.21.15.28:0/2466049227 shutdown_connections 2024-04-26T21:06:31.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.690+0000 7f0e00600640 1 -- 172.21.15.28:0/2466049227 wait complete. 2024-04-26T21:06:31.710 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:06:31.928 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.928+0000 7fd001400640 1 Processor -- start 2024-04-26T21:06:31.929 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.928+0000 7fd001400640 1 -- start start 2024-04-26T21:06:31.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.930+0000 7fd001400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcffc15f0d0 0x7fcffc06cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:31.930 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.930+0000 7fd001400640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fcffc163830 0x7fcffc0d3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:31.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.930+0000 7fd001400640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcffc06d2a0 con 0x7fcffc162cc0 2024-04-26T21:06:31.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.930+0000 7fd001400640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fcffc0d2ac0 con 0x7fcffc15f0d0 2024-04-26T21:06:31.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.930+0000 7fd001400640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fcffc0d2da0 con 0x7fcffc163830 2024-04-26T21:06:31.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.931+0000 7fcffbe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcffc15f0d0 0x7fcffc06cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:31.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.931+0000 7fcffbe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcffc15f0d0 0x7fcffc06cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37306/0 (socket says 172.21.15.28:37306) 2024-04-26T21:06:31.931 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.931+0000 7fcffbe00640 1 -- 172.21.15.28:0/786452281 learned_addr learned my addr 172.21.15.28:0/786452281 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:31.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.931+0000 7fd000a00640 1 --2- 172.21.15.28:0/786452281 >> v2:172.21.15.103:3300/0 conn(0x7fcffc163830 0x7fcffc0d3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:31.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.931+0000 7fcffbe00640 1 -- 172.21.15.28:0/786452281 >> v2:172.21.15.103:3300/0 conn(0x7fcffc163830 msgr2=0x7fcffc0d3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:31.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.931+0000 7fcffbe00640 1 --2- 172.21.15.28:0/786452281 >> v2:172.21.15.103:3300/0 conn(0x7fcffc163830 0x7fcffc0d3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.931+0000 7fcffbe00640 1 -- 172.21.15.28:0/786452281 >> v1:172.21.15.103:6789/0 conn(0x7fcffc162cc0 legacy=0x7fcffc163120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:31.932 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.931+0000 7fcffbe00640 1 -- 172.21.15.28:0/786452281 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcffc0d3080 con 0x7fcffc15f0d0 2024-04-26T21:06:31.933 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.933+0000 7fcffbe00640 1 --2- 172.21.15.28:0/786452281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcffc15f0d0 0x7fcffc06cd60 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fcffc0d5a00 tx=0x7fcfec02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9124c60e45fd0c1e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:31.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.933+0000 7fcffaa00640 1 -- 172.21.15.28:0/786452281 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec037070 con 0x7fcffc15f0d0 2024-04-26T21:06:31.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.934+0000 7fcffaa00640 1 -- 172.21.15.28:0/786452281 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfec032440 con 0x7fcffc15f0d0 2024-04-26T21:06:31.934 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.934+0000 7fcffaa00640 1 -- 172.21.15.28:0/786452281 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfec032810 con 0x7fcffc15f0d0 2024-04-26T21:06:31.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.934+0000 7fd001400640 1 -- 172.21.15.28:0/786452281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcffc15f0d0 msgr2=0x7fcffc06cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:31.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.934+0000 7fd001400640 1 --2- 172.21.15.28:0/786452281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcffc15f0d0 0x7fcffc06cd60 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fcffc0d5a00 tx=0x7fcfec02a230 comp rx=0 tx=0).stop 2024-04-26T21:06:31.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.934+0000 7fd001400640 1 -- 172.21.15.28:0/786452281 shutdown_connections 2024-04-26T21:06:31.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.934+0000 7fd001400640 1 --2- 172.21.15.28:0/786452281 >> v2:172.21.15.103:3300/0 conn(0x7fcffc163830 0x7fcffc0d3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.934+0000 7fd001400640 1 --2- 172.21.15.28:0/786452281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcffc15f0d0 0x7fcffc06cd60 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.934+0000 7fd001400640 1 -- 172.21.15.28:0/786452281 >> 172.21.15.28:0/786452281 conn(0x7fcffc0d4e50 msgr2=0x7fcffc0d5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:31.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.935+0000 7fd001400640 1 -- 172.21.15.28:0/786452281 shutdown_connections 2024-04-26T21:06:31.935 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.935+0000 7fd001400640 1 -- 172.21.15.28:0/786452281 wait complete. 2024-04-26T21:06:31.936 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.937+0000 7fd001400640 1 Processor -- start 2024-04-26T21:06:31.937 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.937+0000 7fd001400640 1 -- start start 2024-04-26T21:06:31.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.938+0000 7fd001400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcffc15f0d0 0x7fcffc1f8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:31.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.938+0000 7fd001400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcffc162cc0 0x7fcffc0ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:31.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.938+0000 7fd001400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcffc163830 0x7fcffc0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:31.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.938+0000 7fd001400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc15f4b0 con 0x7fcffc162cc0 2024-04-26T21:06:31.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.938+0000 7fd001400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcffc06d540 con 0x7fcffc15f0d0 2024-04-26T21:06:31.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.938+0000 7fd001400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fcffc159d30 con 0x7fcffc163830 2024-04-26T21:06:31.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.938+0000 7fcffb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcffc162cc0 0x7fcffc0ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:31.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.938+0000 7fcffb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcffc162cc0 0x7fcffc0ca270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37312/0 (socket says 172.21.15.28:37312) 2024-04-26T21:06:31.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.938+0000 7fcffb400640 1 -- 172.21.15.28:0/758393777 learned_addr learned my addr 172.21.15.28:0/758393777 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:31.939 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.939+0000 7fcffbe00640 1 --2- 172.21.15.28:0/758393777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcffc15f0d0 0x7fcffc1f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:31.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.939+0000 7fd000a00640 1 --2- 172.21.15.28:0/758393777 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcffc163830 0x7fcffc0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:31.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.939+0000 7fcffb400640 1 -- 172.21.15.28:0/758393777 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcffc163830 msgr2=0x7fcffc0ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:31.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.939+0000 7fcffb400640 1 --2- 172.21.15.28:0/758393777 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcffc163830 0x7fcffc0ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.939+0000 7fcffb400640 1 -- 172.21.15.28:0/758393777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcffc15f0d0 msgr2=0x7fcffc1f8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:31.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.939+0000 7fcffb400640 1 --2- 172.21.15.28:0/758393777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcffc15f0d0 0x7fcffc1f8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:31.940 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.939+0000 7fcffb400640 1 -- 172.21.15.28:0/758393777 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcffc15a010 con 0x7fcffc162cc0 2024-04-26T21:06:31.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.939+0000 7fcffb400640 1 --2- 172.21.15.28:0/758393777 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcffc162cc0 0x7fcffc0ca270 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800c430 tx=0x7fcfe800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:31.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.939+0000 7fcff3e00640 1 -- 172.21.15.28:0/758393777 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8011790 con 0x7fcffc162cc0 2024-04-26T21:06:31.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.940+0000 7fcff3e00640 1 -- 172.21.15.28:0/758393777 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfe8017070 con 0x7fcffc162cc0 2024-04-26T21:06:31.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.940+0000 7fd001400640 1 -- 172.21.15.28:0/758393777 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcffc15a2f0 con 0x7fcffc162cc0 2024-04-26T21:06:31.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.940+0000 7fcff3e00640 1 -- 172.21.15.28:0/758393777 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8011b70 con 0x7fcffc162cc0 2024-04-26T21:06:31.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.940+0000 7fd001400640 1 -- 172.21.15.28:0/758393777 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcffc15a8b0 con 0x7fcffc162cc0 2024-04-26T21:06:31.941 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.941+0000 7fd001400640 1 -- 172.21.15.28:0/758393777 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcffc15aab0 con 0x7fcffc162cc0 2024-04-26T21:06:31.942 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.942+0000 7fcff3e00640 1 -- 172.21.15.28:0/758393777 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcfe8011d10 con 0x7fcffc162cc0 2024-04-26T21:06:31.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.943+0000 7fcff3e00640 1 --2- 172.21.15.28:0/758393777 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcfdc06c8f0 0x7fcfdc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:31.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.943+0000 7fcff3e00640 1 -- 172.21.15.28:0/758393777 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fcfe808ba50 con 0x7fcffc162cc0 2024-04-26T21:06:31.943 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.943+0000 7fcffbe00640 1 --2- 172.21.15.28:0/758393777 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcfdc06c8f0 0x7fcfdc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:31.944 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.944+0000 7fcffbe00640 1 --2- 172.21.15.28:0/758393777 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcfdc06c8f0 0x7fcfdc06edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcfec034000 tx=0x7fcfec02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:31.948 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:31.948+0000 7fcff3e00640 1 -- 172.21.15.28:0/758393777 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcffc15aab0 con 0x7fcffc162cc0 2024-04-26T21:06:32.152 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.152+0000 7fd001400640 1 -- 172.21.15.28:0/758393777 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcffc06a2c0 con 0x7fcffc162cc0 2024-04-26T21:06:32.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.153+0000 7fcff3e00640 1 -- 172.21.15.28:0/758393777 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) ==== 100+0+11 (secure 0 0 0) 0x7fcfe8059e40 con 0x7fcffc162cc0 2024-04-26T21:06:32.153 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-04-26T21:06:32.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.156+0000 7fd001400640 1 -- 172.21.15.28:0/758393777 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcfdc06c8f0 msgr2=0x7fcfdc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:32.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.156+0000 7fd001400640 1 --2- 172.21.15.28:0/758393777 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcfdc06c8f0 0x7fcfdc06edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcfec034000 tx=0x7fcfec02e000 comp rx=0 tx=0).stop 2024-04-26T21:06:32.156 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.156+0000 7fd001400640 1 -- 172.21.15.28:0/758393777 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcffc162cc0 msgr2=0x7fcffc0ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:32.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.156+0000 7fd001400640 1 --2- 172.21.15.28:0/758393777 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcffc162cc0 0x7fcffc0ca270 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800c430 tx=0x7fcfe800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:06:32.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.156+0000 7fd001400640 1 -- 172.21.15.28:0/758393777 shutdown_connections 2024-04-26T21:06:32.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.156+0000 7fd001400640 1 --2- 172.21.15.28:0/758393777 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fcfdc06c8f0 0x7fcfdc06edb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:32.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.156+0000 7fd001400640 1 --2- 172.21.15.28:0/758393777 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fcffc163830 0x7fcffc0ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:32.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.156+0000 7fd001400640 1 --2- 172.21.15.28:0/758393777 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcffc162cc0 0x7fcffc0ca270 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:32.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.156+0000 7fd001400640 1 --2- 172.21.15.28:0/758393777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcffc15f0d0 0x7fcffc1f8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:32.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.156+0000 7fd001400640 1 -- 172.21.15.28:0/758393777 >> 172.21.15.28:0/758393777 conn(0x7fcffc0d4e50 msgr2=0x7fcffc1610d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:32.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.156+0000 7fd001400640 1 -- 172.21.15.28:0/758393777 shutdown_connections 2024-04-26T21:06:32.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.156+0000 7fd001400640 1 -- 172.21.15.28:0/758393777 wait complete. 2024-04-26T21:06:32.177 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T21:06:32.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.390+0000 7ffa0ec00640 1 Processor -- start 2024-04-26T21:06:32.390 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.390+0000 7ffa0ec00640 1 -- start start 2024-04-26T21:06:32.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.391+0000 7ffa0ec00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa08145080 0x7ffa080cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:32.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.391+0000 7ffa0ec00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffa08073800 0x7ffa08073c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:32.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.391+0000 7ffa0ec00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa08072760 con 0x7ffa08144b90 2024-04-26T21:06:32.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.391+0000 7ffa0ec00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7ffa08072a40 con 0x7ffa08073800 2024-04-26T21:06:32.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.391+0000 7ffa0ec00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ffa08072d20 con 0x7ffa08145080 2024-04-26T21:06:32.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.392+0000 7ffa0d800640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ffa08144b90 0x7ffa080ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45064/0 (socket says 172.21.15.28:45064) 2024-04-26T21:06:32.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.392+0000 7ffa0d800640 1 -- 172.21.15.28:0/3803452092 learned_addr learned my addr 172.21.15.28:0/3803452092 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:32.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.392+0000 7ffa0e200640 1 --2- 172.21.15.28:0/3803452092 >> v2:172.21.15.28:3300/0 conn(0x7ffa08073800 0x7ffa08073c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:32.392 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.392+0000 7ffa0ce00640 1 --2- 172.21.15.28:0/3803452092 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa08145080 0x7ffa080cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:32.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.392+0000 7ffa07e00640 1 -- 172.21.15.28:0/3803452092 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 491943575 0 0) 0x7ffa08072760 con 0x7ffa08144b90 2024-04-26T21:06:32.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.392+0000 7ffa07e00640 1 -- 172.21.15.28:0/3803452092 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9f0005730 con 0x7ffa08144b90 2024-04-26T21:06:32.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.392+0000 7ffa0e200640 1 -- 172.21.15.28:0/3803452092 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa08145080 msgr2=0x7ffa080cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:32.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.392+0000 7ffa0e200640 1 --2- 172.21.15.28:0/3803452092 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa08145080 0x7ffa080cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:32.393 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.392+0000 7ffa0e200640 1 -- 172.21.15.28:0/3803452092 >> v1:172.21.15.28:6789/0 conn(0x7ffa08144b90 legacy=0x7ffa080ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:32.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.392+0000 7ffa0e200640 1 -- 172.21.15.28:0/3803452092 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa08073000 con 0x7ffa08073800 2024-04-26T21:06:32.394 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.393+0000 7ffa0e200640 1 --2- 172.21.15.28:0/3803452092 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa08073800 0x7ffa08073c80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009d90 tx=0x7ffa0002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a85e5104eed1c22 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:32.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.394+0000 7ffa07e00640 1 -- 172.21.15.28:0/3803452092 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00031600 con 0x7ffa08073800 2024-04-26T21:06:32.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.394+0000 7ffa07e00640 1 -- 172.21.15.28:0/3803452092 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa000317a0 con 0x7ffa08073800 2024-04-26T21:06:32.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.394+0000 7ffa07e00640 1 -- 172.21.15.28:0/3803452092 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00031b70 con 0x7ffa08073800 2024-04-26T21:06:32.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.394+0000 7ffa0ec00640 1 -- 172.21.15.28:0/3803452092 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa08073800 msgr2=0x7ffa08073c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:32.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.394+0000 7ffa0ec00640 1 --2- 172.21.15.28:0/3803452092 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa08073800 0x7ffa08073c80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009d90 tx=0x7ffa0002a5d0 comp rx=0 tx=0).stop 2024-04-26T21:06:32.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.394+0000 7ffa0ec00640 1 -- 172.21.15.28:0/3803452092 shutdown_connections 2024-04-26T21:06:32.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.394+0000 7ffa0ec00640 1 --2- 172.21.15.28:0/3803452092 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa08073800 0x7ffa08073c80 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:32.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.394+0000 7ffa0ec00640 1 --2- 172.21.15.28:0/3803452092 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa08145080 0x7ffa080cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:32.395 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.394+0000 7ffa0ec00640 1 -- 172.21.15.28:0/3803452092 >> 172.21.15.28:0/3803452092 conn(0x7ffa080d0a50 msgr2=0x7ffa080d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:32.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.395+0000 7ffa0ec00640 1 -- 172.21.15.28:0/3803452092 shutdown_connections 2024-04-26T21:06:32.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.395+0000 7ffa0ec00640 1 -- 172.21.15.28:0/3803452092 wait complete. 2024-04-26T21:06:32.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.396+0000 7ffa0ec00640 1 Processor -- start 2024-04-26T21:06:32.396 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.396+0000 7ffa0ec00640 1 -- start start 2024-04-26T21:06:32.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.397+0000 7ffa0ec00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa08073800 0x7ffa080b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:32.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.397+0000 7ffa0ec00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa08144b90 0x7ffa08142100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:32.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.397+0000 7ffa0ec00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa08145080 0x7ffa0813cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:32.399 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.397+0000 7ffa0ec00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7ffa080ced30 con 0x7ffa08145080 2024-04-26T21:06:32.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.397+0000 7ffa0ec00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffa0806e710 con 0x7ffa08144b90 2024-04-26T21:06:32.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.397+0000 7ffa0ec00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7ffa0813c570 con 0x7ffa08073800 2024-04-26T21:06:32.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.398+0000 7ffa0e200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa08145080 0x7ffa0813cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:32.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.398+0000 7ffa0e200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa08145080 0x7ffa0813cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37338/0 (socket says 172.21.15.28:37338) 2024-04-26T21:06:32.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.398+0000 7ffa0e200640 1 -- 172.21.15.28:0/348358776 learned_addr learned my addr 172.21.15.28:0/348358776 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:32.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.398+0000 7ffa0ce00640 1 --2- 172.21.15.28:0/348358776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa08144b90 0x7ffa08142100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:32.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.398+0000 7ffa0d800640 1 --2- 172.21.15.28:0/348358776 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa08073800 0x7ffa080b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:32.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.398+0000 7ffa0e200640 1 -- 172.21.15.28:0/348358776 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa08073800 msgr2=0x7ffa080b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:32.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.398+0000 7ffa0e200640 1 --2- 172.21.15.28:0/348358776 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa08073800 0x7ffa080b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:32.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.398+0000 7ffa0e200640 1 -- 172.21.15.28:0/348358776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa08144b90 msgr2=0x7ffa08142100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:32.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.398+0000 7ffa0e200640 1 --2- 172.21.15.28:0/348358776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa08144b90 0x7ffa08142100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:32.400 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.398+0000 7ffa0e200640 1 -- 172.21.15.28:0/348358776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa080b5290 con 0x7ffa08145080 2024-04-26T21:06:32.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.398+0000 7ffa0ce00640 1 --2- 172.21.15.28:0/348358776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa08144b90 0x7ffa08142100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:32.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.399+0000 7ffa0e200640 1 --2- 172.21.15.28:0/348358776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa08145080 0x7ffa0813cce0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009eb0 tx=0x7ffa00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:32.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.399+0000 7ffa05600640 1 -- 172.21.15.28:0/348358776 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00040070 con 0x7ffa08145080 2024-04-26T21:06:32.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.399+0000 7ffa05600640 1 -- 172.21.15.28:0/348358776 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa00030b60 con 0x7ffa08145080 2024-04-26T21:06:32.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.399+0000 7ffa05600640 1 -- 172.21.15.28:0/348358776 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00009450 con 0x7ffa08145080 2024-04-26T21:06:32.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.399+0000 7ffa0ec00640 1 -- 172.21.15.28:0/348358776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa080b5570 con 0x7ffa08145080 2024-04-26T21:06:32.401 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.399+0000 7ffa0ec00640 1 -- 172.21.15.28:0/348358776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa080b4ed0 con 0x7ffa08145080 2024-04-26T21:06:32.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.400+0000 7ffa0ec00640 1 -- 172.21.15.28:0/348358776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa0806a360 con 0x7ffa08145080 2024-04-26T21:06:32.402 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.401+0000 7ffa05600640 1 -- 172.21.15.28:0/348358776 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa00039040 con 0x7ffa08145080 2024-04-26T21:06:32.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.402+0000 7ffa05600640 1 --2- 172.21.15.28:0/348358776 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff9e806c8f0 0x7ff9e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:32.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.402+0000 7ffa05600640 1 -- 172.21.15.28:0/348358776 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7ffa000ad250 con 0x7ffa08145080 2024-04-26T21:06:32.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.403+0000 7ffa0d800640 1 --2- 172.21.15.28:0/348358776 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff9e806c8f0 0x7ff9e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:32.403 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.403+0000 7ffa0d800640 1 --2- 172.21.15.28:0/348358776 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff9e806c8f0 0x7ff9e806edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc005880 tx=0x7ff9fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:32.407 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.407+0000 7ffa05600640 1 -- 172.21.15.28:0/348358776 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ffa0007b750 con 0x7ffa08145080 2024-04-26T21:06:32.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.629+0000 7ffa0ec00640 1 -- 172.21.15.28:0/348358776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ffa080cea80 con 0x7ffa08145080 2024-04-26T21:06:32.630 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.630+0000 7ffa05600640 1 -- 172.21.15.28:0/348358776 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) ==== 80+0+0 (secure 0 0 0) 0x7ffa00044e80 con 0x7ffa08145080 2024-04-26T21:06:32.631 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2024-04-26T21:06:32.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.633+0000 7ffa0ec00640 1 -- 172.21.15.28:0/348358776 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff9e806c8f0 msgr2=0x7ff9e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:32.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.633+0000 7ffa0ec00640 1 --2- 172.21.15.28:0/348358776 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff9e806c8f0 0x7ff9e806edb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc005880 tx=0x7ff9fc009040 comp rx=0 tx=0).stop 2024-04-26T21:06:32.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.633+0000 7ffa0ec00640 1 -- 172.21.15.28:0/348358776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa08145080 msgr2=0x7ffa0813cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:32.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.633+0000 7ffa0ec00640 1 --2- 172.21.15.28:0/348358776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa08145080 0x7ffa0813cce0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009eb0 tx=0x7ffa00004a40 comp rx=0 tx=0).stop 2024-04-26T21:06:32.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.633+0000 7ffa0ec00640 1 -- 172.21.15.28:0/348358776 shutdown_connections 2024-04-26T21:06:32.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.633+0000 7ffa0ec00640 1 --2- 172.21.15.28:0/348358776 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7ff9e806c8f0 0x7ff9e806edb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:32.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.633+0000 7ffa0ec00640 1 --2- 172.21.15.28:0/348358776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa08145080 0x7ffa0813cce0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:32.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.633+0000 7ffa0ec00640 1 --2- 172.21.15.28:0/348358776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa08144b90 0x7ffa08142100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:32.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.633+0000 7ffa0ec00640 1 --2- 172.21.15.28:0/348358776 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7ffa08073800 0x7ffa080b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:32.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.633+0000 7ffa0ec00640 1 -- 172.21.15.28:0/348358776 >> 172.21.15.28:0/348358776 conn(0x7ffa080d0a50 msgr2=0x7ffa0806ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:32.634 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.634+0000 7ffa0ec00640 1 -- 172.21.15.28:0/348358776 shutdown_connections 2024-04-26T21:06:32.635 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:32.634+0000 7ffa0ec00640 1 -- 172.21.15.28:0/348358776 wait complete. 2024-04-26T21:06:32.654 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi028.front.sepia.ceph.com... 2024-04-26T21:06:32.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:06:32.655 DEBUG:teuthology.orchestra.run.smithi028:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-26T21:06:32.733 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:ceph-fuse[48828]: fuse finished with error 0 and tester_r 0 2024-04-26T21:06:32.757 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T21:06:32.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:06:32.757 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T21:06:32.953 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:06:32.954 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T21:06:32.954 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:06:32.993 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052734,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:06:32.994 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:06:32.994 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T21:06:32.994 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:06:33.033 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:06:33.034 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T21:06:33.034 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T21:06:33.034 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:06:33.080 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052734,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:06:33.080 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:06:33.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.277+0000 7efc83e00640 1 Processor -- start 2024-04-26T21:06:33.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.277+0000 7efc83e00640 1 -- start start 2024-04-26T21:06:33.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.279+0000 7efc83e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7efc840c9da0 0x7efc840ca180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:33.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.279+0000 7efc83e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7efc840cb0a0 0x7efc84162bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:33.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.279+0000 7efc83e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efc840cb500 0x7efc84163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:33.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.279+0000 7efc83e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7efc841603f0 con 0x7efc840c9da0 2024-04-26T21:06:33.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.279+0000 7efc83e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7efc8415f580 con 0x7efc840cb0a0 2024-04-26T21:06:33.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.279+0000 7efc83e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7efc84145a10 con 0x7efc840cb500 2024-04-26T21:06:33.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.279+0000 7efc82a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7efc840c9da0 0x7efc840ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:33.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.279+0000 7efc82a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7efc840c9da0 0x7efc840ca180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37350/0 (socket says 172.21.15.28:37350) 2024-04-26T21:06:33.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.279+0000 7efc82a00640 1 -- 172.21.15.28:0/2623714646 learned_addr learned my addr 172.21.15.28:0/2623714646 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:33.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.280+0000 7efc7be00640 1 --2- 172.21.15.28:0/2623714646 >> v2:172.21.15.103:3300/0 conn(0x7efc840cb0a0 0x7efc84162bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:33.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.280+0000 7efc82a00640 1 -- 172.21.15.28:0/2623714646 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efc840cb500 msgr2=0x7efc84163100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:33.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.280+0000 7efc82a00640 1 --2- 172.21.15.28:0/2623714646 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efc840cb500 0x7efc84163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.280+0000 7efc82a00640 1 -- 172.21.15.28:0/2623714646 >> v2:172.21.15.103:3300/0 conn(0x7efc840cb0a0 msgr2=0x7efc84162bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:33.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.280+0000 7efc82a00640 1 --2- 172.21.15.28:0/2623714646 >> v2:172.21.15.103:3300/0 conn(0x7efc840cb0a0 0x7efc84162bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.280+0000 7efc82a00640 1 -- 172.21.15.28:0/2623714646 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc840ca6c0 con 0x7efc840c9da0 2024-04-26T21:06:33.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.280+0000 7efc7be00640 1 --2- 172.21.15.28:0/2623714646 >> v2:172.21.15.103:3300/0 conn(0x7efc840cb0a0 0x7efc84162bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:33.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.281+0000 7efc82a00640 1 --2- 172.21.15.28:0/2623714646 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efc840c9da0 0x7efc840ca180 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7efc84071a90 tx=0x7efc7402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c8cdc025556fdd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:33.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.281+0000 7efc82000640 1 -- 172.21.15.28:0/2623714646 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74037070 con 0x7efc840c9da0 2024-04-26T21:06:33.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.281+0000 7efc82000640 1 -- 172.21.15.28:0/2623714646 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc74032440 con 0x7efc840c9da0 2024-04-26T21:06:33.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.281+0000 7efc82000640 1 -- 172.21.15.28:0/2623714646 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74032810 con 0x7efc840c9da0 2024-04-26T21:06:33.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.282+0000 7efc83e00640 1 -- 172.21.15.28:0/2623714646 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efc840c9da0 msgr2=0x7efc840ca180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:33.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.282+0000 7efc83e00640 1 --2- 172.21.15.28:0/2623714646 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efc840c9da0 0x7efc840ca180 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7efc84071a90 tx=0x7efc7402a210 comp rx=0 tx=0).stop 2024-04-26T21:06:33.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.282+0000 7efc83e00640 1 -- 172.21.15.28:0/2623714646 shutdown_connections 2024-04-26T21:06:33.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.282+0000 7efc83e00640 1 --2- 172.21.15.28:0/2623714646 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efc840cb500 0x7efc84163100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.282+0000 7efc83e00640 1 --2- 172.21.15.28:0/2623714646 >> v2:172.21.15.103:3300/0 conn(0x7efc840cb0a0 0x7efc84162bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.282+0000 7efc83e00640 1 --2- 172.21.15.28:0/2623714646 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efc840c9da0 0x7efc840ca180 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.282+0000 7efc83e00640 1 -- 172.21.15.28:0/2623714646 >> 172.21.15.28:0/2623714646 conn(0x7efc840d0a50 msgr2=0x7efc840d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:33.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.282+0000 7efc83e00640 1 -- 172.21.15.28:0/2623714646 shutdown_connections 2024-04-26T21:06:33.283 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.282+0000 7efc83e00640 1 -- 172.21.15.28:0/2623714646 wait complete. 2024-04-26T21:06:33.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.284+0000 7efc83e00640 1 Processor -- start 2024-04-26T21:06:33.284 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.284+0000 7efc83e00640 1 -- start start 2024-04-26T21:06:33.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.285+0000 7efc83e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efc840c9da0 0x7efc840b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:33.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.285+0000 7efc83e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efc840cb0a0 0x7efc841523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:33.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.285+0000 7efc83e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc840cb500 0x7efc84163170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:33.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.285+0000 7efc83e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efc8415fda0 con 0x7efc840cb0a0 2024-04-26T21:06:33.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.285+0000 7efc83e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efc8415fac0 con 0x7efc840cb500 2024-04-26T21:06:33.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.285+0000 7efc83e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7efc84160080 con 0x7efc840c9da0 2024-04-26T21:06:33.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.285+0000 7efc7be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efc840cb0a0 0x7efc841523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:33.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.285+0000 7efc7be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efc840cb0a0 0x7efc841523a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37356/0 (socket says 172.21.15.28:37356) 2024-04-26T21:06:33.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.285+0000 7efc7be00640 1 -- 172.21.15.28:0/4091011359 learned_addr learned my addr 172.21.15.28:0/4091011359 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:33.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.286+0000 7efc83400640 1 --2- 172.21.15.28:0/4091011359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc840cb500 0x7efc84163170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:33.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.286+0000 7efc82a00640 1 --2- 172.21.15.28:0/4091011359 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efc840c9da0 0x7efc840b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:33.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.286+0000 7efc7be00640 1 -- 172.21.15.28:0/4091011359 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efc840c9da0 msgr2=0x7efc840b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:33.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.286+0000 7efc7be00640 1 --2- 172.21.15.28:0/4091011359 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efc840c9da0 0x7efc840b5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.286+0000 7efc7be00640 1 -- 172.21.15.28:0/4091011359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc840cb500 msgr2=0x7efc84163170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:33.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.286+0000 7efc7be00640 1 --2- 172.21.15.28:0/4091011359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc840cb500 0x7efc84163170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.286+0000 7efc7be00640 1 -- 172.21.15.28:0/4091011359 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc840b6540 con 0x7efc840cb0a0 2024-04-26T21:06:33.287 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.286+0000 7efc7be00640 1 --2- 172.21.15.28:0/4091011359 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efc840cb0a0 0x7efc841523a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00c7f0 tx=0x7efc6c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:33.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.286+0000 7efc7aa00640 1 -- 172.21.15.28:0/4091011359 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c015070 con 0x7efc840cb0a0 2024-04-26T21:06:33.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.286+0000 7efc7aa00640 1 -- 172.21.15.28:0/4091011359 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc6c007a30 con 0x7efc840cb0a0 2024-04-26T21:06:33.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.286+0000 7efc7aa00640 1 -- 172.21.15.28:0/4091011359 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc6c007e00 con 0x7efc840cb0a0 2024-04-26T21:06:33.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.286+0000 7efc83e00640 1 -- 172.21.15.28:0/4091011359 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc84072010 con 0x7efc840cb0a0 2024-04-26T21:06:33.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.287+0000 7efc83e00640 1 -- 172.21.15.28:0/4091011359 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc84154250 con 0x7efc840cb0a0 2024-04-26T21:06:33.288 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.288+0000 7efc83e00640 1 -- 172.21.15.28:0/4091011359 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc84071e00 con 0x7efc840cb0a0 2024-04-26T21:06:33.289 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.289+0000 7efc7aa00640 1 -- 172.21.15.28:0/4091011359 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc6c018070 con 0x7efc840cb0a0 2024-04-26T21:06:33.290 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.290+0000 7efc7aa00640 1 --2- 172.21.15.28:0/4091011359 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efc6006c940 0x7efc6006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:33.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.290+0000 7efc7aa00640 1 -- 172.21.15.28:0/4091011359 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7efc6c08bb00 con 0x7efc840cb0a0 2024-04-26T21:06:33.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.291+0000 7efc82a00640 1 --2- 172.21.15.28:0/4091011359 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efc6006c940 0x7efc6006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:33.291 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.291+0000 7efc82a00640 1 --2- 172.21.15.28:0/4091011359 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efc6006c940 0x7efc6006ee00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efc74034000 tx=0x7efc7402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:33.295 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.295+0000 7efc7aa00640 1 -- 172.21.15.28:0/4091011359 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efc6c059f60 con 0x7efc840cb0a0 2024-04-26T21:06:33.538 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.538+0000 7efc83e00640 1 -- 172.21.15.28:0/4091011359 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc8406a360 con 0x7efc840cb0a0 2024-04-26T21:06:33.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.539+0000 7efc7aa00640 1 -- 172.21.15.28:0/4091011359 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5184 (secure 0 0 0) 0x7efc6c059860 con 0x7efc840cb0a0 2024-04-26T21:06:33.540 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T21:06:33.540 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/356870645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":356870645},{"type":"v1","addr":"172.21.15.28:6835","nonce":356870645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4829,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2957511478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":2957511478},{"type":"v1","addr":"172.21.15.103:6835","nonce":2957511478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/2060632176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2060632176},{"type":"v1","addr":"172.21.15.103:6834","nonce":2060632176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:58:43.534341+0000","modified":"2024-04-26T20:58:46.581816+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5571},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6837/2997047727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":2997047727},{"type":"v1","addr":"172.21.15.28:6837","nonce":2997047727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5571,"qdb_cluster":[5571]},"id":3}]} 2024-04-26T21:06:33.540 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 26 2024-04-26T21:06:33.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.542+0000 7efc83e00640 1 -- 172.21.15.28:0/4091011359 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efc6006c940 msgr2=0x7efc6006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:33.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.542+0000 7efc83e00640 1 --2- 172.21.15.28:0/4091011359 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efc6006c940 0x7efc6006ee00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7efc74034000 tx=0x7efc7402e000 comp rx=0 tx=0).stop 2024-04-26T21:06:33.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.542+0000 7efc83e00640 1 -- 172.21.15.28:0/4091011359 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efc840cb0a0 msgr2=0x7efc841523a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:33.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.542+0000 7efc83e00640 1 --2- 172.21.15.28:0/4091011359 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efc840cb0a0 0x7efc841523a0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00c7f0 tx=0x7efc6c00cbf0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.542+0000 7efc83e00640 1 -- 172.21.15.28:0/4091011359 shutdown_connections 2024-04-26T21:06:33.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.542+0000 7efc83e00640 1 --2- 172.21.15.28:0/4091011359 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efc6006c940 0x7efc6006ee00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.542+0000 7efc83e00640 1 --2- 172.21.15.28:0/4091011359 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efc840cb500 0x7efc84163170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.542+0000 7efc83e00640 1 --2- 172.21.15.28:0/4091011359 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efc840cb0a0 0x7efc841523a0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.542+0000 7efc83e00640 1 --2- 172.21.15.28:0/4091011359 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efc840c9da0 0x7efc840b5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.542+0000 7efc83e00640 1 -- 172.21.15.28:0/4091011359 >> 172.21.15.28:0/4091011359 conn(0x7efc840d0a50 msgr2=0x7efc840b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:33.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.543+0000 7efc83e00640 1 -- 172.21.15.28:0/4091011359 shutdown_connections 2024-04-26T21:06:33.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.543+0000 7efc83e00640 1 -- 172.21.15.28:0/4091011359 wait complete. 2024-04-26T21:06:33.562 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:06:33.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.766+0000 7f239f200640 1 Processor -- start 2024-04-26T21:06:33.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.767+0000 7f239f200640 1 -- start start 2024-04-26T21:06:33.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.768+0000 7f239f200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f23a014c300 0x7f23a00ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:33.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.768+0000 7f239f200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f23a014ce60 0x7f23a00ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:33.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.768+0000 7f239f200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f23a00cf0a0 0x7f23a0167080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:33.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.768+0000 7f239f200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f23a014c6e0 con 0x7f23a014ce60 2024-04-26T21:06:33.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.768+0000 7f239f200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f23a006f9e0 con 0x7f23a00cf0a0 2024-04-26T21:06:33.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.768+0000 7f239f200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f23a0070190 con 0x7f23a014c300 2024-04-26T21:06:33.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.769+0000 7f239d400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f23a014ce60 0x7f23a00ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:33.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.769+0000 7f239d400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f23a014ce60 0x7f23a00ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37364/0 (socket says 172.21.15.28:37364) 2024-04-26T21:06:33.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.769+0000 7f239d400640 1 -- 172.21.15.28:0/2394592270 learned_addr learned my addr 172.21.15.28:0/2394592270 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:33.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.769+0000 7f239e800640 1 --2- 172.21.15.28:0/2394592270 >> v2:172.21.15.103:3300/0 conn(0x7f23a00cf0a0 0x7f23a0167080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:33.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.769+0000 7f239de00640 1 --2- 172.21.15.28:0/2394592270 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f23a014c300 0x7f23a00ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:33.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.769+0000 7f239d400640 1 -- 172.21.15.28:0/2394592270 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f23a014c300 msgr2=0x7f23a00ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:33.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.769+0000 7f239d400640 1 --2- 172.21.15.28:0/2394592270 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f23a014c300 0x7f23a00ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.769+0000 7f239d400640 1 -- 172.21.15.28:0/2394592270 >> v2:172.21.15.103:3300/0 conn(0x7f23a00cf0a0 msgr2=0x7f23a0167080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:33.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.769+0000 7f239d400640 1 --2- 172.21.15.28:0/2394592270 >> v2:172.21.15.103:3300/0 conn(0x7f23a00cf0a0 0x7f23a0167080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.769+0000 7f239d400640 1 -- 172.21.15.28:0/2394592270 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a006fc10 con 0x7f23a014ce60 2024-04-26T21:06:33.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.770+0000 7f239d400640 1 --2- 172.21.15.28:0/2394592270 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23a014ce60 0x7f23a00ceb60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f23a014cd60 tx=0x7f238802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bc85d7df0009b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:33.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.771+0000 7f239ca00640 1 -- 172.21.15.28:0/2394592270 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388036070 con 0x7f23a014ce60 2024-04-26T21:06:33.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.771+0000 7f239ca00640 1 -- 172.21.15.28:0/2394592270 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2388031600 con 0x7f23a014ce60 2024-04-26T21:06:33.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.771+0000 7f239ca00640 1 -- 172.21.15.28:0/2394592270 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23880319d0 con 0x7f23a014ce60 2024-04-26T21:06:33.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.771+0000 7f239f200640 1 -- 172.21.15.28:0/2394592270 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23a014ce60 msgr2=0x7f23a00ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:33.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.771+0000 7f239f200640 1 --2- 172.21.15.28:0/2394592270 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23a014ce60 0x7f23a00ceb60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f23a014cd60 tx=0x7f238802a6a0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.772+0000 7f239f200640 1 -- 172.21.15.28:0/2394592270 shutdown_connections 2024-04-26T21:06:33.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.772+0000 7f239f200640 1 --2- 172.21.15.28:0/2394592270 >> v2:172.21.15.103:3300/0 conn(0x7f23a00cf0a0 0x7f23a0167080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.772+0000 7f239f200640 1 --2- 172.21.15.28:0/2394592270 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23a014ce60 0x7f23a00ceb60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.772+0000 7f239f200640 1 --2- 172.21.15.28:0/2394592270 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f23a014c300 0x7f23a00ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.772+0000 7f239f200640 1 -- 172.21.15.28:0/2394592270 >> 172.21.15.28:0/2394592270 conn(0x7f23a00d0a50 msgr2=0x7f23a00d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:33.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.772+0000 7f239f200640 1 -- 172.21.15.28:0/2394592270 shutdown_connections 2024-04-26T21:06:33.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.772+0000 7f239f200640 1 -- 172.21.15.28:0/2394592270 wait complete. 2024-04-26T21:06:33.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.773+0000 7f239f200640 1 Processor -- start 2024-04-26T21:06:33.774 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.774+0000 7f239f200640 1 -- start start 2024-04-26T21:06:33.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.775+0000 7f239f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23a00cf0a0 0x7f23a0144b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:33.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.775+0000 7f239f200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23a014c300 0x7f23a0148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:33.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.775+0000 7f239f200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f23a014ce60 0x7f23a0143080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:33.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.775+0000 7f239f200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f23a006e4d0 con 0x7f23a014c300 2024-04-26T21:06:33.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.775+0000 7f239f200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f23a006feb0 con 0x7f23a00cf0a0 2024-04-26T21:06:33.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.775+0000 7f239f200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f23a006de80 con 0x7f23a014ce60 2024-04-26T21:06:33.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.775+0000 7f239d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23a014c300 0x7f23a0148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:33.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.775+0000 7f239d400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23a014c300 0x7f23a0148670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37368/0 (socket says 172.21.15.28:37368) 2024-04-26T21:06:33.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.775+0000 7f239d400640 1 -- 172.21.15.28:0/928292838 learned_addr learned my addr 172.21.15.28:0/928292838 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:33.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.775+0000 7f239de00640 1 --2- 172.21.15.28:0/928292838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23a00cf0a0 0x7f23a0144b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:33.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.775+0000 7f239e800640 1 --2- 172.21.15.28:0/928292838 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f23a014ce60 0x7f23a0143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:33.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.775+0000 7f239d400640 1 -- 172.21.15.28:0/928292838 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f23a014ce60 msgr2=0x7f23a0143080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:33.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.775+0000 7f239d400640 1 --2- 172.21.15.28:0/928292838 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f23a014ce60 0x7f23a0143080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.775+0000 7f239d400640 1 -- 172.21.15.28:0/928292838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23a00cf0a0 msgr2=0x7f23a0144b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:33.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.775+0000 7f239d400640 1 --2- 172.21.15.28:0/928292838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23a00cf0a0 0x7f23a0144b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.775+0000 7f239d400640 1 -- 172.21.15.28:0/928292838 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a00b49e0 con 0x7f23a014c300 2024-04-26T21:06:33.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.776+0000 7f239d400640 1 --2- 172.21.15.28:0/928292838 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23a014c300 0x7f23a0148670 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2388000f80 tx=0x7f2388004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:33.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.776+0000 7f2396000640 1 -- 172.21.15.28:0/928292838 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388036050 con 0x7f23a014c300 2024-04-26T21:06:33.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.776+0000 7f2396000640 1 -- 172.21.15.28:0/928292838 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2388042070 con 0x7f23a014c300 2024-04-26T21:06:33.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.776+0000 7f2396000640 1 -- 172.21.15.28:0/928292838 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388009510 con 0x7f23a014c300 2024-04-26T21:06:33.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.776+0000 7f239f200640 1 -- 172.21.15.28:0/928292838 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a00b4c80 con 0x7f23a014c300 2024-04-26T21:06:33.777 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.776+0000 7f239f200640 1 -- 172.21.15.28:0/928292838 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a00b5240 con 0x7f23a014c300 2024-04-26T21:06:33.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.777+0000 7f239f200640 1 -- 172.21.15.28:0/928292838 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23a006a360 con 0x7f23a014c300 2024-04-26T21:06:33.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.778+0000 7f2396000640 1 -- 172.21.15.28:0/928292838 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2388007c70 con 0x7f23a014c300 2024-04-26T21:06:33.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.779+0000 7f2396000640 1 --2- 172.21.15.28:0/928292838 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f237c06c8f0 0x7f237c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:33.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.779+0000 7f2396000640 1 -- 172.21.15.28:0/928292838 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f23880acb50 con 0x7f23a014c300 2024-04-26T21:06:33.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.780+0000 7f239de00640 1 --2- 172.21.15.28:0/928292838 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f237c06c8f0 0x7f237c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:33.780 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.780+0000 7f239de00640 1 --2- 172.21.15.28:0/928292838 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f237c06c8f0 0x7f237c06edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f239000a970 tx=0x7f2390008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:33.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.784+0000 7f2396000640 1 -- 172.21.15.28:0/928292838 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f238807aea0 con 0x7f23a014c300 2024-04-26T21:06:33.989 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.989+0000 7f239f200640 1 -- 172.21.15.28:0/928292838 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23a0074550 con 0x7f23a014c300 2024-04-26T21:06:33.991 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.991+0000 7f2396000640 1 -- 172.21.15.28:0/928292838 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16685 (secure 0 0 0) 0x7f2388081070 con 0x7f23a014c300 2024-04-26T21:06:33.991 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T21:06:33.991 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":47,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:59:38.275251+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:58:41.306286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T20:58:42.497340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.16","mappings":[{"from":0,"to":5}]},{"pgid":"7.30","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:06:33.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.994+0000 7f239f200640 1 -- 172.21.15.28:0/928292838 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f237c06c8f0 msgr2=0x7f237c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:33.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.994+0000 7f239f200640 1 --2- 172.21.15.28:0/928292838 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f237c06c8f0 0x7f237c06edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f239000a970 tx=0x7f2390008040 comp rx=0 tx=0).stop 2024-04-26T21:06:33.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.994+0000 7f239f200640 1 -- 172.21.15.28:0/928292838 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23a014c300 msgr2=0x7f23a0148670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:33.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.994+0000 7f239f200640 1 --2- 172.21.15.28:0/928292838 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23a014c300 0x7f23a0148670 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2388000f80 tx=0x7f2388004a40 comp rx=0 tx=0).stop 2024-04-26T21:06:33.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.994+0000 7f239f200640 1 -- 172.21.15.28:0/928292838 shutdown_connections 2024-04-26T21:06:33.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.994+0000 7f239f200640 1 --2- 172.21.15.28:0/928292838 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f237c06c8f0 0x7f237c06edb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.994+0000 7f239f200640 1 --2- 172.21.15.28:0/928292838 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f23a014ce60 0x7f23a0143080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.994+0000 7f239f200640 1 --2- 172.21.15.28:0/928292838 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f23a014c300 0x7f23a0148670 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.994+0000 7f239f200640 1 --2- 172.21.15.28:0/928292838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f23a00cf0a0 0x7f23a0144b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:33.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.994+0000 7f239f200640 1 -- 172.21.15.28:0/928292838 >> 172.21.15.28:0/928292838 conn(0x7f23a00d0a50 msgr2=0x7f23a0072030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:33.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.994+0000 7f239f200640 1 -- 172.21.15.28:0/928292838 shutdown_connections 2024-04-26T21:06:33.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:33.994+0000 7f239f200640 1 -- 172.21.15.28:0/928292838 wait complete. 2024-04-26T21:06:34.015 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T20:56:44.737461+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T20:58:41.306286+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T20:58:42.497340+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T21:06:34.016 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T21:06:34.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.232+0000 7f4bde400640 1 Processor -- start 2024-04-26T21:06:34.233 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.232+0000 7f4bde400640 1 -- start start 2024-04-26T21:06:34.234 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.234+0000 7f4bde400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bd80ceaa0 0x7f4bd806e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:34.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.234+0000 7f4bde400640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bd8074400 con 0x7f4bd806fc10 2024-04-26T21:06:34.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.234+0000 7f4bde400640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bd815e310 con 0x7f4bd8073f50 2024-04-26T21:06:34.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.234+0000 7f4bde400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd8072aa0 con 0x7f4bd80ceaa0 2024-04-26T21:06:34.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.235+0000 7f4bd7400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f4bd806fc10 0x7f4bd8073580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45074/0 (socket says 172.21.15.28:45074) 2024-04-26T21:06:34.235 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.235+0000 7f4bd7400640 1 -- 172.21.15.28:0/1360779843 learned_addr learned my addr 172.21.15.28:0/1360779843 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:34.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.235+0000 7f4bd6a00640 1 -- 172.21.15.28:0/1360779843 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2077054962 0 0) 0x7f4bd8074400 con 0x7f4bd806fc10 2024-04-26T21:06:34.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.235+0000 7f4bd6a00640 1 -- 172.21.15.28:0/1360779843 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bbc003520 con 0x7f4bd806fc10 2024-04-26T21:06:34.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.235+0000 7f4bd7e00640 1 --2- 172.21.15.28:0/1360779843 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bd80ceaa0 0x7f4bd806e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:34.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.236+0000 7f4bd6a00640 1 -- 172.21.15.28:0/1360779843 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3216556173 0 0) 0x7f4bbc003520 con 0x7f4bd806fc10 2024-04-26T21:06:34.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.236+0000 7f4bd6a00640 1 -- 172.21.15.28:0/1360779843 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bd80ceaa0 msgr2=0x7f4bd806e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:34.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.236+0000 7f4bd6a00640 1 --2- 172.21.15.28:0/1360779843 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bd80ceaa0 0x7f4bd806e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:34.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.236+0000 7f4bd6a00640 1 -- 172.21.15.28:0/1360779843 >> v1:172.21.15.103:6789/0 conn(0x7f4bd8073f50 legacy=0x7f4bd80ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:34.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.236+0000 7f4bd6a00640 1 -- 172.21.15.28:0/1360779843 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd8072d80 con 0x7f4bd806fc10 2024-04-26T21:06:34.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.236+0000 7f4bd6a00640 1 -- 172.21.15.28:0/1360779843 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f4bc8001690 con 0x7f4bd806fc10 2024-04-26T21:06:34.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.236+0000 7f4bd6a00640 1 -- 172.21.15.28:0/1360779843 >> v1:172.21.15.28:6789/0 conn(0x7f4bd806fc10 legacy=0x7f4bd8073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:34.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.236+0000 7f4bd6a00640 1 --2- 172.21.15.28:0/1360779843 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bd80ceaa0 0x7f4bbc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:34.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.236+0000 7f4bd6a00640 1 --2- 172.21.15.28:0/1360779843 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bbc004010 0x7f4bbc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:34.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.236+0000 7f4bd6a00640 1 --2- 172.21.15.28:0/1360779843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bbc004d40 0x7f4bbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:34.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.236+0000 7f4bd6a00640 1 -- 172.21.15.28:0/1360779843 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd8072aa0 con 0x7f4bbc004010 2024-04-26T21:06:34.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.236+0000 7f4bd6a00640 1 -- 172.21.15.28:0/1360779843 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4bbc00bc00 con 0x7f4bbc004d40 2024-04-26T21:06:34.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.236+0000 7f4bd6a00640 1 -- 172.21.15.28:0/1360779843 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f4bbc00be00 con 0x7f4bd80ceaa0 2024-04-26T21:06:34.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.236+0000 7f4bcfe00640 1 --2- 172.21.15.28:0/1360779843 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bbc004010 0x7f4bbc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:34.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.236+0000 7f4bd7400640 1 --2- 172.21.15.28:0/1360779843 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bd80ceaa0 0x7f4bbc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:34.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.237+0000 7f4bd7e00640 1 --2- 172.21.15.28:0/1360779843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bbc004d40 0x7f4bbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:34.238 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.237+0000 7f4bcfe00640 1 -- 172.21.15.28:0/1360779843 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bd80ceaa0 msgr2=0x7f4bbc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:34.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.237+0000 7f4bcfe00640 1 --2- 172.21.15.28:0/1360779843 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bd80ceaa0 0x7f4bbc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:34.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.237+0000 7f4bcfe00640 1 -- 172.21.15.28:0/1360779843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bbc004d40 msgr2=0x7f4bbc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:34.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.237+0000 7f4bcfe00640 1 --2- 172.21.15.28:0/1360779843 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bbc004d40 0x7f4bbc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:34.239 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.237+0000 7f4bcfe00640 1 -- 172.21.15.28:0/1360779843 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bbc003520 con 0x7f4bbc004010 2024-04-26T21:06:34.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.238+0000 7f4bcfe00640 1 --2- 172.21.15.28:0/1360779843 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bbc004010 0x7f4bbc004800 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0001750 tx=0x7f4bbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1263937a459339 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:34.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.238+0000 7f4bd6a00640 1 -- 172.21.15.28:0/1360779843 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0001370 con 0x7f4bbc004010 2024-04-26T21:06:34.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.238+0000 7f4bd6a00640 1 -- 172.21.15.28:0/1360779843 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc00052d0 con 0x7f4bbc004010 2024-04-26T21:06:34.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.238+0000 7f4bd6a00640 1 -- 172.21.15.28:0/1360779843 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc002abe0 con 0x7f4bbc004010 2024-04-26T21:06:34.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.239+0000 7f4bde400640 1 -- 172.21.15.28:0/1360779843 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bbc004010 msgr2=0x7f4bbc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:34.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.239+0000 7f4bde400640 1 --2- 172.21.15.28:0/1360779843 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bbc004010 0x7f4bbc004800 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0001750 tx=0x7f4bbc005830 comp rx=0 tx=0).stop 2024-04-26T21:06:34.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.239+0000 7f4bd7e00640 1 -- 172.21.15.28:0/1360779843 reap_dead start 2024-04-26T21:06:34.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.239+0000 7f4bde400640 1 -- 172.21.15.28:0/1360779843 shutdown_connections 2024-04-26T21:06:34.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.239+0000 7f4bde400640 1 -- 172.21.15.28:0/1360779843 >> 172.21.15.28:0/1360779843 conn(0x7f4bd806c770 msgr2=0x7f4bd806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:34.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.239+0000 7f4bde400640 1 -- 172.21.15.28:0/1360779843 shutdown_connections 2024-04-26T21:06:34.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.239+0000 7f4bde400640 1 -- 172.21.15.28:0/1360779843 wait complete. 2024-04-26T21:06:34.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.241+0000 7f4bde400640 1 Processor -- start 2024-04-26T21:06:34.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.241+0000 7f4bde400640 1 -- start start 2024-04-26T21:06:34.242 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.242+0000 7f4bde400640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bbc004010 0x7f4bd80afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:34.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.242+0000 7f4bde400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bd814b7e0 0x7f4bd814afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:34.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.242+0000 7f4bde400640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bd80ae720 0x7f4bd81e63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:34.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.242+0000 7f4bde400640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd80d07d0 con 0x7f4bd814b7e0 2024-04-26T21:06:34.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.242+0000 7f4bde400640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd815dfa0 con 0x7f4bd80ae720 2024-04-26T21:06:34.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.242+0000 7f4bde400640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd80d0950 con 0x7f4bbc004010 2024-04-26T21:06:34.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.242+0000 7f4bcfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bd814b7e0 0x7f4bd814afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:34.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.242+0000 7f4bcfe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bd814b7e0 0x7f4bd814afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37390/0 (socket says 172.21.15.28:37390) 2024-04-26T21:06:34.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.242+0000 7f4bcfe00640 1 -- 172.21.15.28:0/1903465571 learned_addr learned my addr 172.21.15.28:0/1903465571 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:34.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.243+0000 7f4bd7400640 1 --2- 172.21.15.28:0/1903465571 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bbc004010 0x7f4bd80afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:34.243 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.243+0000 7f4bd7e00640 1 --2- 172.21.15.28:0/1903465571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bd80ae720 0x7f4bd81e63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:34.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.243+0000 7f4bcfe00640 1 -- 172.21.15.28:0/1903465571 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bbc004010 msgr2=0x7f4bd80afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:34.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.243+0000 7f4bcfe00640 1 --2- 172.21.15.28:0/1903465571 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bbc004010 0x7f4bd80afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:34.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.243+0000 7f4bcfe00640 1 -- 172.21.15.28:0/1903465571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bd80ae720 msgr2=0x7f4bd81e63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:34.244 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.243+0000 7f4bcfe00640 1 --2- 172.21.15.28:0/1903465571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bd80ae720 0x7f4bd81e63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:34.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.243+0000 7f4bcfe00640 1 -- 172.21.15.28:0/1903465571 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd814b520 con 0x7f4bd814b7e0 2024-04-26T21:06:34.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.243+0000 7f4bcfe00640 1 --2- 172.21.15.28:0/1903465571 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bd814b7e0 0x7f4bd814afe0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0008fd0 tx=0x7f4bc0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:34.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.244+0000 7f4bcf400640 1 -- 172.21.15.28:0/1903465571 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0037040 con 0x7f4bd814b7e0 2024-04-26T21:06:34.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.244+0000 7f4bcf400640 1 -- 172.21.15.28:0/1903465571 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc0005480 con 0x7f4bd814b7e0 2024-04-26T21:06:34.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.244+0000 7f4bcf400640 1 -- 172.21.15.28:0/1903465571 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc0007a30 con 0x7f4bd814b7e0 2024-04-26T21:06:34.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.244+0000 7f4bde400640 1 -- 172.21.15.28:0/1903465571 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd80aef70 con 0x7f4bd814b7e0 2024-04-26T21:06:34.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.244+0000 7f4bde400640 1 -- 172.21.15.28:0/1903465571 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd80af530 con 0x7f4bd814b7e0 2024-04-26T21:06:34.245 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.245+0000 7f4bde400640 1 -- 172.21.15.28:0/1903465571 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd806a360 con 0x7f4bd814b7e0 2024-04-26T21:06:34.246 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.246+0000 7f4bcf400640 1 -- 172.21.15.28:0/1903465571 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4bc0044020 con 0x7f4bd814b7e0 2024-04-26T21:06:34.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.247+0000 7f4bcf400640 1 --2- 172.21.15.28:0/1903465571 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4bb406c920 0x7f4bb406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:34.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.247+0000 7f4bcf400640 1 -- 172.21.15.28:0/1903465571 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f4bc00ab7b0 con 0x7f4bd814b7e0 2024-04-26T21:06:34.247 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.247+0000 7f4bd7400640 1 --2- 172.21.15.28:0/1903465571 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4bb406c920 0x7f4bb406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:34.248 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.248+0000 7f4bd7400640 1 --2- 172.21.15.28:0/1903465571 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4bb406c920 0x7f4bb406ede0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8001050 tx=0x7f4bc8001e30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:34.252 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.252+0000 7f4bcf400640 1 -- 172.21.15.28:0/1903465571 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4bc0079b90 con 0x7f4bd814b7e0 2024-04-26T21:06:34.459 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.459+0000 7f4bde400640 1 -- 172.21.15.28:0/1903465571 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4bd8156ca0 con 0x7f4bd814b7e0 2024-04-26T21:06:34.460 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.460+0000 7f4bcf400640 1 -- 172.21.15.28:0/1903465571 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) ==== 93+0+10 (secure 0 0 0) 0x7f4bc0079490 con 0x7f4bd814b7e0 2024-04-26T21:06:34.460 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2024-04-26T21:06:34.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.463+0000 7f4bde400640 1 -- 172.21.15.28:0/1903465571 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4bb406c920 msgr2=0x7f4bb406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:34.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.463+0000 7f4bde400640 1 --2- 172.21.15.28:0/1903465571 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4bb406c920 0x7f4bb406ede0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8001050 tx=0x7f4bc8001e30 comp rx=0 tx=0).stop 2024-04-26T21:06:34.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.463+0000 7f4bde400640 1 -- 172.21.15.28:0/1903465571 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bd814b7e0 msgr2=0x7f4bd814afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:34.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.463+0000 7f4bde400640 1 --2- 172.21.15.28:0/1903465571 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bd814b7e0 0x7f4bd814afe0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0008fd0 tx=0x7f4bc0001120 comp rx=0 tx=0).stop 2024-04-26T21:06:34.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.463+0000 7f4bde400640 1 -- 172.21.15.28:0/1903465571 shutdown_connections 2024-04-26T21:06:34.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.463+0000 7f4bde400640 1 --2- 172.21.15.28:0/1903465571 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4bb406c920 0x7f4bb406ede0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:34.464 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.463+0000 7f4bde400640 1 --2- 172.21.15.28:0/1903465571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4bd80ae720 0x7f4bd81e63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:34.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.463+0000 7f4bde400640 1 --2- 172.21.15.28:0/1903465571 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4bd814b7e0 0x7f4bd814afe0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:34.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.463+0000 7f4bde400640 1 --2- 172.21.15.28:0/1903465571 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4bbc004010 0x7f4bd80afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:34.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.463+0000 7f4bde400640 1 -- 172.21.15.28:0/1903465571 >> 172.21.15.28:0/1903465571 conn(0x7f4bd806c770 msgr2=0x7f4bd80b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:34.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.463+0000 7f4bde400640 1 -- 172.21.15.28:0/1903465571 shutdown_connections 2024-04-26T21:06:34.465 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.463+0000 7f4bde400640 1 -- 172.21.15.28:0/1903465571 wait complete. 2024-04-26T21:06:34.484 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T21:06:34.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.691+0000 7f62e9e00640 1 Processor -- start 2024-04-26T21:06:34.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.691+0000 7f62e9e00640 1 -- start start 2024-04-26T21:06:34.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.693+0000 7f62e9e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f62e4160650 0x7f62e4073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:34.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.693+0000 7f62e9e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62e40b3d90 con 0x7f62e415bb80 2024-04-26T21:06:34.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.693+0000 7f62e9e00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62e415fb70 con 0x7f62e40d1420 2024-04-26T21:06:34.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.693+0000 7f62e9e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f62e415f920 con 0x7f62e4160650 2024-04-26T21:06:34.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.693+0000 7f62e8a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f62e415bb80 0x7f62e40d0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45090/0 (socket says 172.21.15.28:45090) 2024-04-26T21:06:34.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.693+0000 7f62e8a00640 1 -- 172.21.15.28:0/3372463637 learned_addr learned my addr 172.21.15.28:0/3372463637 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:34.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.693+0000 7f62e9400640 1 --2- 172.21.15.28:0/3372463637 >> v2:172.21.15.28:3300/0 conn(0x7f62e4160650 0x7f62e4073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:34.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.694+0000 7f62e9400640 1 -- 172.21.15.28:0/3372463637 >> v1:172.21.15.103:6789/0 conn(0x7f62e40d1420 legacy=0x7f62e40cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:34.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.694+0000 7f62e9400640 1 -- 172.21.15.28:0/3372463637 >> v1:172.21.15.28:6789/0 conn(0x7f62e415bb80 legacy=0x7f62e40d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:34.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.694+0000 7f62e3400640 1 -- 172.21.15.28:0/3372463637 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1359391191 0 0) 0x7f62e40b3d90 con 0x7f62e415bb80 2024-04-26T21:06:34.694 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.694+0000 7f62e9400640 1 -- 172.21.15.28:0/3372463637 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e415fe50 con 0x7f62e4160650 2024-04-26T21:06:34.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.695+0000 7f62e9400640 1 --2- 172.21.15.28:0/3372463637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e4160650 0x7f62e4073bb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009d90 tx=0x7f62dc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a012e43bf059efd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:34.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.695+0000 7f62e3400640 1 -- 172.21.15.28:0/3372463637 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc007740 con 0x7f62e4160650 2024-04-26T21:06:34.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.695+0000 7f62e3400640 1 -- 172.21.15.28:0/3372463637 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62dc0078e0 con 0x7f62e4160650 2024-04-26T21:06:34.695 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.695+0000 7f62e3400640 1 -- 172.21.15.28:0/3372463637 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc007cb0 con 0x7f62e4160650 2024-04-26T21:06:34.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.696+0000 7f62e9e00640 1 -- 172.21.15.28:0/3372463637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e4160650 msgr2=0x7f62e4073bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:34.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.696+0000 7f62e9e00640 1 --2- 172.21.15.28:0/3372463637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e4160650 0x7f62e4073bb0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009d90 tx=0x7f62dc02a6a0 comp rx=0 tx=0).stop 2024-04-26T21:06:34.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.696+0000 7f62e9e00640 1 -- 172.21.15.28:0/3372463637 shutdown_connections 2024-04-26T21:06:34.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.696+0000 7f62e9e00640 1 --2- 172.21.15.28:0/3372463637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e4160650 0x7f62e4073bb0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:34.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.696+0000 7f62e9e00640 1 -- 172.21.15.28:0/3372463637 >> 172.21.15.28:0/3372463637 conn(0x7f62e406c530 msgr2=0x7f62e406c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:34.696 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.696+0000 7f62e9e00640 1 -- 172.21.15.28:0/3372463637 shutdown_connections 2024-04-26T21:06:34.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.696+0000 7f62e9e00640 1 -- 172.21.15.28:0/3372463637 wait complete. 2024-04-26T21:06:34.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.698+0000 7f62e9e00640 1 Processor -- start 2024-04-26T21:06:34.698 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.698+0000 7f62e9e00640 1 -- start start 2024-04-26T21:06:34.700 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.699+0000 7f62e9e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62e40d1420 0x7f62e41466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:34.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.699+0000 7f62e9e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f62e415bb80 0x7f62e415c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:34.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.699+0000 7f62e9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e4160650 0x7f62e415e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:34.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.699+0000 7f62e9e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f62e415bf60 con 0x7f62e4160650 2024-04-26T21:06:34.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.699+0000 7f62e9e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62e40b3a20 con 0x7f62e40d1420 2024-04-26T21:06:34.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.699+0000 7f62e9e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f62e415c0e0 con 0x7f62e415bb80 2024-04-26T21:06:34.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.699+0000 7f62e9400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e4160650 0x7f62e415e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:34.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.699+0000 7f62e9400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e4160650 0x7f62e415e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37406/0 (socket says 172.21.15.28:37406) 2024-04-26T21:06:34.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.699+0000 7f62e9400640 1 -- 172.21.15.28:0/3157585573 learned_addr learned my addr 172.21.15.28:0/3157585573 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:34.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.700+0000 7f62e3e00640 1 --2- 172.21.15.28:0/3157585573 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f62e415bb80 0x7f62e415c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:34.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.700+0000 7f62e8a00640 1 --2- 172.21.15.28:0/3157585573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62e40d1420 0x7f62e41466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:34.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.700+0000 7f62e9400640 1 -- 172.21.15.28:0/3157585573 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f62e415bb80 msgr2=0x7f62e415c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:34.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.700+0000 7f62e9400640 1 --2- 172.21.15.28:0/3157585573 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f62e415bb80 0x7f62e415c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:34.701 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.700+0000 7f62e9400640 1 -- 172.21.15.28:0/3157585573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62e40d1420 msgr2=0x7f62e41466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.700+0000 7f62e9400640 1 --2- 172.21.15.28:0/3157585573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62e40d1420 0x7f62e41466e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.700+0000 7f62e9400640 1 -- 172.21.15.28:0/3157585573 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62e4159f20 con 0x7f62e4160650 2024-04-26T21:06:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.700+0000 7f62e3e00640 1 --2- 172.21.15.28:0/3157585573 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f62e415bb80 0x7f62e415c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.700+0000 7f62e9400640 1 --2- 172.21.15.28:0/3157585573 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e4160650 0x7f62e415e820 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009eb0 tx=0x7f62dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.700+0000 7f62e0c00640 1 -- 172.21.15.28:0/3157585573 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc040070 con 0x7f62e4160650 2024-04-26T21:06:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.701+0000 7f62e0c00640 1 -- 172.21.15.28:0/3157585573 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62dc031c00 con 0x7f62e4160650 2024-04-26T21:06:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.701+0000 7f62e0c00640 1 -- 172.21.15.28:0/3157585573 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc045460 con 0x7f62e4160650 2024-04-26T21:06:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.701+0000 7f62e9e00640 1 -- 172.21.15.28:0/3157585573 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62e415a200 con 0x7f62e4160650 2024-04-26T21:06:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.701+0000 7f62e9e00640 1 -- 172.21.15.28:0/3157585573 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62e4146c20 con 0x7f62e4160650 2024-04-26T21:06:34.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.702+0000 7f62e9e00640 1 -- 172.21.15.28:0/3157585573 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62e4154dc0 con 0x7f62e4160650 2024-04-26T21:06:34.703 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.703+0000 7f62e0c00640 1 -- 172.21.15.28:0/3157585573 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f62dc039040 con 0x7f62e4160650 2024-04-26T21:06:34.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.704+0000 7f62e0c00640 1 --2- 172.21.15.28:0/3157585573 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f62c406c920 0x7f62c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:34.704 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.705+0000 7f62e0c00640 1 -- 172.21.15.28:0/3157585573 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f62dc0ad220 con 0x7f62e4160650 2024-04-26T21:06:34.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.705+0000 7f62e8a00640 1 --2- 172.21.15.28:0/3157585573 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f62c406c920 0x7f62c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:34.705 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.705+0000 7f62e8a00640 1 --2- 172.21.15.28:0/3157585573 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f62c406c920 0x7f62c406ede0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f62d8007fd0 tx=0x7f62d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:34.709 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.709+0000 7f62e0c00640 1 -- 172.21.15.28:0/3157585573 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f62dc07b600 con 0x7f62e4160650 2024-04-26T21:06:34.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.914+0000 7f62e9e00640 1 -- 172.21.15.28:0/3157585573 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f62e4158810 con 0x7f62e4160650 2024-04-26T21:06:34.914 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.914+0000 7f62e0c00640 1 -- 172.21.15.28:0/3157585573 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) ==== 104+0+11 (secure 0 0 0) 0x7f62dc07af00 con 0x7f62e4160650 2024-04-26T21:06:34.915 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2024-04-26T21:06:34.917 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.917+0000 7f62e9e00640 1 -- 172.21.15.28:0/3157585573 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f62c406c920 msgr2=0x7f62c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:34.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.917+0000 7f62e9e00640 1 --2- 172.21.15.28:0/3157585573 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f62c406c920 0x7f62c406ede0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f62d8007fd0 tx=0x7f62d8009040 comp rx=0 tx=0).stop 2024-04-26T21:06:34.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.917+0000 7f62e9e00640 1 -- 172.21.15.28:0/3157585573 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e4160650 msgr2=0x7f62e415e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:34.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.917+0000 7f62e9e00640 1 --2- 172.21.15.28:0/3157585573 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e4160650 0x7f62e415e820 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f62dc009eb0 tx=0x7f62dc004a40 comp rx=0 tx=0).stop 2024-04-26T21:06:34.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.917+0000 7f62e9e00640 1 -- 172.21.15.28:0/3157585573 shutdown_connections 2024-04-26T21:06:34.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.917+0000 7f62e9e00640 1 --2- 172.21.15.28:0/3157585573 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f62c406c920 0x7f62c406ede0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:34.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.917+0000 7f62e9e00640 1 --2- 172.21.15.28:0/3157585573 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e4160650 0x7f62e415e820 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:34.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.917+0000 7f62e9e00640 1 --2- 172.21.15.28:0/3157585573 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f62e415bb80 0x7f62e415c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:34.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.917+0000 7f62e9e00640 1 --2- 172.21.15.28:0/3157585573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62e40d1420 0x7f62e41466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:34.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.917+0000 7f62e9e00640 1 -- 172.21.15.28:0/3157585573 >> 172.21.15.28:0/3157585573 conn(0x7f62e406c530 msgr2=0x7f62e4158a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:34.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.917+0000 7f62e9e00640 1 -- 172.21.15.28:0/3157585573 shutdown_connections 2024-04-26T21:06:34.918 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:34.917+0000 7f62e9e00640 1 -- 172.21.15.28:0/3157585573 wait complete. 2024-04-26T21:06:34.938 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T21:06:35.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.153+0000 7fb80ce00640 1 Processor -- start 2024-04-26T21:06:35.153 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.153+0000 7fb80ce00640 1 -- start start 2024-04-26T21:06:35.157 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.154+0000 7fb80ce00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb8080cf6b0 0x7fb8080cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:35.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.154+0000 7fb80ce00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8080d04c0 con 0x7fb80815f0d0 2024-04-26T21:06:35.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.154+0000 7fb80ce00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8080d2ac0 con 0x7fb8080d0050 2024-04-26T21:06:35.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.154+0000 7fb80ce00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fb8080d2da0 con 0x7fb8080cf6b0 2024-04-26T21:06:35.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.155+0000 7fb807400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb80815f0d0 0x7fb80806cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45106/0 (socket says 172.21.15.28:45106) 2024-04-26T21:06:35.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.155+0000 7fb807400640 1 -- 172.21.15.28:0/846193619 learned_addr learned my addr 172.21.15.28:0/846193619 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:35.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.155+0000 7fb806a00640 1 --2- 172.21.15.28:0/846193619 >> v2:172.21.15.28:3300/0 conn(0x7fb8080cf6b0 0x7fb8080cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:35.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.155+0000 7fb806000640 1 -- 172.21.15.28:0/846193619 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1099984177 0 0) 0x7fb8080d04c0 con 0x7fb80815f0d0 2024-04-26T21:06:35.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.155+0000 7fb806000640 1 -- 172.21.15.28:0/846193619 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7ec003520 con 0x7fb80815f0d0 2024-04-26T21:06:35.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.155+0000 7fb806000640 1 -- 172.21.15.28:0/846193619 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1024413636 0 0) 0x7fb7ec003520 con 0x7fb80815f0d0 2024-04-26T21:06:35.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.156+0000 7fb806000640 1 -- 172.21.15.28:0/846193619 >> v2:172.21.15.28:3300/0 conn(0x7fb8080cf6b0 msgr2=0x7fb8080cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:35.158 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.156+0000 7fb806000640 1 --2- 172.21.15.28:0/846193619 >> v2:172.21.15.28:3300/0 conn(0x7fb8080cf6b0 0x7fb8080cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.156+0000 7fb806000640 1 -- 172.21.15.28:0/846193619 >> v1:172.21.15.103:6789/0 conn(0x7fb8080d0050 legacy=0x7fb8080d3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.156+0000 7fb806000640 1 -- 172.21.15.28:0/846193619 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8080d3080 con 0x7fb80815f0d0 2024-04-26T21:06:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.156+0000 7fb806000640 1 -- 172.21.15.28:0/846193619 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fb7f8001670 con 0x7fb80815f0d0 2024-04-26T21:06:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.156+0000 7fb806a00640 1 --2- 172.21.15.28:0/846193619 >> v2:172.21.15.28:3300/0 conn(0x7fb8080cf6b0 0x7fb8080cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T21:06:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.156+0000 7fb806000640 1 -- 172.21.15.28:0/846193619 >> v1:172.21.15.28:6789/0 conn(0x7fb80815f0d0 legacy=0x7fb80806cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.156+0000 7fb806000640 1 --2- 172.21.15.28:0/846193619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7ec0039b0 0x7fb7ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.156+0000 7fb806000640 1 --2- 172.21.15.28:0/846193619 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7ec004800 0x7fb7ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.156+0000 7fb806000640 1 --2- 172.21.15.28:0/846193619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7ec0051c0 0x7fb7ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.156+0000 7fb806000640 1 -- 172.21.15.28:0/846193619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb8080d2da0 con 0x7fb7ec0051c0 2024-04-26T21:06:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.156+0000 7fb806000640 1 -- 172.21.15.28:0/846193619 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ec00c060 con 0x7fb7ec0039b0 2024-04-26T21:06:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.156+0000 7fb806000640 1 -- 172.21.15.28:0/846193619 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb7ec00c260 con 0x7fb7ec004800 2024-04-26T21:06:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.156+0000 7fb807e00640 1 --2- 172.21.15.28:0/846193619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7ec0051c0 0x7fb7ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.156+0000 7fb807400640 1 --2- 172.21.15.28:0/846193619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7ec0039b0 0x7fb7ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:35.159 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.156+0000 7fb806a00640 1 --2- 172.21.15.28:0/846193619 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7ec004800 0x7fb7ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:35.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.157+0000 7fb807e00640 1 -- 172.21.15.28:0/846193619 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7ec004800 msgr2=0x7fb7ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:35.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.157+0000 7fb807e00640 1 --2- 172.21.15.28:0/846193619 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7ec004800 0x7fb7ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.157+0000 7fb807e00640 1 -- 172.21.15.28:0/846193619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7ec0039b0 msgr2=0x7fb7ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:35.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.157+0000 7fb807e00640 1 --2- 172.21.15.28:0/846193619 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb7ec0039b0 0x7fb7ec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.157+0000 7fb807e00640 1 -- 172.21.15.28:0/846193619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ec003520 con 0x7fb7ec0051c0 2024-04-26T21:06:35.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.157+0000 7fb806a00640 1 --2- 172.21.15.28:0/846193619 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7ec004800 0x7fb7ec004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:35.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.157+0000 7fb807e00640 1 -- 172.21.15.28:0/846193619 reap_dead start 2024-04-26T21:06:35.160 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.158+0000 7fb807e00640 1 --2- 172.21.15.28:0/846193619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7ec0051c0 0x7fb7ec00b9c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb800003db0 tx=0x7fb7ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b924ee426a9bb1d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:35.162 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.158+0000 7fb806000640 1 -- 172.21.15.28:0/846193619 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800003980 con 0x7fb7ec0051c0 2024-04-26T21:06:35.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.158+0000 7fb806000640 1 -- 172.21.15.28:0/846193619 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb800007240 con 0x7fb7ec0051c0 2024-04-26T21:06:35.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.158+0000 7fb806000640 1 -- 172.21.15.28:0/846193619 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800007630 con 0x7fb7ec0051c0 2024-04-26T21:06:35.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.158+0000 7fb80ce00640 1 -- 172.21.15.28:0/846193619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7ec0051c0 msgr2=0x7fb7ec00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:35.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.158+0000 7fb80ce00640 1 --2- 172.21.15.28:0/846193619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7ec0051c0 0x7fb7ec00b9c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb800003db0 tx=0x7fb7ec005830 comp rx=0 tx=0).stop 2024-04-26T21:06:35.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.159+0000 7fb80ce00640 1 -- 172.21.15.28:0/846193619 shutdown_connections 2024-04-26T21:06:35.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.159+0000 7fb80ce00640 1 --2- 172.21.15.28:0/846193619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7ec0051c0 0x7fb7ec00b9c0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.159+0000 7fb80ce00640 1 -- 172.21.15.28:0/846193619 >> 172.21.15.28:0/846193619 conn(0x7fb8080d4e50 msgr2=0x7fb8080d5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:35.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.159+0000 7fb80ce00640 1 -- 172.21.15.28:0/846193619 shutdown_connections 2024-04-26T21:06:35.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.159+0000 7fb80ce00640 1 -- 172.21.15.28:0/846193619 wait complete. 2024-04-26T21:06:35.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.160+0000 7fb80ce00640 1 Processor -- start 2024-04-26T21:06:35.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.161+0000 7fb80ce00640 1 -- start start 2024-04-26T21:06:35.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.162+0000 7fb80ce00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7ec0051c0 0x7fb8081f8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:35.163 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.162+0000 7fb80ce00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb80814a960 0x7fb8080ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:35.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.162+0000 7fb80ce00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8080cbf20 0x7fb8080cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:35.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.162+0000 7fb80ce00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fb80815f4b0 con 0x7fb8080cbf20 2024-04-26T21:06:35.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.162+0000 7fb80ce00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb80806d540 con 0x7fb80814a960 2024-04-26T21:06:35.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.162+0000 7fb80ce00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fb8080ce9a0 con 0x7fb7ec0051c0 2024-04-26T21:06:35.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.162+0000 7fb807e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8080cbf20 0x7fb8080cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:35.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.162+0000 7fb807e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8080cbf20 0x7fb8080cc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37438/0 (socket says 172.21.15.28:37438) 2024-04-26T21:06:35.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.162+0000 7fb807e00640 1 -- 172.21.15.28:0/3691367859 learned_addr learned my addr 172.21.15.28:0/3691367859 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:35.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.162+0000 7fb806a00640 1 --2- 172.21.15.28:0/3691367859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb80814a960 0x7fb8080ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:35.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.162+0000 7fb807400640 1 --2- 172.21.15.28:0/3691367859 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7ec0051c0 0x7fb8081f8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:35.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.162+0000 7fb807e00640 1 -- 172.21.15.28:0/3691367859 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7ec0051c0 msgr2=0x7fb8081f8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:35.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.162+0000 7fb807e00640 1 --2- 172.21.15.28:0/3691367859 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7ec0051c0 0x7fb8081f8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.162+0000 7fb807e00640 1 -- 172.21.15.28:0/3691367859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb80814a960 msgr2=0x7fb8080ce030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:35.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.162+0000 7fb807e00640 1 --2- 172.21.15.28:0/3691367859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb80814a960 0x7fb8080ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.162+0000 7fb807e00640 1 -- 172.21.15.28:0/3691367859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8080cec80 con 0x7fb8080cbf20 2024-04-26T21:06:35.164 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.163+0000 7fb807e00640 1 --2- 172.21.15.28:0/3691367859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8080cbf20 0x7fb8080cc3a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb80002a4b0 tx=0x7fb8000036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:35.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.163+0000 7fb7ff400640 1 -- 172.21.15.28:0/3691367859 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800007a70 con 0x7fb8080cbf20 2024-04-26T21:06:35.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.163+0000 7fb7ff400640 1 -- 172.21.15.28:0/3691367859 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb800042070 con 0x7fb8080cbf20 2024-04-26T21:06:35.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.163+0000 7fb7ff400640 1 -- 172.21.15.28:0/3691367859 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800006a10 con 0x7fb8080cbf20 2024-04-26T21:06:35.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.163+0000 7fb80ce00640 1 -- 172.21.15.28:0/3691367859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8080cef60 con 0x7fb8080cbf20 2024-04-26T21:06:35.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.163+0000 7fb80ce00640 1 -- 172.21.15.28:0/3691367859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb80814adc0 con 0x7fb8080cbf20 2024-04-26T21:06:35.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.164+0000 7fb80ce00640 1 -- 172.21.15.28:0/3691367859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8080cf480 con 0x7fb8080cbf20 2024-04-26T21:06:35.166 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.165+0000 7fb7ff400640 1 -- 172.21.15.28:0/3691367859 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb800006bb0 con 0x7fb8080cbf20 2024-04-26T21:06:35.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.167+0000 7fb7ff400640 1 --2- 172.21.15.28:0/3691367859 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb7e406c920 0x7fb7e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:35.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.167+0000 7fb7ff400640 1 -- 172.21.15.28:0/3691367859 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fb8000abb70 con 0x7fb8080cbf20 2024-04-26T21:06:35.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.167+0000 7fb807400640 1 --2- 172.21.15.28:0/3691367859 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb7e406c920 0x7fb7e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:35.167 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.168+0000 7fb807400640 1 --2- 172.21.15.28:0/3691367859 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb7e406c920 0x7fb7e406ede0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8001eb0 tx=0x7fb7f8000f60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:35.171 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.171+0000 7fb7ff400640 1 -- 172.21.15.28:0/3691367859 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb8080cf480 con 0x7fb8080cbf20 2024-04-26T21:06:35.376 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.376+0000 7fb80ce00640 1 -- 172.21.15.28:0/3691367859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb80806a2c0 con 0x7fb8080cbf20 2024-04-26T21:06:35.377 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.377+0000 7fb7ff400640 1 -- 172.21.15.28:0/3691367859 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) ==== 100+0+11 (secure 0 0 0) 0x7fb800079ec0 con 0x7fb8080cbf20 2024-04-26T21:06:35.377 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2024-04-26T21:06:35.379 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.380+0000 7fb80ce00640 1 -- 172.21.15.28:0/3691367859 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb7e406c920 msgr2=0x7fb7e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:35.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.380+0000 7fb80ce00640 1 --2- 172.21.15.28:0/3691367859 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb7e406c920 0x7fb7e406ede0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8001eb0 tx=0x7fb7f8000f60 comp rx=0 tx=0).stop 2024-04-26T21:06:35.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.380+0000 7fb80ce00640 1 -- 172.21.15.28:0/3691367859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8080cbf20 msgr2=0x7fb8080cc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:35.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.380+0000 7fb80ce00640 1 --2- 172.21.15.28:0/3691367859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8080cbf20 0x7fb8080cc3a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb80002a4b0 tx=0x7fb8000036f0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.380 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.380+0000 7fb80ce00640 1 -- 172.21.15.28:0/3691367859 shutdown_connections 2024-04-26T21:06:35.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.380+0000 7fb80ce00640 1 --2- 172.21.15.28:0/3691367859 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fb7e406c920 0x7fb7e406ede0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.380+0000 7fb80ce00640 1 --2- 172.21.15.28:0/3691367859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb8080cbf20 0x7fb8080cc3a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.380+0000 7fb80ce00640 1 --2- 172.21.15.28:0/3691367859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb80814a960 0x7fb8080ce030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.380+0000 7fb80ce00640 1 --2- 172.21.15.28:0/3691367859 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fb7ec0051c0 0x7fb8081f8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.380+0000 7fb80ce00640 1 -- 172.21.15.28:0/3691367859 >> 172.21.15.28:0/3691367859 conn(0x7fb8080d4e50 msgr2=0x7fb80815f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:35.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.380+0000 7fb80ce00640 1 -- 172.21.15.28:0/3691367859 shutdown_connections 2024-04-26T21:06:35.381 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.380+0000 7fb80ce00640 1 -- 172.21.15.28:0/3691367859 wait complete. 2024-04-26T21:06:35.400 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:06:35.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.612+0000 7fe5d8800640 1 Processor -- start 2024-04-26T21:06:35.613 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.612+0000 7fe5d8800640 1 -- start start 2024-04-26T21:06:35.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.614+0000 7fe5d8800640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe5d015dcc0 0x7fe5d0167f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:35.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.614+0000 7fe5d8800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5d00d0a30 con 0x7fe5d0146980 2024-04-26T21:06:35.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.614+0000 7fe5d8800640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5d015e230 con 0x7fe5d00b6810 2024-04-26T21:06:35.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.614+0000 7fe5d8800640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fe5d015e510 con 0x7fe5d015dcc0 2024-04-26T21:06:35.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.614+0000 7fe5d5e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe5d0146980 0x7fe5d00b6100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45116/0 (socket says 172.21.15.28:45116) 2024-04-26T21:06:35.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.615+0000 7fe5d5e00640 1 -- 172.21.15.28:0/1839986681 learned_addr learned my addr 172.21.15.28:0/1839986681 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:35.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.615+0000 7fe5d4a00640 1 -- 172.21.15.28:0/1839986681 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2956516057 0 0) 0x7fe5c4001110 con 0x7fe5d0146980 2024-04-26T21:06:35.615 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.615+0000 7fe5d4a00640 1 -- 172.21.15.28:0/1839986681 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe5b8003520 con 0x7fe5d0146980 2024-04-26T21:06:35.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.615+0000 7fe5d6800640 1 --2- 172.21.15.28:0/1839986681 >> v2:172.21.15.103:3300/0 conn(0x7fe5d015dcc0 0x7fe5d0167f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:35.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.616+0000 7fe5d4a00640 1 -- 172.21.15.28:0/1839986681 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1365907491 0 0) 0x7fe5b8003520 con 0x7fe5d0146980 2024-04-26T21:06:35.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.616+0000 7fe5d4a00640 1 -- 172.21.15.28:0/1839986681 >> v2:172.21.15.103:3300/0 conn(0x7fe5d015dcc0 msgr2=0x7fe5d0167f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:35.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.616+0000 7fe5d4a00640 1 --2- 172.21.15.28:0/1839986681 >> v2:172.21.15.103:3300/0 conn(0x7fe5d015dcc0 0x7fe5d0167f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.616+0000 7fe5d4a00640 1 -- 172.21.15.28:0/1839986681 >> v1:172.21.15.103:6789/0 conn(0x7fe5d00b6810 legacy=0x7fe5d015d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:35.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.616+0000 7fe5d4a00640 1 -- 172.21.15.28:0/1839986681 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d0073600 con 0x7fe5d0146980 2024-04-26T21:06:35.617 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.616+0000 7fe5d4a00640 1 -- 172.21.15.28:0/1839986681 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fe5c4001840 con 0x7fe5d0146980 2024-04-26T21:06:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.616+0000 7fe5d6800640 1 --2- 172.21.15.28:0/1839986681 >> v2:172.21.15.103:3300/0 conn(0x7fe5d015dcc0 0x7fe5d0167f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.616+0000 7fe5d4a00640 1 -- 172.21.15.28:0/1839986681 >> v1:172.21.15.28:6789/0 conn(0x7fe5d0146980 legacy=0x7fe5d00b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.616+0000 7fe5d4a00640 1 --2- 172.21.15.28:0/1839986681 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b8003a00 0x7fe5b8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.616+0000 7fe5d4a00640 1 --2- 172.21.15.28:0/1839986681 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe5b8004800 0x7fe5b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.616+0000 7fe5d4a00640 1 --2- 172.21.15.28:0/1839986681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5b80053c0 0x7fe5b800ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.616+0000 7fe5d4a00640 1 -- 172.21.15.28:0/1839986681 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d015e510 con 0x7fe5b8003a00 2024-04-26T21:06:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.616+0000 7fe5d4a00640 1 -- 172.21.15.28:0/1839986681 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b800c2e0 con 0x7fe5b80053c0 2024-04-26T21:06:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.616+0000 7fe5d4a00640 1 -- 172.21.15.28:0/1839986681 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe5b800c4e0 con 0x7fe5b8004800 2024-04-26T21:06:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.616+0000 7fe5d5e00640 1 --2- 172.21.15.28:0/1839986681 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b8003a00 0x7fe5b8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.617+0000 7fe5d5e00640 1 -- 172.21.15.28:0/1839986681 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe5b8004800 msgr2=0x7fe5b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.617+0000 7fe5d5e00640 1 --2- 172.21.15.28:0/1839986681 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe5b8004800 0x7fe5b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.617+0000 7fe5d5e00640 1 -- 172.21.15.28:0/1839986681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5b80053c0 msgr2=0x7fe5b800ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.617+0000 7fe5d5e00640 1 --2- 172.21.15.28:0/1839986681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5b80053c0 0x7fe5b800ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.617+0000 7fe5d5e00640 1 -- 172.21.15.28:0/1839986681 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b8003520 con 0x7fe5b8003a00 2024-04-26T21:06:35.618 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.617+0000 7fe5d6800640 1 -- 172.21.15.28:0/1839986681 reap_dead start 2024-04-26T21:06:35.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.618+0000 7fe5d5e00640 1 --2- 172.21.15.28:0/1839986681 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b8003a00 0x7fe5b8003e20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4007fd0 tx=0x7fe5b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c08868a7764229 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:35.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.618+0000 7fe5d4a00640 1 -- 172.21.15.28:0/1839986681 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4007e30 con 0x7fe5b8003a00 2024-04-26T21:06:35.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.618+0000 7fe5d4a00640 1 -- 172.21.15.28:0/1839986681 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c4005af0 con 0x7fe5b8003a00 2024-04-26T21:06:35.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.618+0000 7fe5d4a00640 1 -- 172.21.15.28:0/1839986681 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4007070 con 0x7fe5b8003a00 2024-04-26T21:06:35.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.619+0000 7fe5d8800640 1 -- 172.21.15.28:0/1839986681 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b8003a00 msgr2=0x7fe5b8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:35.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.619+0000 7fe5d8800640 1 --2- 172.21.15.28:0/1839986681 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b8003a00 0x7fe5b8003e20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4007fd0 tx=0x7fe5b8005830 comp rx=0 tx=0).stop 2024-04-26T21:06:35.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.619+0000 7fe5d8800640 1 -- 172.21.15.28:0/1839986681 shutdown_connections 2024-04-26T21:06:35.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.619+0000 7fe5d8800640 1 --2- 172.21.15.28:0/1839986681 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b8003a00 0x7fe5b8003e20 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.619+0000 7fe5d8800640 1 -- 172.21.15.28:0/1839986681 >> 172.21.15.28:0/1839986681 conn(0x7fe5d006e220 msgr2=0x7fe5d006e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:35.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.619+0000 7fe5d8800640 1 -- 172.21.15.28:0/1839986681 shutdown_connections 2024-04-26T21:06:35.620 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.619+0000 7fe5d8800640 1 -- 172.21.15.28:0/1839986681 wait complete. 2024-04-26T21:06:35.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.621+0000 7fe5d8800640 1 Processor -- start 2024-04-26T21:06:35.621 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.621+0000 7fe5d8800640 1 -- start start 2024-04-26T21:06:35.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.622+0000 7fe5d8800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5b8003a00 0x7fe5d0139590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:35.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.622+0000 7fe5d8800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe5d013b5b0 0x7fe5d01458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:35.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.622+0000 7fe5d8800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5d013caf0 0x7fe5d01580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:35.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.622+0000 7fe5d8800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d00740e0 con 0x7fe5d013caf0 2024-04-26T21:06:35.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.622+0000 7fe5d8800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe5d0073bc0 con 0x7fe5b8003a00 2024-04-26T21:06:35.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.622+0000 7fe5d8800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe5d0074260 con 0x7fe5d013b5b0 2024-04-26T21:06:35.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.622+0000 7fe5d6800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5d013caf0 0x7fe5d01580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:35.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.622+0000 7fe5d6800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5d013caf0 0x7fe5d01580d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37452/0 (socket says 172.21.15.28:37452) 2024-04-26T21:06:35.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.622+0000 7fe5d6800640 1 -- 172.21.15.28:0/1283110734 learned_addr learned my addr 172.21.15.28:0/1283110734 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:35.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.622+0000 7fe5d5400640 1 --2- 172.21.15.28:0/1283110734 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe5d013b5b0 0x7fe5d01458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:35.623 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.622+0000 7fe5d5e00640 1 --2- 172.21.15.28:0/1283110734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5b8003a00 0x7fe5d0139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:35.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.622+0000 7fe5d6800640 1 -- 172.21.15.28:0/1283110734 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe5d013b5b0 msgr2=0x7fe5d01458b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:35.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.622+0000 7fe5d6800640 1 --2- 172.21.15.28:0/1283110734 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe5d013b5b0 0x7fe5d01458b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.622+0000 7fe5d6800640 1 -- 172.21.15.28:0/1283110734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5b8003a00 msgr2=0x7fe5d0139590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:35.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.622+0000 7fe5d6800640 1 --2- 172.21.15.28:0/1283110734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5b8003a00 0x7fe5d0139590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.623+0000 7fe5d6800640 1 -- 172.21.15.28:0/1283110734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5d013bd30 con 0x7fe5d013caf0 2024-04-26T21:06:35.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.623+0000 7fe5d5400640 1 --2- 172.21.15.28:0/1283110734 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe5d013b5b0 0x7fe5d01458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:35.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.623+0000 7fe5d5e00640 1 --2- 172.21.15.28:0/1283110734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5b8003a00 0x7fe5d0139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:35.624 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.623+0000 7fe5d6800640 1 --2- 172.21.15.28:0/1283110734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5d013caf0 0x7fe5d01580d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00d0410 tx=0x7fe5cc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:35.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.623+0000 7fe5ca000640 1 -- 172.21.15.28:0/1283110734 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc01a070 con 0x7fe5d013caf0 2024-04-26T21:06:35.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.623+0000 7fe5ca000640 1 -- 172.21.15.28:0/1283110734 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5cc015440 con 0x7fe5d013caf0 2024-04-26T21:06:35.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.623+0000 7fe5d8800640 1 -- 172.21.15.28:0/1283110734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5d013c010 con 0x7fe5d013caf0 2024-04-26T21:06:35.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.623+0000 7fe5ca000640 1 -- 172.21.15.28:0/1283110734 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5cc0157f0 con 0x7fe5d013caf0 2024-04-26T21:06:35.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.623+0000 7fe5d8800640 1 -- 172.21.15.28:0/1283110734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5d013ba30 con 0x7fe5d013caf0 2024-04-26T21:06:35.625 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.625+0000 7fe5d8800640 1 -- 172.21.15.28:0/1283110734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5d013c400 con 0x7fe5d013caf0 2024-04-26T21:06:35.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.626+0000 7fe5ca000640 1 -- 172.21.15.28:0/1283110734 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe5cc01d070 con 0x7fe5d013caf0 2024-04-26T21:06:35.626 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.626+0000 7fe5ca000640 1 --2- 172.21.15.28:0/1283110734 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe5b406c8f0 0x7fe5b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:35.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.627+0000 7fe5ca000640 1 -- 172.21.15.28:0/1283110734 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fe5cc08bfa0 con 0x7fe5d013caf0 2024-04-26T21:06:35.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.627+0000 7fe5d5e00640 1 --2- 172.21.15.28:0/1283110734 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe5b406c8f0 0x7fe5b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:35.627 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.627+0000 7fe5d5e00640 1 --2- 172.21.15.28:0/1283110734 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe5b406c8f0 0x7fe5b406edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4033000 tx=0x7fe5c402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:35.631 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.631+0000 7fe5ca000640 1 -- 172.21.15.28:0/1283110734 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe5cc059ca0 con 0x7fe5d013caf0 2024-04-26T21:06:35.880 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.879+0000 7fe5d8800640 1 -- 172.21.15.28:0/1283110734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5d006a400 con 0x7fe5d013caf0 2024-04-26T21:06:35.881 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.881+0000 7fe5ca000640 1 -- 172.21.15.28:0/1283110734 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5184 (secure 0 0 0) 0x7fe5cc05dfa0 con 0x7fe5d013caf0 2024-04-26T21:06:35.881 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T21:06:35.881 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/356870645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":356870645},{"type":"v1","addr":"172.21.15.28:6835","nonce":356870645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4829,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2957511478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":2957511478},{"type":"v1","addr":"172.21.15.103:6835","nonce":2957511478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/2060632176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2060632176},{"type":"v1","addr":"172.21.15.103:6834","nonce":2060632176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:58:43.534341+0000","modified":"2024-04-26T20:58:46.581816+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5571},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6837/2997047727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":2997047727},{"type":"v1","addr":"172.21.15.28:6837","nonce":2997047727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5571,"qdb_cluster":[5571]},"id":3}]} 2024-04-26T21:06:35.881 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 26 2024-04-26T21:06:35.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.883+0000 7fe5d8800640 1 -- 172.21.15.28:0/1283110734 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe5b406c8f0 msgr2=0x7fe5b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:35.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.883+0000 7fe5d8800640 1 --2- 172.21.15.28:0/1283110734 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe5b406c8f0 0x7fe5b406edb0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4033000 tx=0x7fe5c402e000 comp rx=0 tx=0).stop 2024-04-26T21:06:35.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.883+0000 7fe5d8800640 1 -- 172.21.15.28:0/1283110734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5d013caf0 msgr2=0x7fe5d01580d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:35.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.883+0000 7fe5d8800640 1 --2- 172.21.15.28:0/1283110734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5d013caf0 0x7fe5d01580d0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00d0410 tx=0x7fe5cc00a710 comp rx=0 tx=0).stop 2024-04-26T21:06:35.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.884+0000 7fe5d8800640 1 -- 172.21.15.28:0/1283110734 shutdown_connections 2024-04-26T21:06:35.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.884+0000 7fe5d8800640 1 --2- 172.21.15.28:0/1283110734 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe5b406c8f0 0x7fe5b406edb0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.884+0000 7fe5d8800640 1 --2- 172.21.15.28:0/1283110734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5d013caf0 0x7fe5d01580d0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.884+0000 7fe5d8800640 1 --2- 172.21.15.28:0/1283110734 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe5d013b5b0 0x7fe5d01458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.884+0000 7fe5d8800640 1 --2- 172.21.15.28:0/1283110734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe5b8003a00 0x7fe5d0139590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:35.884 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.884+0000 7fe5d8800640 1 -- 172.21.15.28:0/1283110734 >> 172.21.15.28:0/1283110734 conn(0x7fe5d006e220 msgr2=0x7fe5d00b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:35.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.884+0000 7fe5d8800640 1 -- 172.21.15.28:0/1283110734 shutdown_connections 2024-04-26T21:06:35.885 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:35.884+0000 7fe5d8800640 1 -- 172.21.15.28:0/1283110734 wait complete. 2024-04-26T21:06:35.904 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:06:36.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.124+0000 7f2a4f600640 1 Processor -- start 2024-04-26T21:06:36.125 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.125+0000 7f2a4f600640 1 -- start start 2024-04-26T21:06:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.126+0000 7f2a4f600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2a4806da70 0x7f2a48160b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.127+0000 7f2a4f600640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a4815d470 con 0x7f2a480d1730 2024-04-26T21:06:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.127+0000 7f2a4f600640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a480b1650 con 0x7f2a4815d090 2024-04-26T21:06:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.127+0000 7f2a4f600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f2a480b1400 con 0x7f2a4806da70 2024-04-26T21:06:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.127+0000 7f2a4d600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2a4806da70 0x7f2a48160b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.127+0000 7f2a4d600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2a4806da70 0x7f2a48160b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37464/0 (socket says 172.21.15.28:37464) 2024-04-26T21:06:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.127+0000 7f2a4d600640 1 -- 172.21.15.28:0/1020372903 learned_addr learned my addr 172.21.15.28:0/1020372903 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:36.127 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.127+0000 7f2a47e00640 1 --1- 172.21.15.28:0/1020372903 >> v1:172.21.15.28:6789/0 conn(0x7f2a480d1730 0x7f2a4806d360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45130/0 (socket says 172.21.15.28:45130) 2024-04-26T21:06:36.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.127+0000 7f2a4d600640 1 -- 172.21.15.28:0/1020372903 >> v1:172.21.15.103:6789/0 conn(0x7f2a4815d090 legacy=0x7f2a480d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:36.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.127+0000 7f2a4d600640 1 -- 172.21.15.28:0/1020372903 >> v1:172.21.15.28:6789/0 conn(0x7f2a480d1730 legacy=0x7f2a4806d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:36.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.128+0000 7f2a4d600640 1 -- 172.21.15.28:0/1020372903 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a480aef30 con 0x7f2a4806da70 2024-04-26T21:06:36.128 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.129+0000 7f2a4d600640 1 --2- 172.21.15.28:0/1020372903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a4806da70 0x7f2a48160b80 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009e50 tx=0x7f2a4002a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f54d72738a02c2e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:36.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.129+0000 7f2a47400640 1 -- 172.21.15.28:0/1020372903 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a40036070 con 0x7f2a4806da70 2024-04-26T21:06:36.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.129+0000 7f2a47400640 1 -- 172.21.15.28:0/1020372903 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a40031600 con 0x7f2a4806da70 2024-04-26T21:06:36.129 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.129+0000 7f2a47400640 1 -- 172.21.15.28:0/1020372903 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a400319d0 con 0x7f2a4806da70 2024-04-26T21:06:36.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.130+0000 7f2a4f600640 1 -- 172.21.15.28:0/1020372903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a4806da70 msgr2=0x7f2a48160b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:36.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.130+0000 7f2a4f600640 1 --2- 172.21.15.28:0/1020372903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a4806da70 0x7f2a48160b80 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009e50 tx=0x7f2a4002a760 comp rx=0 tx=0).stop 2024-04-26T21:06:36.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.130+0000 7f2a4f600640 1 -- 172.21.15.28:0/1020372903 shutdown_connections 2024-04-26T21:06:36.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.130+0000 7f2a4f600640 1 --2- 172.21.15.28:0/1020372903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a4806da70 0x7f2a48160b80 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:36.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.130+0000 7f2a4f600640 1 -- 172.21.15.28:0/1020372903 >> 172.21.15.28:0/1020372903 conn(0x7f2a48073100 msgr2=0x7f2a48073510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:36.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.130+0000 7f2a4f600640 1 -- 172.21.15.28:0/1020372903 shutdown_connections 2024-04-26T21:06:36.130 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.130+0000 7f2a4f600640 1 -- 172.21.15.28:0/1020372903 wait complete. 2024-04-26T21:06:36.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.132+0000 7f2a4f600640 1 Processor -- start 2024-04-26T21:06:36.132 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.132+0000 7f2a4f600640 1 -- start start 2024-04-26T21:06:36.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.133+0000 7f2a4f600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a4806da70 0x7f2a48150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:36.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.133+0000 7f2a4f600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2a480d1730 0x7f2a480b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:36.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.133+0000 7f2a4f600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a4815d090 0x7f2a481475f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:36.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.133+0000 7f2a4f600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2a48074230 con 0x7f2a4815d090 2024-04-26T21:06:36.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.133+0000 7f2a4f600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2a480740b0 con 0x7f2a4806da70 2024-04-26T21:06:36.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.133+0000 7f2a4f600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f2a480743b0 con 0x7f2a480d1730 2024-04-26T21:06:36.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.133+0000 7f2a4d600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a4815d090 0x7f2a481475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:36.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.133+0000 7f2a4d600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a4815d090 0x7f2a481475f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37472/0 (socket says 172.21.15.28:37472) 2024-04-26T21:06:36.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.133+0000 7f2a4d600640 1 -- 172.21.15.28:0/1281541905 learned_addr learned my addr 172.21.15.28:0/1281541905 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:36.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.133+0000 7f2a47e00640 1 --2- 172.21.15.28:0/1281541905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2a480d1730 0x7f2a480b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:36.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.133+0000 7f2a4cc00640 1 --2- 172.21.15.28:0/1281541905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a4806da70 0x7f2a48150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:36.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.134+0000 7f2a4d600640 1 -- 172.21.15.28:0/1281541905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2a480d1730 msgr2=0x7f2a480b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:36.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.134+0000 7f2a4d600640 1 --2- 172.21.15.28:0/1281541905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2a480d1730 0x7f2a480b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:36.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.134+0000 7f2a4d600640 1 -- 172.21.15.28:0/1281541905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a4806da70 msgr2=0x7f2a48150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:36.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.134+0000 7f2a4d600640 1 --2- 172.21.15.28:0/1281541905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a4806da70 0x7f2a48150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:36.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.134+0000 7f2a4d600640 1 -- 172.21.15.28:0/1281541905 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a48154250 con 0x7f2a4815d090 2024-04-26T21:06:36.135 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.134+0000 7f2a47e00640 1 --2- 172.21.15.28:0/1281541905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2a480d1730 0x7f2a480b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:36.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.134+0000 7f2a4cc00640 1 --2- 172.21.15.28:0/1281541905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a4806da70 0x7f2a48150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:36.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.134+0000 7f2a4d600640 1 --2- 172.21.15.28:0/1281541905 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a4815d090 0x7f2a481475f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009e50 tx=0x7f2a40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:36.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.134+0000 7f2a44c00640 1 -- 172.21.15.28:0/1281541905 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a40041020 con 0x7f2a4815d090 2024-04-26T21:06:36.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.134+0000 7f2a44c00640 1 -- 172.21.15.28:0/1281541905 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a40007800 con 0x7f2a4815d090 2024-04-26T21:06:36.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.134+0000 7f2a44c00640 1 -- 172.21.15.28:0/1281541905 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a40036040 con 0x7f2a4815d090 2024-04-26T21:06:36.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.134+0000 7f2a4f600640 1 -- 172.21.15.28:0/1281541905 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a48154530 con 0x7f2a4815d090 2024-04-26T21:06:36.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.134+0000 7f2a4f600640 1 -- 172.21.15.28:0/1281541905 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a48154af0 con 0x7f2a4815d090 2024-04-26T21:06:36.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.136+0000 7f2a4f600640 1 -- 172.21.15.28:0/1281541905 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a4806a400 con 0x7f2a4815d090 2024-04-26T21:06:36.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.137+0000 7f2a44c00640 1 -- 172.21.15.28:0/1281541905 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a40044020 con 0x7f2a4815d090 2024-04-26T21:06:36.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.138+0000 7f2a44c00640 1 --2- 172.21.15.28:0/1281541905 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2a2806c8f0 0x7f2a2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:36.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.138+0000 7f2a44c00640 1 -- 172.21.15.28:0/1281541905 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f2a400ac980 con 0x7f2a4815d090 2024-04-26T21:06:36.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.138+0000 7f2a4cc00640 1 --2- 172.21.15.28:0/1281541905 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2a2806c8f0 0x7f2a2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:36.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.139+0000 7f2a4cc00640 1 --2- 172.21.15.28:0/1281541905 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2a2806c8f0 0x7f2a2806edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00a830 tx=0x7f2a3c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:36.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.142+0000 7f2a44c00640 1 -- 172.21.15.28:0/1281541905 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2a4004b030 con 0x7f2a4815d090 2024-04-26T21:06:36.349 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.348+0000 7f2a4f600640 1 -- 172.21.15.28:0/1281541905 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2a4809fef0 con 0x7f2a4815d090 2024-04-26T21:06:36.352 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.351+0000 7f2a44c00640 1 -- 172.21.15.28:0/1281541905 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16685 (secure 0 0 0) 0x7f2a4007acd0 con 0x7f2a4815d090 2024-04-26T21:06:36.353 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T21:06:36.353 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":47,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:59:38.275251+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:58:41.306286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T20:58:42.497340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.16","mappings":[{"from":0,"to":5}]},{"pgid":"7.30","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:06:36.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.353+0000 7f2a4f600640 1 -- 172.21.15.28:0/1281541905 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2a2806c8f0 msgr2=0x7f2a2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:36.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.353+0000 7f2a4f600640 1 --2- 172.21.15.28:0/1281541905 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2a2806c8f0 0x7f2a2806edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00a830 tx=0x7f2a3c008040 comp rx=0 tx=0).stop 2024-04-26T21:06:36.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.353+0000 7f2a4f600640 1 -- 172.21.15.28:0/1281541905 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a4815d090 msgr2=0x7f2a481475f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:36.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.353+0000 7f2a4f600640 1 --2- 172.21.15.28:0/1281541905 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a4815d090 0x7f2a481475f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009e50 tx=0x7f2a40004a40 comp rx=0 tx=0).stop 2024-04-26T21:06:36.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.354+0000 7f2a4f600640 1 -- 172.21.15.28:0/1281541905 shutdown_connections 2024-04-26T21:06:36.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.354+0000 7f2a4f600640 1 --2- 172.21.15.28:0/1281541905 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2a2806c8f0 0x7f2a2806edb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:36.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.354+0000 7f2a4f600640 1 --2- 172.21.15.28:0/1281541905 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a4815d090 0x7f2a481475f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:36.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.354+0000 7f2a4f600640 1 --2- 172.21.15.28:0/1281541905 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2a480d1730 0x7f2a480b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:36.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.354+0000 7f2a4f600640 1 --2- 172.21.15.28:0/1281541905 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2a4806da70 0x7f2a48150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:36.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.354+0000 7f2a4f600640 1 -- 172.21.15.28:0/1281541905 >> 172.21.15.28:0/1281541905 conn(0x7f2a48073100 msgr2=0x7f2a4806d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:36.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.354+0000 7f2a4f600640 1 -- 172.21.15.28:0/1281541905 shutdown_connections 2024-04-26T21:06:36.356 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.354+0000 7f2a4f600640 1 -- 172.21.15.28:0/1281541905 wait complete. 2024-04-26T21:06:36.377 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:06:36.377 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:06:36.588 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.587+0000 7fc459200640 1 Processor -- start 2024-04-26T21:06:36.589 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.587+0000 7fc459200640 1 -- start start 2024-04-26T21:06:36.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.589+0000 7fc459200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc4540cb0a0 0x7fc45406e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:36.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.589+0000 7fc459200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4540cb500 0x7fc45406ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:36.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.590+0000 7fc459200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc454145a10 con 0x7fc4540c9da0 2024-04-26T21:06:36.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.590+0000 7fc459200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fc4541603f0 con 0x7fc4540cb0a0 2024-04-26T21:06:36.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.590+0000 7fc459200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc45415f580 con 0x7fc4540cb500 2024-04-26T21:06:36.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.590+0000 7fc452a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc4540cb0a0 0x7fc45406e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:36.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.590+0000 7fc453400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc4540c9da0 0x7fc4540ca180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45140/0 (socket says 172.21.15.28:45140) 2024-04-26T21:06:36.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.590+0000 7fc453400640 1 -- 172.21.15.28:0/2481734758 learned_addr learned my addr 172.21.15.28:0/2481734758 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:36.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.590+0000 7fc453e00640 1 --2- 172.21.15.28:0/2481734758 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4540cb500 0x7fc45406ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:36.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.590+0000 7fc452a00640 1 -- 172.21.15.28:0/2481734758 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4540cb500 msgr2=0x7fc45406ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:36.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.590+0000 7fc452a00640 1 --2- 172.21.15.28:0/2481734758 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4540cb500 0x7fc45406ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:36.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.591+0000 7fc452a00640 1 -- 172.21.15.28:0/2481734758 >> v1:172.21.15.28:6789/0 conn(0x7fc4540c9da0 legacy=0x7fc4540ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:36.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.591+0000 7fc452a00640 1 -- 172.21.15.28:0/2481734758 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45415f7e0 con 0x7fc4540cb0a0 2024-04-26T21:06:36.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.591+0000 7fc453e00640 1 --2- 172.21.15.28:0/2481734758 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4540cb500 0x7fc45406ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:36.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.592+0000 7fc452a00640 1 --2- 172.21.15.28:0/2481734758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4540cb0a0 0x7fc45406e9d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc4400097a0 tx=0x7fc44002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940027175c0eb6a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:36.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.592+0000 7fc452000640 1 -- 172.21.15.28:0/2481734758 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440037070 con 0x7fc4540cb0a0 2024-04-26T21:06:36.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.592+0000 7fc452000640 1 -- 172.21.15.28:0/2481734758 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc440032440 con 0x7fc4540cb0a0 2024-04-26T21:06:36.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.592+0000 7fc452000640 1 -- 172.21.15.28:0/2481734758 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440032810 con 0x7fc4540cb0a0 2024-04-26T21:06:36.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.593+0000 7fc459200640 1 -- 172.21.15.28:0/2481734758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4540cb0a0 msgr2=0x7fc45406e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:36.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.593+0000 7fc459200640 1 --2- 172.21.15.28:0/2481734758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4540cb0a0 0x7fc45406e9d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc4400097a0 tx=0x7fc44002a180 comp rx=0 tx=0).stop 2024-04-26T21:06:36.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.593+0000 7fc459200640 1 -- 172.21.15.28:0/2481734758 shutdown_connections 2024-04-26T21:06:36.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.593+0000 7fc459200640 1 --2- 172.21.15.28:0/2481734758 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4540cb500 0x7fc45406ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:36.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.593+0000 7fc459200640 1 --2- 172.21.15.28:0/2481734758 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4540cb0a0 0x7fc45406e9d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:36.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.593+0000 7fc459200640 1 -- 172.21.15.28:0/2481734758 >> 172.21.15.28:0/2481734758 conn(0x7fc4540d0a50 msgr2=0x7fc4540d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:36.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.593+0000 7fc459200640 1 -- 172.21.15.28:0/2481734758 shutdown_connections 2024-04-26T21:06:36.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.593+0000 7fc459200640 1 -- 172.21.15.28:0/2481734758 wait complete. 2024-04-26T21:06:36.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.595+0000 7fc459200640 1 Processor -- start 2024-04-26T21:06:36.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.595+0000 7fc459200640 1 -- start start 2024-04-26T21:06:36.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.596+0000 7fc459200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4540c9da0 0x7fc45413c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:36.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.596+0000 7fc459200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4540cb0a0 0x7fc45413c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:36.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.596+0000 7fc459200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4540cb500 0x7fc454162ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:36.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.596+0000 7fc459200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fc454160080 con 0x7fc4540cb0a0 2024-04-26T21:06:36.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.596+0000 7fc459200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc45415fda0 con 0x7fc4540c9da0 2024-04-26T21:06:36.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.596+0000 7fc459200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fc4541489a0 con 0x7fc4540cb500 2024-04-26T21:06:36.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.596+0000 7fc452a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4540cb0a0 0x7fc45413c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:36.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.596+0000 7fc452a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4540cb0a0 0x7fc45413c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37498/0 (socket says 172.21.15.28:37498) 2024-04-26T21:06:36.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.596+0000 7fc452a00640 1 -- 172.21.15.28:0/2748881438 learned_addr learned my addr 172.21.15.28:0/2748881438 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:36.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.597+0000 7fc453e00640 1 --2- 172.21.15.28:0/2748881438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4540cb500 0x7fc454162ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:36.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.597+0000 7fc453400640 1 --2- 172.21.15.28:0/2748881438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4540c9da0 0x7fc45413c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:36.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.597+0000 7fc452a00640 1 -- 172.21.15.28:0/2748881438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4540cb500 msgr2=0x7fc454162ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:36.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.597+0000 7fc452a00640 1 --2- 172.21.15.28:0/2748881438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4540cb500 0x7fc454162ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:36.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.597+0000 7fc452a00640 1 -- 172.21.15.28:0/2748881438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4540c9da0 msgr2=0x7fc45413c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:36.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.597+0000 7fc452a00640 1 --2- 172.21.15.28:0/2748881438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4540c9da0 0x7fc45413c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:36.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.597+0000 7fc452a00640 1 -- 172.21.15.28:0/2748881438 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc454160dc0 con 0x7fc4540cb0a0 2024-04-26T21:06:36.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.597+0000 7fc453e00640 1 --2- 172.21.15.28:0/2748881438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4540cb500 0x7fc454162ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:36.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.597+0000 7fc452a00640 1 --2- 172.21.15.28:0/2748881438 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4540cb0a0 0x7fc45413c640 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc440035040 tx=0x7fc4400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:36.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.597+0000 7fc44b400640 1 -- 172.21.15.28:0/2748881438 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440037040 con 0x7fc4540cb0a0 2024-04-26T21:06:36.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.597+0000 7fc44b400640 1 -- 172.21.15.28:0/2748881438 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc440030770 con 0x7fc4540cb0a0 2024-04-26T21:06:36.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.597+0000 7fc44b400640 1 -- 172.21.15.28:0/2748881438 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc440030b80 con 0x7fc4540cb0a0 2024-04-26T21:06:36.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.597+0000 7fc459200640 1 -- 172.21.15.28:0/2748881438 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4541610a0 con 0x7fc4540cb0a0 2024-04-26T21:06:36.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.598+0000 7fc459200640 1 -- 172.21.15.28:0/2748881438 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc454161660 con 0x7fc4540cb0a0 2024-04-26T21:06:36.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.599+0000 7fc459200640 1 -- 172.21.15.28:0/2748881438 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc45406a360 con 0x7fc4540cb0a0 2024-04-26T21:06:36.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.600+0000 7fc44b400640 1 -- 172.21.15.28:0/2748881438 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc440044020 con 0x7fc4540cb0a0 2024-04-26T21:06:36.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.601+0000 7fc44b400640 1 --2- 172.21.15.28:0/2748881438 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc43406c8f0 0x7fc43406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:36.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.601+0000 7fc44b400640 1 -- 172.21.15.28:0/2748881438 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fc4400ac190 con 0x7fc4540cb0a0 2024-04-26T21:06:36.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.601+0000 7fc453400640 1 --2- 172.21.15.28:0/2748881438 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc43406c8f0 0x7fc43406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:36.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.602+0000 7fc453400640 1 --2- 172.21.15.28:0/2748881438 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc43406c8f0 0x7fc43406edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc444000f60 tx=0x7fc444009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:36.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.606+0000 7fc44b400640 1 -- 172.21.15.28:0/2748881438 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc440032e20 con 0x7fc4540cb0a0 2024-04-26T21:06:36.854 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.853+0000 7fc459200640 1 -- 172.21.15.28:0/2748881438 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc454074550 con 0x7fc4540cb0a0 2024-04-26T21:06:36.855 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.854+0000 7fc44b400640 1 -- 172.21.15.28:0/2748881438 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5184 (secure 0 0 0) 0x7fc44007a570 con 0x7fc4540cb0a0 2024-04-26T21:06:36.856 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T21:06:36.856 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/356870645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":356870645},{"type":"v1","addr":"172.21.15.28:6835","nonce":356870645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4829,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2957511478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":2957511478},{"type":"v1","addr":"172.21.15.103:6835","nonce":2957511478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/2060632176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2060632176},{"type":"v1","addr":"172.21.15.103:6834","nonce":2060632176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:58:43.534341+0000","modified":"2024-04-26T20:58:46.581816+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5571},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6837/2997047727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":2997047727},{"type":"v1","addr":"172.21.15.28:6837","nonce":2997047727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5571,"qdb_cluster":[5571]},"id":3}]} 2024-04-26T21:06:36.856 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 26 2024-04-26T21:06:36.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.858+0000 7fc459200640 1 -- 172.21.15.28:0/2748881438 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc43406c8f0 msgr2=0x7fc43406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:36.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.858+0000 7fc459200640 1 --2- 172.21.15.28:0/2748881438 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc43406c8f0 0x7fc43406edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc444000f60 tx=0x7fc444009040 comp rx=0 tx=0).stop 2024-04-26T21:06:36.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.858+0000 7fc459200640 1 -- 172.21.15.28:0/2748881438 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4540cb0a0 msgr2=0x7fc45413c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:36.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.858+0000 7fc459200640 1 --2- 172.21.15.28:0/2748881438 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4540cb0a0 0x7fc45413c640 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fc440035040 tx=0x7fc4400046c0 comp rx=0 tx=0).stop 2024-04-26T21:06:36.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.858+0000 7fc459200640 1 -- 172.21.15.28:0/2748881438 shutdown_connections 2024-04-26T21:06:36.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.858+0000 7fc459200640 1 --2- 172.21.15.28:0/2748881438 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fc43406c8f0 0x7fc43406edb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:36.859 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.858+0000 7fc459200640 1 --2- 172.21.15.28:0/2748881438 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fc4540cb500 0x7fc454162ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:36.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.858+0000 7fc459200640 1 --2- 172.21.15.28:0/2748881438 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4540cb0a0 0x7fc45413c640 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:36.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.858+0000 7fc459200640 1 --2- 172.21.15.28:0/2748881438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc4540c9da0 0x7fc45413c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:36.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.858+0000 7fc459200640 1 -- 172.21.15.28:0/2748881438 >> 172.21.15.28:0/2748881438 conn(0x7fc4540d0a50 msgr2=0x7fc4540cbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:36.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.858+0000 7fc459200640 1 -- 172.21.15.28:0/2748881438 shutdown_connections 2024-04-26T21:06:36.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:36.858+0000 7fc459200640 1 -- 172.21.15.28:0/2748881438 wait complete. 2024-04-26T21:06:36.879 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:06:37.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.091+0000 7efe33e00640 1 Processor -- start 2024-04-26T21:06:37.092 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.091+0000 7efe33e00640 1 -- start start 2024-04-26T21:06:37.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.092+0000 7efe33e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7efe3415f1b0 0x7efe340d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:37.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.092+0000 7efe33e00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7efe340cf700 0x7efe340d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:37.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.092+0000 7efe33e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efe340d5370 0x7efe340b80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:37.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.092+0000 7efe33e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7efe340cff00 con 0x7efe3415f1b0 2024-04-26T21:06:37.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.092+0000 7efe33e00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7efe340d4270 con 0x7efe340cf700 2024-04-26T21:06:37.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.092+0000 7efe33e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7efe340d0710 con 0x7efe340d5370 2024-04-26T21:06:37.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.093+0000 7efe32a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7efe3415f1b0 0x7efe340d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:37.094 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.093+0000 7efe32a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7efe3415f1b0 0x7efe340d4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37508/0 (socket says 172.21.15.28:37508) 2024-04-26T21:06:37.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.093+0000 7efe32a00640 1 -- 172.21.15.28:0/104944742 learned_addr learned my addr 172.21.15.28:0/104944742 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:37.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.093+0000 7efe33400640 1 --2- 172.21.15.28:0/104944742 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efe340d5370 0x7efe340b80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:37.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.093+0000 7efe32000640 1 --2- 172.21.15.28:0/104944742 >> v2:172.21.15.103:3300/0 conn(0x7efe340cf700 0x7efe340d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:37.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.094+0000 7efe32a00640 1 -- 172.21.15.28:0/104944742 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efe340d5370 msgr2=0x7efe340b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:37.095 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.094+0000 7efe32a00640 1 --2- 172.21.15.28:0/104944742 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efe340d5370 0x7efe340b80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.094+0000 7efe32a00640 1 -- 172.21.15.28:0/104944742 >> v2:172.21.15.103:3300/0 conn(0x7efe340cf700 msgr2=0x7efe340d4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:37.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.094+0000 7efe32a00640 1 --2- 172.21.15.28:0/104944742 >> v2:172.21.15.103:3300/0 conn(0x7efe340cf700 0x7efe340d4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.094+0000 7efe32a00640 1 -- 172.21.15.28:0/104944742 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe340d0150 con 0x7efe3415f1b0 2024-04-26T21:06:37.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.094+0000 7efe33400640 1 --2- 172.21.15.28:0/104944742 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efe340d5370 0x7efe340b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:06:37.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.094+0000 7efe32000640 1 --2- 172.21.15.28:0/104944742 >> v2:172.21.15.103:3300/0 conn(0x7efe340cf700 0x7efe340d4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:06:37.096 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.095+0000 7efe32a00640 1 --2- 172.21.15.28:0/104944742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe3415f1b0 0x7efe340d4700 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7efe340b7a90 tx=0x7efe2402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12d7168982e9859 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:37.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.095+0000 7efe31600640 1 -- 172.21.15.28:0/104944742 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24037070 con 0x7efe3415f1b0 2024-04-26T21:06:37.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.096+0000 7efe31600640 1 -- 172.21.15.28:0/104944742 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe24032440 con 0x7efe3415f1b0 2024-04-26T21:06:37.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.096+0000 7efe31600640 1 -- 172.21.15.28:0/104944742 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24032810 con 0x7efe3415f1b0 2024-04-26T21:06:37.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.096+0000 7efe33e00640 1 -- 172.21.15.28:0/104944742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe3415f1b0 msgr2=0x7efe340d4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:37.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.096+0000 7efe33e00640 1 --2- 172.21.15.28:0/104944742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe3415f1b0 0x7efe340d4700 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7efe340b7a90 tx=0x7efe2402a270 comp rx=0 tx=0).stop 2024-04-26T21:06:37.097 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.096+0000 7efe33e00640 1 -- 172.21.15.28:0/104944742 shutdown_connections 2024-04-26T21:06:37.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.096+0000 7efe33e00640 1 --2- 172.21.15.28:0/104944742 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efe340d5370 0x7efe340b80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.096+0000 7efe33e00640 1 --2- 172.21.15.28:0/104944742 >> v2:172.21.15.103:3300/0 conn(0x7efe340cf700 0x7efe340d4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.096+0000 7efe33e00640 1 --2- 172.21.15.28:0/104944742 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe3415f1b0 0x7efe340d4700 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.096+0000 7efe33e00640 1 -- 172.21.15.28:0/104944742 >> 172.21.15.28:0/104944742 conn(0x7efe3406c770 msgr2=0x7efe3406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:37.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.096+0000 7efe33e00640 1 -- 172.21.15.28:0/104944742 shutdown_connections 2024-04-26T21:06:37.098 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.097+0000 7efe33e00640 1 -- 172.21.15.28:0/104944742 wait complete. 2024-04-26T21:06:37.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.099+0000 7efe33e00640 1 Processor -- start 2024-04-26T21:06:37.100 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.099+0000 7efe33e00640 1 -- start start 2024-04-26T21:06:37.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.100+0000 7efe33e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe340cf700 0x7efe3415f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:37.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.100+0000 7efe33e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe340d5370 0x7efe340caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:37.102 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.100+0000 7efe33e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efe3415f1b0 0x7efe341611f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:37.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.100+0000 7efe33e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7efe3415f590 con 0x7efe340cf700 2024-04-26T21:06:37.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.100+0000 7efe33e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efe340d3f00 con 0x7efe340d5370 2024-04-26T21:06:37.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.100+0000 7efe33e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7efe340b4410 con 0x7efe3415f1b0 2024-04-26T21:06:37.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.101+0000 7efe32a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe340cf700 0x7efe3415f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:37.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.101+0000 7efe32a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe340cf700 0x7efe3415f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37522/0 (socket says 172.21.15.28:37522) 2024-04-26T21:06:37.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.101+0000 7efe32a00640 1 -- 172.21.15.28:0/124452490 learned_addr learned my addr 172.21.15.28:0/124452490 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:37.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.101+0000 7efe32000640 1 --2- 172.21.15.28:0/124452490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe340d5370 0x7efe340caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:37.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.101+0000 7efe32a00640 1 -- 172.21.15.28:0/124452490 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efe3415f1b0 msgr2=0x7efe341611f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:37.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.101+0000 7efe33400640 1 --2- 172.21.15.28:0/124452490 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efe3415f1b0 0x7efe341611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:37.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.101+0000 7efe32a00640 1 --2- 172.21.15.28:0/124452490 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efe3415f1b0 0x7efe341611f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.101+0000 7efe32a00640 1 -- 172.21.15.28:0/124452490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe340d5370 msgr2=0x7efe340caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:37.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.101+0000 7efe32a00640 1 --2- 172.21.15.28:0/124452490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe340d5370 0x7efe340caca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.101+0000 7efe32a00640 1 -- 172.21.15.28:0/124452490 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe34161730 con 0x7efe340cf700 2024-04-26T21:06:37.103 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.101+0000 7efe33400640 1 --2- 172.21.15.28:0/124452490 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efe3415f1b0 0x7efe341611f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:37.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.101+0000 7efe32000640 1 --2- 172.21.15.28:0/124452490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe340d5370 0x7efe340caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:37.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.101+0000 7efe32a00640 1 --2- 172.21.15.28:0/124452490 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe340cf700 0x7efe3415f710 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7efe24000c00 tx=0x7efe240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:37.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.102+0000 7efe2aa00640 1 -- 172.21.15.28:0/124452490 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24041020 con 0x7efe340cf700 2024-04-26T21:06:37.104 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.102+0000 7efe2aa00640 1 -- 172.21.15.28:0/124452490 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe24030780 con 0x7efe340cf700 2024-04-26T21:06:37.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.102+0000 7efe2aa00640 1 -- 172.21.15.28:0/124452490 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24037040 con 0x7efe340cf700 2024-04-26T21:06:37.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.102+0000 7efe33e00640 1 -- 172.21.15.28:0/124452490 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3415fcf0 con 0x7efe340cf700 2024-04-26T21:06:37.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.102+0000 7efe33e00640 1 -- 172.21.15.28:0/124452490 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe341602b0 con 0x7efe340cf700 2024-04-26T21:06:37.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.103+0000 7efe33e00640 1 -- 172.21.15.28:0/124452490 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe3406a360 con 0x7efe340cf700 2024-04-26T21:06:37.105 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.104+0000 7efe2aa00640 1 -- 172.21.15.28:0/124452490 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe24044020 con 0x7efe340cf700 2024-04-26T21:06:37.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.106+0000 7efe2aa00640 1 --2- 172.21.15.28:0/124452490 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efe1406c8f0 0x7efe1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:37.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.106+0000 7efe2aa00640 1 -- 172.21.15.28:0/124452490 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7efe240ac370 con 0x7efe340cf700 2024-04-26T21:06:37.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.106+0000 7efe32000640 1 --2- 172.21.15.28:0/124452490 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efe1406c8f0 0x7efe1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:37.107 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.107+0000 7efe32000640 1 --2- 172.21.15.28:0/124452490 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efe1406c8f0 0x7efe1406edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efe200095c0 tx=0x7efe2000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:37.111 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.110+0000 7efe2aa00640 1 -- 172.21.15.28:0/124452490 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efe2404b030 con 0x7efe340cf700 2024-04-26T21:06:37.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.360+0000 7efe33e00640 1 -- 172.21.15.28:0/124452490 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe340ce580 con 0x7efe340cf700 2024-04-26T21:06:37.362 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.362+0000 7efe2aa00640 1 -- 172.21.15.28:0/124452490 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5184 (secure 0 0 0) 0x7efe2407a760 con 0x7efe340cf700 2024-04-26T21:06:37.363 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T21:06:37.363 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/356870645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":356870645},{"type":"v1","addr":"172.21.15.28:6835","nonce":356870645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4829,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2957511478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":2957511478},{"type":"v1","addr":"172.21.15.103:6835","nonce":2957511478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/2060632176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2060632176},{"type":"v1","addr":"172.21.15.103:6834","nonce":2060632176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T20:58:43.534341+0000","modified":"2024-04-26T20:58:46.581816+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5571},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5571":{"gid":5571,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.28:6837/2997047727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":2997047727},{"type":"v1","addr":"172.21.15.28:6837","nonce":2997047727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5571,"qdb_cluster":[5571]},"id":3}]} 2024-04-26T21:06:37.363 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 26 2024-04-26T21:06:37.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.364+0000 7efe33e00640 1 -- 172.21.15.28:0/124452490 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efe1406c8f0 msgr2=0x7efe1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:37.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.364+0000 7efe33e00640 1 --2- 172.21.15.28:0/124452490 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efe1406c8f0 0x7efe1406edb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efe200095c0 tx=0x7efe2000b040 comp rx=0 tx=0).stop 2024-04-26T21:06:37.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.364+0000 7efe33e00640 1 -- 172.21.15.28:0/124452490 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe340cf700 msgr2=0x7efe3415f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:37.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.364+0000 7efe33e00640 1 --2- 172.21.15.28:0/124452490 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe340cf700 0x7efe3415f710 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7efe24000c00 tx=0x7efe240046c0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.364+0000 7efe33e00640 1 -- 172.21.15.28:0/124452490 shutdown_connections 2024-04-26T21:06:37.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.364+0000 7efe33e00640 1 --2- 172.21.15.28:0/124452490 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7efe1406c8f0 0x7efe1406edb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.364+0000 7efe33e00640 1 --2- 172.21.15.28:0/124452490 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7efe3415f1b0 0x7efe341611f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.364+0000 7efe33e00640 1 --2- 172.21.15.28:0/124452490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe340d5370 0x7efe340caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.364+0000 7efe33e00640 1 --2- 172.21.15.28:0/124452490 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe340cf700 0x7efe3415f710 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.364+0000 7efe33e00640 1 -- 172.21.15.28:0/124452490 >> 172.21.15.28:0/124452490 conn(0x7efe3406c770 msgr2=0x7efe340cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:37.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.365+0000 7efe33e00640 1 -- 172.21.15.28:0/124452490 shutdown_connections 2024-04-26T21:06:37.367 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.365+0000 7efe33e00640 1 -- 172.21.15.28:0/124452490 wait complete. 2024-04-26T21:06:37.385 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:06:37.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.591+0000 7fe95ea00640 1 Processor -- start 2024-04-26T21:06:37.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.591+0000 7fe95ea00640 1 -- start start 2024-04-26T21:06:37.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.593+0000 7fe95ea00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe958146980 0x7fe9580b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:37.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.593+0000 7fe95ea00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe9580740e0 0x7fe9580b6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:37.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.593+0000 7fe95ea00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9580d0a30 con 0x7fe95815d5b0 2024-04-26T21:06:37.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.593+0000 7fe95ea00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe95815e510 con 0x7fe9580740e0 2024-04-26T21:06:37.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.593+0000 7fe95ea00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe95815e230 con 0x7fe958146980 2024-04-26T21:06:37.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.593+0000 7fe957400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe9580740e0 0x7fe9580b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:37.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.594+0000 7fe95ca00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe95815d5b0 0x7fe95815da40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45154/0 (socket says 172.21.15.28:45154) 2024-04-26T21:06:37.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.594+0000 7fe95ca00640 1 -- 172.21.15.28:0/3944705365 learned_addr learned my addr 172.21.15.28:0/3944705365 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:37.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.594+0000 7fe957e00640 1 --2- 172.21.15.28:0/3944705365 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe958146980 0x7fe9580b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:37.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.594+0000 7fe957400640 1 -- 172.21.15.28:0/3944705365 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe958146980 msgr2=0x7fe9580b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:37.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.594+0000 7fe957400640 1 --2- 172.21.15.28:0/3944705365 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe958146980 0x7fe9580b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.594+0000 7fe957400640 1 -- 172.21.15.28:0/3944705365 >> v1:172.21.15.28:6789/0 conn(0x7fe95815d5b0 legacy=0x7fe95815da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:37.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.594+0000 7fe956a00640 1 -- 172.21.15.28:0/3944705365 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4082398925 0 0) 0x7fe950001490 con 0x7fe95815d5b0 2024-04-26T21:06:37.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.594+0000 7fe957400640 1 -- 172.21.15.28:0/3944705365 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe958073600 con 0x7fe9580740e0 2024-04-26T21:06:37.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.595+0000 7fe957400640 1 --2- 172.21.15.28:0/3944705365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9580740e0 0x7fe9580b6640 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe9480097a0 tx=0x7fe94802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c5ae6efd67f491 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:37.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.596+0000 7fe956a00640 1 -- 172.21.15.28:0/3944705365 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe948037070 con 0x7fe9580740e0 2024-04-26T21:06:37.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.596+0000 7fe956a00640 1 -- 172.21.15.28:0/3944705365 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe94802ae50 con 0x7fe9580740e0 2024-04-26T21:06:37.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.596+0000 7fe956a00640 1 -- 172.21.15.28:0/3944705365 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe948032660 con 0x7fe9580740e0 2024-04-26T21:06:37.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.596+0000 7fe95ea00640 1 -- 172.21.15.28:0/3944705365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9580740e0 msgr2=0x7fe9580b6640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:37.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.596+0000 7fe95ea00640 1 --2- 172.21.15.28:0/3944705365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9580740e0 0x7fe9580b6640 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe9480097a0 tx=0x7fe94802a100 comp rx=0 tx=0).stop 2024-04-26T21:06:37.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.597+0000 7fe95ea00640 1 -- 172.21.15.28:0/3944705365 shutdown_connections 2024-04-26T21:06:37.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.597+0000 7fe95ea00640 1 --2- 172.21.15.28:0/3944705365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9580740e0 0x7fe9580b6640 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.597+0000 7fe95ea00640 1 --2- 172.21.15.28:0/3944705365 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe958146980 0x7fe9580b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.597+0000 7fe95ea00640 1 -- 172.21.15.28:0/3944705365 >> 172.21.15.28:0/3944705365 conn(0x7fe95806e220 msgr2=0x7fe95806e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:37.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.597+0000 7fe95ea00640 1 -- 172.21.15.28:0/3944705365 shutdown_connections 2024-04-26T21:06:37.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.597+0000 7fe95ea00640 1 -- 172.21.15.28:0/3944705365 wait complete. 2024-04-26T21:06:37.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.599+0000 7fe95ea00640 1 Processor -- start 2024-04-26T21:06:37.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.599+0000 7fe95ea00640 1 -- start start 2024-04-26T21:06:37.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.600+0000 7fe95ea00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe9580740e0 0x7fe9581588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:37.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.600+0000 7fe95ea00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe958146980 0x7fe95815a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:37.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.600+0000 7fe95ea00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe95815d5b0 0x7fe9580b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:37.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.600+0000 7fe95ea00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe958071ff0 con 0x7fe95815d5b0 2024-04-26T21:06:37.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.600+0000 7fe95ea00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe958073bc0 con 0x7fe958146980 2024-04-26T21:06:37.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.600+0000 7fe95ea00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe958072170 con 0x7fe9580740e0 2024-04-26T21:06:37.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.600+0000 7fe95ca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe95815d5b0 0x7fe9580b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:37.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.600+0000 7fe95ca00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe95815d5b0 0x7fe9580b50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37542/0 (socket says 172.21.15.28:37542) 2024-04-26T21:06:37.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.600+0000 7fe95ca00640 1 -- 172.21.15.28:0/2615715145 learned_addr learned my addr 172.21.15.28:0/2615715145 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:37.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.600+0000 7fe95ca00640 1 -- 172.21.15.28:0/2615715145 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe9580740e0 msgr2=0x7fe9581588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:37.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.600+0000 7fe95ca00640 1 --2- 172.21.15.28:0/2615715145 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe9580740e0 0x7fe9581588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.600+0000 7fe95ca00640 1 -- 172.21.15.28:0/2615715145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe958146980 msgr2=0x7fe95815a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:37.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.600+0000 7fe95ca00640 1 --2- 172.21.15.28:0/2615715145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe958146980 0x7fe95815a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.600+0000 7fe95ca00640 1 -- 172.21.15.28:0/2615715145 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9580b5940 con 0x7fe95815d5b0 2024-04-26T21:06:37.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.601+0000 7fe95ca00640 1 --2- 172.21.15.28:0/2615715145 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe95815d5b0 0x7fe9580b50f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe95000c7b0 tx=0x7fe95000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:37.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.601+0000 7fe94fe00640 1 -- 172.21.15.28:0/2615715145 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950018070 con 0x7fe95815d5b0 2024-04-26T21:06:37.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.601+0000 7fe94fe00640 1 -- 172.21.15.28:0/2615715145 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe95000a640 con 0x7fe95815d5b0 2024-04-26T21:06:37.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.601+0000 7fe94fe00640 1 -- 172.21.15.28:0/2615715145 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95000aa30 con 0x7fe95815d5b0 2024-04-26T21:06:37.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.601+0000 7fe95ea00640 1 -- 172.21.15.28:0/2615715145 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9580b5c20 con 0x7fe95815d5b0 2024-04-26T21:06:37.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.601+0000 7fe95ea00640 1 -- 172.21.15.28:0/2615715145 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9580aecd0 con 0x7fe95815d5b0 2024-04-26T21:06:37.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.602+0000 7fe95ea00640 1 -- 172.21.15.28:0/2615715145 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe95806a400 con 0x7fe95815d5b0 2024-04-26T21:06:37.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.603+0000 7fe94fe00640 1 -- 172.21.15.28:0/2615715145 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe95001b070 con 0x7fe95815d5b0 2024-04-26T21:06:37.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.605+0000 7fe94fe00640 1 --2- 172.21.15.28:0/2615715145 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe93406c8f0 0x7fe93406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:37.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.605+0000 7fe94fe00640 1 -- 172.21.15.28:0/2615715145 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fe95008b190 con 0x7fe95815d5b0 2024-04-26T21:06:37.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.605+0000 7fe957e00640 1 --2- 172.21.15.28:0/2615715145 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe93406c8f0 0x7fe93406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:37.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.606+0000 7fe957e00640 1 --2- 172.21.15.28:0/2615715145 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe93406c8f0 0x7fe93406edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe9400098f0 tx=0x7fe940008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:37.610 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.609+0000 7fe94fe00640 1 -- 172.21.15.28:0/2615715145 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe9580d0a30 con 0x7fe95815d5b0 2024-04-26T21:06:37.816 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.815+0000 7fe95ea00640 1 -- 172.21.15.28:0/2615715145 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe95806ff00 con 0x7fe95815d5b0 2024-04-26T21:06:37.819 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.818+0000 7fe94fe00640 1 -- 172.21.15.28:0/2615715145 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16685 (secure 0 0 0) 0x7fe950059570 con 0x7fe95815d5b0 2024-04-26T21:06:37.820 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T21:06:37.821 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":47,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T20:59:38.275251+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-26T20:58:41.306286+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-26T20:58:42.497340+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.16","mappings":[{"from":0,"to":5}]},{"pgid":"7.30","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:06:37.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.821+0000 7fe95ea00640 1 -- 172.21.15.28:0/2615715145 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe93406c8f0 msgr2=0x7fe93406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:37.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.821+0000 7fe95ea00640 1 --2- 172.21.15.28:0/2615715145 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe93406c8f0 0x7fe93406edb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe9400098f0 tx=0x7fe940008040 comp rx=0 tx=0).stop 2024-04-26T21:06:37.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.821+0000 7fe95ea00640 1 -- 172.21.15.28:0/2615715145 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe95815d5b0 msgr2=0x7fe9580b50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:37.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.821+0000 7fe95ea00640 1 --2- 172.21.15.28:0/2615715145 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe95815d5b0 0x7fe9580b50f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe95000c7b0 tx=0x7fe95000cc70 comp rx=0 tx=0).stop 2024-04-26T21:06:37.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.821+0000 7fe95ea00640 1 -- 172.21.15.28:0/2615715145 shutdown_connections 2024-04-26T21:06:37.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.821+0000 7fe95ea00640 1 --2- 172.21.15.28:0/2615715145 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe93406c8f0 0x7fe93406edb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.821+0000 7fe95ea00640 1 --2- 172.21.15.28:0/2615715145 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe95815d5b0 0x7fe9580b50f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.821+0000 7fe95ea00640 1 --2- 172.21.15.28:0/2615715145 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe958146980 0x7fe95815a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.821+0000 7fe95ea00640 1 --2- 172.21.15.28:0/2615715145 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe9580740e0 0x7fe9581588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:37.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.821+0000 7fe95ea00640 1 -- 172.21.15.28:0/2615715145 >> 172.21.15.28:0/2615715145 conn(0x7fe95806e220 msgr2=0x7fe9580b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:37.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.822+0000 7fe95ea00640 1 -- 172.21.15.28:0/2615715145 shutdown_connections 2024-04-26T21:06:37.823 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:37.822+0000 7fe95ea00640 1 -- 172.21.15.28:0/2615715145 wait complete. 2024-04-26T21:06:37.841 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T21:06:38.041 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.040+0000 7f2c1f200640 1 Processor -- start 2024-04-26T21:06:38.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.040+0000 7f2c1f200640 1 -- start start 2024-04-26T21:06:38.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.042+0000 7f2c1f200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2c200cb0a0 0x7f2c2006e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:38.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.042+0000 7f2c1f200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c200cb500 0x7f2c2006ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:38.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.042+0000 7f2c1f200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c20145a10 con 0x7f2c200c9da0 2024-04-26T21:06:38.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.042+0000 7f2c1f200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f2c201603f0 con 0x7f2c200cb0a0 2024-04-26T21:06:38.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.042+0000 7f2c1f200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f2c2015f580 con 0x7f2c200cb500 2024-04-26T21:06:38.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.042+0000 7f2c1d400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2c200cb0a0 0x7f2c2006e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:38.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.042+0000 7f2c1d400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2c200cb0a0 0x7f2c2006e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37544/0 (socket says 172.21.15.28:37544) 2024-04-26T21:06:38.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.042+0000 7f2c1d400640 1 -- 172.21.15.28:0/2294619713 learned_addr learned my addr 172.21.15.28:0/2294619713 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:38.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.042+0000 7f2c1de00640 1 --1- 172.21.15.28:0/2294619713 >> v1:172.21.15.28:6789/0 conn(0x7f2c200c9da0 0x7f2c200ca180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45164/0 (socket says 172.21.15.28:45164) 2024-04-26T21:06:38.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.043+0000 7f2c1e800640 1 --2- 172.21.15.28:0/2294619713 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c200cb500 0x7f2c2006ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:38.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.043+0000 7f2c1d400640 1 -- 172.21.15.28:0/2294619713 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c200cb500 msgr2=0x7f2c2006ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:38.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.043+0000 7f2c1d400640 1 --2- 172.21.15.28:0/2294619713 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c200cb500 0x7f2c2006ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:38.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.043+0000 7f2c1d400640 1 -- 172.21.15.28:0/2294619713 >> v1:172.21.15.28:6789/0 conn(0x7f2c200c9da0 legacy=0x7f2c200ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:38.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.043+0000 7f2c1d400640 1 -- 172.21.15.28:0/2294619713 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2015f7e0 con 0x7f2c200cb0a0 2024-04-26T21:06:38.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.043+0000 7f2c1ca00640 1 -- 172.21.15.28:0/2294619713 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3155612459 0 0) 0x7f2c20145a10 con 0x7f2c200c9da0 2024-04-26T21:06:38.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.043+0000 7f2c1e800640 1 --2- 172.21.15.28:0/2294619713 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c200cb500 0x7f2c2006ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:06:38.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.045+0000 7f2c1d400640 1 --2- 172.21.15.28:0/2294619713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c200cb0a0 0x7f2c2006e9d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c009870 tx=0x7f2c0c02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685e672d7a17cef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:38.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.045+0000 7f2c1ca00640 1 -- 172.21.15.28:0/2294619713 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c037070 con 0x7f2c200cb0a0 2024-04-26T21:06:38.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.045+0000 7f2c1ca00640 1 -- 172.21.15.28:0/2294619713 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c0c032440 con 0x7f2c200cb0a0 2024-04-26T21:06:38.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.045+0000 7f2c1ca00640 1 -- 172.21.15.28:0/2294619713 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0c032810 con 0x7f2c200cb0a0 2024-04-26T21:06:38.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.045+0000 7f2c1f200640 1 -- 172.21.15.28:0/2294619713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c200cb0a0 msgr2=0x7f2c2006e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:38.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.045+0000 7f2c1f200640 1 --2- 172.21.15.28:0/2294619713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c200cb0a0 0x7f2c2006e9d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c009870 tx=0x7f2c0c02a250 comp rx=0 tx=0).stop 2024-04-26T21:06:38.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.045+0000 7f2c1f200640 1 -- 172.21.15.28:0/2294619713 shutdown_connections 2024-04-26T21:06:38.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.045+0000 7f2c1f200640 1 --2- 172.21.15.28:0/2294619713 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c200cb500 0x7f2c2006ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:38.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.046+0000 7f2c1f200640 1 --2- 172.21.15.28:0/2294619713 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c200cb0a0 0x7f2c2006e9d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:38.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.046+0000 7f2c1f200640 1 -- 172.21.15.28:0/2294619713 >> 172.21.15.28:0/2294619713 conn(0x7f2c200d0a50 msgr2=0x7f2c200d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:38.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.046+0000 7f2c1f200640 1 -- 172.21.15.28:0/2294619713 shutdown_connections 2024-04-26T21:06:38.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.046+0000 7f2c1f200640 1 -- 172.21.15.28:0/2294619713 wait complete. 2024-04-26T21:06:38.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.047+0000 7f2c1f200640 1 Processor -- start 2024-04-26T21:06:38.048 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.047+0000 7f2c1f200640 1 -- start start 2024-04-26T21:06:38.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.048+0000 7f2c1f200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c200c9da0 0x7f2c20162cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:38.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.048+0000 7f2c1f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c200cb0a0 0x7f2c20072e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:38.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.048+0000 7f2c1f200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c200cb500 0x7f2c20073390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:38.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.048+0000 7f2c1f200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f2c20160080 con 0x7f2c200c9da0 2024-04-26T21:06:38.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.048+0000 7f2c1f200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c2015fda0 con 0x7f2c200cb0a0 2024-04-26T21:06:38.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.048+0000 7f2c1f200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f2c201489a0 con 0x7f2c200cb500 2024-04-26T21:06:38.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.049+0000 7f2c1de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c200c9da0 0x7f2c20162cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:38.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.049+0000 7f2c1de00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c200c9da0 0x7f2c20162cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37546/0 (socket says 172.21.15.28:37546) 2024-04-26T21:06:38.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.049+0000 7f2c1de00640 1 -- 172.21.15.28:0/4247647846 learned_addr learned my addr 172.21.15.28:0/4247647846 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:38.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.049+0000 7f2c1d400640 1 --2- 172.21.15.28:0/4247647846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c200cb0a0 0x7f2c20072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:38.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.049+0000 7f2c1de00640 1 -- 172.21.15.28:0/4247647846 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c200cb500 msgr2=0x7f2c20073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:38.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.049+0000 7f2c1e800640 1 --2- 172.21.15.28:0/4247647846 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c200cb500 0x7f2c20073390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:38.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.049+0000 7f2c1de00640 1 --2- 172.21.15.28:0/4247647846 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c200cb500 0x7f2c20073390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:38.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.049+0000 7f2c1de00640 1 -- 172.21.15.28:0/4247647846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c200cb0a0 msgr2=0x7f2c20072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:38.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.049+0000 7f2c1de00640 1 --2- 172.21.15.28:0/4247647846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c200cb0a0 0x7f2c20072e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:38.052 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.049+0000 7f2c1de00640 1 -- 172.21.15.28:0/4247647846 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c201444e0 con 0x7f2c200c9da0 2024-04-26T21:06:38.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.050+0000 7f2c1de00640 1 --2- 172.21.15.28:0/4247647846 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c200c9da0 0x7f2c20162cc0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000c430 tx=0x7f2c1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:38.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.050+0000 7f2c16000640 1 -- 172.21.15.28:0/4247647846 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c100074e0 con 0x7f2c200c9da0 2024-04-26T21:06:38.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.050+0000 7f2c16000640 1 -- 172.21.15.28:0/4247647846 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c100047e0 con 0x7f2c200c9da0 2024-04-26T21:06:38.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.050+0000 7f2c1f200640 1 -- 172.21.15.28:0/4247647846 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c201447c0 con 0x7f2c200c9da0 2024-04-26T21:06:38.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.050+0000 7f2c16000640 1 -- 172.21.15.28:0/4247647846 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10004bd0 con 0x7f2c200c9da0 2024-04-26T21:06:38.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.050+0000 7f2c1f200640 1 -- 172.21.15.28:0/4247647846 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c201609b0 con 0x7f2c200c9da0 2024-04-26T21:06:38.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.051+0000 7f2c1f200640 1 -- 172.21.15.28:0/4247647846 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c2006a360 con 0x7f2c200c9da0 2024-04-26T21:06:38.053 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.052+0000 7f2c16000640 1 -- 172.21.15.28:0/4247647846 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c10011070 con 0x7f2c200c9da0 2024-04-26T21:06:38.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.053+0000 7f2c16000640 1 --2- 172.21.15.28:0/4247647846 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2c0006c8f0 0x7f2c0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:38.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.053+0000 7f2c16000640 1 -- 172.21.15.28:0/4247647846 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f2c1008b910 con 0x7f2c200c9da0 2024-04-26T21:06:38.054 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.053+0000 7f2c1d400640 1 --2- 172.21.15.28:0/4247647846 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2c0006c8f0 0x7f2c0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:38.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.054+0000 7f2c1d400640 1 --2- 172.21.15.28:0/4247647846 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2c0006c8f0 0x7f2c0006edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c034000 tx=0x7f2c0c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:38.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.057+0000 7f2c16000640 1 -- 172.21.15.28:0/4247647846 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2c10059c60 con 0x7f2c200c9da0 2024-04-26T21:06:38.292 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.291+0000 7f2c1f200640 1 -- 172.21.15.28:0/4247647846 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2c20074550 con 0x7f2c200c9da0 2024-04-26T21:06:38.293 INFO:tasks.ceph.mon.a.smithi028.stderr:2024-04-26T21:06:38.292+0000 7f9cbd800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T21:06:38.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.295+0000 7f2c16000640 1 -- 172.21.15.28:0/4247647846 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7f2c10059560 con 0x7f2c200c9da0 2024-04-26T21:06:38.297 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T21:06:38.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.298+0000 7f2c1f200640 1 -- 172.21.15.28:0/4247647846 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2c0006c8f0 msgr2=0x7f2c0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:38.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.298+0000 7f2c1f200640 1 --2- 172.21.15.28:0/4247647846 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2c0006c8f0 0x7f2c0006edb0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2c0c034000 tx=0x7f2c0c02e000 comp rx=0 tx=0).stop 2024-04-26T21:06:38.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.298+0000 7f2c1f200640 1 -- 172.21.15.28:0/4247647846 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c200c9da0 msgr2=0x7f2c20162cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:38.299 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.298+0000 7f2c1f200640 1 --2- 172.21.15.28:0/4247647846 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c200c9da0 0x7f2c20162cc0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000c430 tx=0x7f2c1000c8f0 comp rx=0 tx=0).stop 2024-04-26T21:06:38.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.298+0000 7f2c1f200640 1 -- 172.21.15.28:0/4247647846 shutdown_connections 2024-04-26T21:06:38.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.298+0000 7f2c1f200640 1 --2- 172.21.15.28:0/4247647846 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f2c0006c8f0 0x7f2c0006edb0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:38.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.298+0000 7f2c1f200640 1 --2- 172.21.15.28:0/4247647846 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f2c200cb500 0x7f2c20073390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:38.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.298+0000 7f2c1f200640 1 --2- 172.21.15.28:0/4247647846 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c200cb0a0 0x7f2c20072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:38.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.298+0000 7f2c1f200640 1 --2- 172.21.15.28:0/4247647846 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c200c9da0 0x7f2c20162cc0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:38.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.298+0000 7f2c1f200640 1 -- 172.21.15.28:0/4247647846 >> 172.21.15.28:0/4247647846 conn(0x7f2c200d0a50 msgr2=0x7f2c200b1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:38.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.299+0000 7f2c1f200640 1 -- 172.21.15.28:0/4247647846 shutdown_connections 2024-04-26T21:06:38.301 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.299+0000 7f2c1f200640 1 -- 172.21.15.28:0/4247647846 wait complete. 2024-04-26T21:06:38.321 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T21:06:38.408 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2024-04-26T21:06:38.520 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.519+0000 7fe877200640 1 Processor -- start 2024-04-26T21:06:38.521 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.519+0000 7fe877200640 1 -- start start 2024-04-26T21:06:38.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.521+0000 7fe877200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe8780d1de0 0x7fe8780d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:38.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.521+0000 7fe877200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fe87815f360 0x7fe8780d3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:38.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.521+0000 7fe877200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8780d2700 con 0x7fe8780bac00 2024-04-26T21:06:38.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.521+0000 7fe877200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fe8780ba350 con 0x7fe8780d1de0 2024-04-26T21:06:38.522 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.521+0000 7fe877200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fe8780b9ab0 con 0x7fe87815f360 2024-04-26T21:06:38.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.521+0000 7fe875e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe8780d1de0 0x7fe8780d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:38.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.522+0000 7fe875e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe8780d1de0 0x7fe8780d21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37574/0 (socket says 172.21.15.28:37574) 2024-04-26T21:06:38.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.522+0000 7fe875e00640 1 -- 172.21.15.28:0/3236085576 learned_addr learned my addr 172.21.15.28:0/3236085576 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:38.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.522+0000 7fe875400640 1 --2- 172.21.15.28:0/3236085576 >> v2:172.21.15.103:3300/0 conn(0x7fe87815f360 0x7fe8780d3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:38.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.522+0000 7fe875e00640 1 -- 172.21.15.28:0/3236085576 >> v2:172.21.15.103:3300/0 conn(0x7fe87815f360 msgr2=0x7fe8780d3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:38.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.522+0000 7fe875e00640 1 --2- 172.21.15.28:0/3236085576 >> v2:172.21.15.103:3300/0 conn(0x7fe87815f360 0x7fe8780d3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:38.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.522+0000 7fe875e00640 1 -- 172.21.15.28:0/3236085576 >> v1:172.21.15.28:6789/0 conn(0x7fe8780bac00 legacy=0x7fe8780d39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:38.524 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.522+0000 7fe875e00640 1 -- 172.21.15.28:0/3236085576 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8780b9d90 con 0x7fe8780d1de0 2024-04-26T21:06:38.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.523+0000 7fe875e00640 1 --2- 172.21.15.28:0/3236085576 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8780d1de0 0x7fe8780d21c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe8780d0c20 tx=0x7fe86802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbcc9d969dd0480 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:38.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.523+0000 7fe874a00640 1 -- 172.21.15.28:0/3236085576 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868037070 con 0x7fe8780d1de0 2024-04-26T21:06:38.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.523+0000 7fe874a00640 1 -- 172.21.15.28:0/3236085576 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe868032440 con 0x7fe8780d1de0 2024-04-26T21:06:38.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.523+0000 7fe874a00640 1 -- 172.21.15.28:0/3236085576 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868032810 con 0x7fe8780d1de0 2024-04-26T21:06:38.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.524+0000 7fe877200640 1 -- 172.21.15.28:0/3236085576 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8780d1de0 msgr2=0x7fe8780d21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:38.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.524+0000 7fe877200640 1 --2- 172.21.15.28:0/3236085576 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8780d1de0 0x7fe8780d21c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe8780d0c20 tx=0x7fe86802a270 comp rx=0 tx=0).stop 2024-04-26T21:06:38.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.524+0000 7fe877200640 1 -- 172.21.15.28:0/3236085576 shutdown_connections 2024-04-26T21:06:38.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.524+0000 7fe877200640 1 --2- 172.21.15.28:0/3236085576 >> v2:172.21.15.103:3300/0 conn(0x7fe87815f360 0x7fe8780d3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:38.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.524+0000 7fe877200640 1 --2- 172.21.15.28:0/3236085576 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8780d1de0 0x7fe8780d21c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:38.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.524+0000 7fe877200640 1 -- 172.21.15.28:0/3236085576 >> 172.21.15.28:0/3236085576 conn(0x7fe87806c770 msgr2=0x7fe87806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:38.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.524+0000 7fe877200640 1 -- 172.21.15.28:0/3236085576 shutdown_connections 2024-04-26T21:06:38.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.524+0000 7fe877200640 1 -- 172.21.15.28:0/3236085576 wait complete. 2024-04-26T21:06:38.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.526+0000 7fe877200640 1 Processor -- start 2024-04-26T21:06:38.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.526+0000 7fe877200640 1 -- start start 2024-04-26T21:06:38.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.527+0000 7fe877200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8780bac00 0x7fe878155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:38.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.527+0000 7fe877200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8780d1de0 0x7fe878156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:38.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.527+0000 7fe877200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe87815f360 0x7fe8781574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:38.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.527+0000 7fe877200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fe8780d40e0 con 0x7fe8780d1de0 2024-04-26T21:06:38.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.527+0000 7fe877200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe8780d2f70 con 0x7fe8780bac00 2024-04-26T21:06:38.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.527+0000 7fe877200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fe8780d4260 con 0x7fe87815f360 2024-04-26T21:06:38.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.527+0000 7fe875400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8780d1de0 0x7fe878156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:38.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.527+0000 7fe875400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8780d1de0 0x7fe878156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37588/0 (socket says 172.21.15.28:37588) 2024-04-26T21:06:38.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.527+0000 7fe875400640 1 -- 172.21.15.28:0/2752555211 learned_addr learned my addr 172.21.15.28:0/2752555211 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:38.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.528+0000 7fe876800640 1 --2- 172.21.15.28:0/2752555211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe87815f360 0x7fe8781574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:38.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.528+0000 7fe875400640 1 -- 172.21.15.28:0/2752555211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe87815f360 msgr2=0x7fe8781574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:38.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.528+0000 7fe875400640 1 --2- 172.21.15.28:0/2752555211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe87815f360 0x7fe8781574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:38.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.528+0000 7fe875400640 1 -- 172.21.15.28:0/2752555211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8780bac00 msgr2=0x7fe878155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:38.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.528+0000 7fe875400640 1 --2- 172.21.15.28:0/2752555211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8780bac00 0x7fe878155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:38.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.528+0000 7fe875400640 1 -- 172.21.15.28:0/2752555211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe878157de0 con 0x7fe8780d1de0 2024-04-26T21:06:38.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.528+0000 7fe876800640 1 --2- 172.21.15.28:0/2752555211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe87815f360 0x7fe8781574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:38.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.528+0000 7fe875400640 1 --2- 172.21.15.28:0/2752555211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8780d1de0 0x7fe878156f80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe86400d620 tx=0x7fe86400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:38.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.528+0000 7fe86e000640 1 -- 172.21.15.28:0/2752555211 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86400b620 con 0x7fe8780d1de0 2024-04-26T21:06:38.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.528+0000 7fe86e000640 1 -- 172.21.15.28:0/2752555211 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe864019070 con 0x7fe8780d1de0 2024-04-26T21:06:38.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.528+0000 7fe877200640 1 -- 172.21.15.28:0/2752555211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8781580c0 con 0x7fe8780d1de0 2024-04-26T21:06:38.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.528+0000 7fe86e000640 1 -- 172.21.15.28:0/2752555211 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe86400ba00 con 0x7fe8780d1de0 2024-04-26T21:06:38.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.528+0000 7fe877200640 1 -- 172.21.15.28:0/2752555211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe87815ff70 con 0x7fe8780d1de0 2024-04-26T21:06:38.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.529+0000 7fe877200640 1 -- 172.21.15.28:0/2752555211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe878158290 con 0x7fe8780d1de0 2024-04-26T21:06:38.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.530+0000 7fe86e000640 1 -- 172.21.15.28:0/2752555211 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe86400bba0 con 0x7fe8780d1de0 2024-04-26T21:06:38.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.532+0000 7fe86e000640 1 --2- 172.21.15.28:0/2752555211 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe85806c8f0 0x7fe85806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:38.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.532+0000 7fe86e000640 1 -- 172.21.15.28:0/2752555211 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5853+0+0 (secure 0 0 0) 0x7fe86408bff0 con 0x7fe8780d1de0 2024-04-26T21:06:38.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.532+0000 7fe875e00640 1 --2- 172.21.15.28:0/2752555211 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe85806c8f0 0x7fe85806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:38.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.533+0000 7fe875e00640 1 --2- 172.21.15.28:0/2752555211 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe85806c8f0 0x7fe85806edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe868034000 tx=0x7fe86802c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:38.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.536+0000 7fe86e000640 1 -- 172.21.15.28:0/2752555211 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe86405a330 con 0x7fe8780d1de0 2024-04-26T21:06:38.784 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:38.783+0000 7fe877200640 1 -- 172.21.15.28:0/2752555211 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe8780d24f0 con 0x7fe8780d1de0 2024-04-26T21:06:39.300 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.299+0000 7fe86e000640 1 -- 172.21.15.28:0/2752555211 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7fe8780d24f0 con 0x7fe8780d1de0 2024-04-26T21:06:39.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.302+0000 7fe877200640 1 -- 172.21.15.28:0/2752555211 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe85806c8f0 msgr2=0x7fe85806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:39.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.302+0000 7fe877200640 1 --2- 172.21.15.28:0/2752555211 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe85806c8f0 0x7fe85806edb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe868034000 tx=0x7fe86802c000 comp rx=0 tx=0).stop 2024-04-26T21:06:39.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.302+0000 7fe877200640 1 -- 172.21.15.28:0/2752555211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8780d1de0 msgr2=0x7fe878156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:39.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.302+0000 7fe877200640 1 --2- 172.21.15.28:0/2752555211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8780d1de0 0x7fe878156f80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fe86400d620 tx=0x7fe86400dae0 comp rx=0 tx=0).stop 2024-04-26T21:06:39.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.303+0000 7fe877200640 1 -- 172.21.15.28:0/2752555211 shutdown_connections 2024-04-26T21:06:39.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.303+0000 7fe877200640 1 --2- 172.21.15.28:0/2752555211 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fe85806c8f0 0x7fe85806edb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:39.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.303+0000 7fe877200640 1 --2- 172.21.15.28:0/2752555211 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fe87815f360 0x7fe8781574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:39.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.303+0000 7fe877200640 1 --2- 172.21.15.28:0/2752555211 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe8780d1de0 0x7fe878156f80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:39.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.303+0000 7fe877200640 1 --2- 172.21.15.28:0/2752555211 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe8780bac00 0x7fe878155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:39.304 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.303+0000 7fe877200640 1 -- 172.21.15.28:0/2752555211 >> 172.21.15.28:0/2752555211 conn(0x7fe87806c770 msgr2=0x7fe8780d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:39.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.303+0000 7fe877200640 1 -- 172.21.15.28:0/2752555211 shutdown_connections 2024-04-26T21:06:39.305 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.303+0000 7fe877200640 1 -- 172.21.15.28:0/2752555211 wait complete. 2024-04-26T21:06:39.329 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T21:06:39.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.542+0000 7f5285800640 1 Processor -- start 2024-04-26T21:06:39.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.542+0000 7f5285800640 1 -- start start 2024-04-26T21:06:39.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.543+0000 7f5285800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f52800b6100 0x7f52800b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:39.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.543+0000 7f5285800640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f5280071fb0 0x7f52800ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:39.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.543+0000 7f5285800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52800b6a20 con 0x7f52800ceca0 2024-04-26T21:06:39.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.543+0000 7f5285800640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f5280073630 con 0x7f5280071fb0 2024-04-26T21:06:39.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.543+0000 7f5285800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f5280073910 con 0x7f52800b6100 2024-04-26T21:06:39.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.544+0000 7f527f400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f52800ceca0 0x7f5280160c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:45180/0 (socket says 172.21.15.28:45180) 2024-04-26T21:06:39.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.544+0000 7f527f400640 1 -- 172.21.15.28:0/1687019716 learned_addr learned my addr 172.21.15.28:0/1687019716 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:39.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.544+0000 7f527e000640 1 --2- 172.21.15.28:0/1687019716 >> v2:172.21.15.103:3300/0 conn(0x7f5280071fb0 0x7f52800ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:39.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.544+0000 7f527d600640 1 -- 172.21.15.28:0/1687019716 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1631617289 0 0) 0x7f52800b6a20 con 0x7f52800ceca0 2024-04-26T21:06:39.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.544+0000 7f527ea00640 1 --2- 172.21.15.28:0/1687019716 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f52800b6100 0x7f52800b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:39.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.544+0000 7f527d600640 1 -- 172.21.15.28:0/1687019716 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5264003520 con 0x7f52800ceca0 2024-04-26T21:06:39.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.545+0000 7f527d600640 1 -- 172.21.15.28:0/1687019716 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1456363844 0 0) 0x7f5264003520 con 0x7f52800ceca0 2024-04-26T21:06:39.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.545+0000 7f527d600640 1 -- 172.21.15.28:0/1687019716 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f52800b6100 msgr2=0x7f52800b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:39.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.545+0000 7f527d600640 1 --2- 172.21.15.28:0/1687019716 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f52800b6100 0x7f52800b64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:39.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.545+0000 7f527d600640 1 -- 172.21.15.28:0/1687019716 >> v2:172.21.15.103:3300/0 conn(0x7f5280071fb0 msgr2=0x7f52800ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:39.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.545+0000 7f527d600640 1 --2- 172.21.15.28:0/1687019716 >> v2:172.21.15.103:3300/0 conn(0x7f5280071fb0 0x7f52800ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:39.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.545+0000 7f527d600640 1 -- 172.21.15.28:0/1687019716 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5280073bf0 con 0x7f52800ceca0 2024-04-26T21:06:39.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.545+0000 7f527d600640 1 -- 172.21.15.28:0/1687019716 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f5278001670 con 0x7f52800ceca0 2024-04-26T21:06:39.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.545+0000 7f527d600640 1 -- 172.21.15.28:0/1687019716 >> v1:172.21.15.28:6789/0 conn(0x7f52800ceca0 legacy=0x7f5280160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:39.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.545+0000 7f527d600640 1 --2- 172.21.15.28:0/1687019716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52640039b0 0x7f5264003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:39.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.545+0000 7f527d600640 1 --2- 172.21.15.28:0/1687019716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5264004800 0x7f5264004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:39.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.545+0000 7f527d600640 1 --2- 172.21.15.28:0/1687019716 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f52800b6100 0x7f52640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:39.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.545+0000 7f527d600640 1 -- 172.21.15.28:0/1687019716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f5280073630 con 0x7f52640039b0 2024-04-26T21:06:39.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.545+0000 7f527d600640 1 -- 172.21.15.28:0/1687019716 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5280073910 con 0x7f5264004800 2024-04-26T21:06:39.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.545+0000 7f527d600640 1 -- 172.21.15.28:0/1687019716 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f526400b9c0 con 0x7f52800b6100 2024-04-26T21:06:39.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.545+0000 7f527ea00640 1 --2- 172.21.15.28:0/1687019716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52640039b0 0x7f5264003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:39.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.546+0000 7f527ea00640 1 -- 172.21.15.28:0/1687019716 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f52800b6100 msgr2=0x7f52640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:39.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.546+0000 7f527ea00640 1 --2- 172.21.15.28:0/1687019716 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f52800b6100 0x7f52640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:39.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.546+0000 7f527ea00640 1 -- 172.21.15.28:0/1687019716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5264004800 msgr2=0x7f5264004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:39.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.546+0000 7f527e000640 1 --2- 172.21.15.28:0/1687019716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5264004800 0x7f5264004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:39.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.546+0000 7f527ea00640 1 --2- 172.21.15.28:0/1687019716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5264004800 0x7f5264004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:39.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.546+0000 7f527ea00640 1 -- 172.21.15.28:0/1687019716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5264003520 con 0x7f52640039b0 2024-04-26T21:06:39.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.546+0000 7f527e000640 1 --2- 172.21.15.28:0/1687019716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5264004800 0x7f5264004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:39.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.547+0000 7f527ea00640 1 --2- 172.21.15.28:0/1687019716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52640039b0 0x7f5264003dd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5270009fc0 tx=0x7f5264005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9e0ab1032b61ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:39.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.547+0000 7f527d600640 1 -- 172.21.15.28:0/1687019716 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270009c10 con 0x7f52640039b0 2024-04-26T21:06:39.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.547+0000 7f527d600640 1 -- 172.21.15.28:0/1687019716 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5270036070 con 0x7f52640039b0 2024-04-26T21:06:39.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.547+0000 7f527d600640 1 -- 172.21.15.28:0/1687019716 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527002d070 con 0x7f52640039b0 2024-04-26T21:06:39.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.548+0000 7f5285800640 1 -- 172.21.15.28:0/1687019716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52640039b0 msgr2=0x7f5264003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:39.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.548+0000 7f5285800640 1 --2- 172.21.15.28:0/1687019716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52640039b0 0x7f5264003dd0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f5270009fc0 tx=0x7f5264005830 comp rx=0 tx=0).stop 2024-04-26T21:06:39.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.548+0000 7f527f400640 1 -- 172.21.15.28:0/1687019716 reap_dead start 2024-04-26T21:06:39.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.548+0000 7f5285800640 1 -- 172.21.15.28:0/1687019716 shutdown_connections 2024-04-26T21:06:39.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.548+0000 7f5285800640 1 -- 172.21.15.28:0/1687019716 >> 172.21.15.28:0/1687019716 conn(0x7f528006e220 msgr2=0x7f528006e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:39.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.548+0000 7f5285800640 1 -- 172.21.15.28:0/1687019716 shutdown_connections 2024-04-26T21:06:39.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.548+0000 7f5285800640 1 -- 172.21.15.28:0/1687019716 wait complete. 2024-04-26T21:06:39.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.550+0000 7f5285800640 1 Processor -- start 2024-04-26T21:06:39.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.550+0000 7f5285800640 1 -- start start 2024-04-26T21:06:39.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.551+0000 7f5285800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52640039b0 0x7f52801f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:39.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.551+0000 7f5285800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5280139010 0x7f5280139470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:39.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.551+0000 7f5285800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f528014f4d0 0x7f528014f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:39.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.551+0000 7f5285800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f52800cf0f0 con 0x7f528014f4d0 2024-04-26T21:06:39.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.551+0000 7f5285800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f52800741b0 con 0x7f52640039b0 2024-04-26T21:06:39.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.551+0000 7f5285800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f5280145fe0 con 0x7f5280139010 2024-04-26T21:06:39.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.551+0000 7f527f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f528014f4d0 0x7f528014f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:39.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.551+0000 7f527f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f528014f4d0 0x7f528014f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:37598/0 (socket says 172.21.15.28:37598) 2024-04-26T21:06:39.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.551+0000 7f527f400640 1 -- 172.21.15.28:0/3635137243 learned_addr learned my addr 172.21.15.28:0/3635137243 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:39.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.551+0000 7f527ea00640 1 --2- 172.21.15.28:0/3635137243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52640039b0 0x7f52801f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:39.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.551+0000 7f527e000640 1 --2- 172.21.15.28:0/3635137243 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5280139010 0x7f5280139470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:39.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.552+0000 7f527f400640 1 -- 172.21.15.28:0/3635137243 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5280139010 msgr2=0x7f5280139470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:39.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.552+0000 7f527f400640 1 --2- 172.21.15.28:0/3635137243 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5280139010 0x7f5280139470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:39.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.552+0000 7f527f400640 1 -- 172.21.15.28:0/3635137243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52640039b0 msgr2=0x7f52801f6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:39.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.552+0000 7f527f400640 1 --2- 172.21.15.28:0/3635137243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52640039b0 0x7f52801f6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:39.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.552+0000 7f527f400640 1 -- 172.21.15.28:0/3635137243 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5280150240 con 0x7f528014f4d0 2024-04-26T21:06:39.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.552+0000 7f527f400640 1 --2- 172.21.15.28:0/3635137243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f528014f4d0 0x7f528014f950 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f52800d0c30 tx=0x7f527800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:39.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.552+0000 7f5276a00640 1 -- 172.21.15.28:0/3635137243 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278007440 con 0x7f528014f4d0 2024-04-26T21:06:39.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.552+0000 7f5276a00640 1 -- 172.21.15.28:0/3635137243 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5278007600 con 0x7f528014f4d0 2024-04-26T21:06:39.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.552+0000 7f5276a00640 1 -- 172.21.15.28:0/3635137243 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278004a20 con 0x7f528014f4d0 2024-04-26T21:06:39.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.552+0000 7f5285800640 1 -- 172.21.15.28:0/3635137243 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5280150520 con 0x7f528014f4d0 2024-04-26T21:06:39.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.553+0000 7f5285800640 1 -- 172.21.15.28:0/3635137243 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5280150ae0 con 0x7f528014f4d0 2024-04-26T21:06:39.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.554+0000 7f5285800640 1 -- 172.21.15.28:0/3635137243 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5280076520 con 0x7f528014f4d0 2024-04-26T21:06:39.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.555+0000 7f5276a00640 1 -- 172.21.15.28:0/3635137243 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5278004bc0 con 0x7f528014f4d0 2024-04-26T21:06:39.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.556+0000 7f5276a00640 1 --2- 172.21.15.28:0/3635137243 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f526006c940 0x7f526006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:39.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.556+0000 7f5276a00640 1 -- 172.21.15.28:0/3635137243 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5853+0+0 (secure 0 0 0) 0x7f527808b540 con 0x7f528014f4d0 2024-04-26T21:06:39.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.556+0000 7f527ea00640 1 --2- 172.21.15.28:0/3635137243 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f526006c940 0x7f526006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:39.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.557+0000 7f527ea00640 1 --2- 172.21.15.28:0/3635137243 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f526006c940 0x7f526006ee00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5270035000 tx=0x7f527002f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:39.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.560+0000 7f5276a00640 1 -- 172.21.15.28:0/3635137243 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f52780598c0 con 0x7f528014f4d0 2024-04-26T21:06:39.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:39.770+0000 7f5285800640 1 -- 172.21.15.28:0/3635137243 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f528006ff90 con 0x7f528014f4d0 2024-04-26T21:06:40.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.310+0000 7f5276a00640 1 -- 172.21.15.28:0/3635137243 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) ==== 181+0+0 (secure 0 0 0) 0x7f52780591c0 con 0x7f528014f4d0 2024-04-26T21:06:40.312 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' removed 2024-04-26T21:06:40.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.315+0000 7f5285800640 1 -- 172.21.15.28:0/3635137243 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f526006c940 msgr2=0x7f526006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:40.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.315+0000 7f5285800640 1 --2- 172.21.15.28:0/3635137243 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f526006c940 0x7f526006ee00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5270035000 tx=0x7f527002f000 comp rx=0 tx=0).stop 2024-04-26T21:06:40.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.315+0000 7f5285800640 1 -- 172.21.15.28:0/3635137243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f528014f4d0 msgr2=0x7f528014f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:40.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.315+0000 7f5285800640 1 --2- 172.21.15.28:0/3635137243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f528014f4d0 0x7f528014f950 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f52800d0c30 tx=0x7f527800c8e0 comp rx=0 tx=0).stop 2024-04-26T21:06:40.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.315+0000 7f5285800640 1 -- 172.21.15.28:0/3635137243 shutdown_connections 2024-04-26T21:06:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.315+0000 7f5285800640 1 --2- 172.21.15.28:0/3635137243 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f526006c940 0x7f526006ee00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.315+0000 7f5285800640 1 --2- 172.21.15.28:0/3635137243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f528014f4d0 0x7f528014f950 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.315+0000 7f5285800640 1 --2- 172.21.15.28:0/3635137243 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5280139010 0x7f5280139470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.315+0000 7f5285800640 1 --2- 172.21.15.28:0/3635137243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52640039b0 0x7f52801f6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.315+0000 7f5285800640 1 -- 172.21.15.28:0/3635137243 >> 172.21.15.28:0/3635137243 conn(0x7f528006e220 msgr2=0x7f528006e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.315+0000 7f5285800640 1 -- 172.21.15.28:0/3635137243 shutdown_connections 2024-04-26T21:06:40.317 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.316+0000 7f5285800640 1 -- 172.21.15.28:0/3635137243 wait complete. 2024-04-26T21:06:40.347 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T21:06:40.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.554+0000 7f803f200640 1 Processor -- start 2024-04-26T21:06:40.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.555+0000 7f803f200640 1 -- start start 2024-04-26T21:06:40.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.556+0000 7f803f200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f80400cd500 0x7f80400af260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:40.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.556+0000 7f803f200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f804006f8b0 con 0x7f80400ceac0 2024-04-26T21:06:40.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.556+0000 7f803f200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f804006fb90 con 0x7f80400d01a0 2024-04-26T21:06:40.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.556+0000 7f803f200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f80400aea60 con 0x7f80400cd500 2024-04-26T21:06:40.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.557+0000 7f8037e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f80400ceac0 0x7f80400cd8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40176/0 (socket says 172.21.15.28:40176) 2024-04-26T21:06:40.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.557+0000 7f8037e00640 1 -- 172.21.15.28:0/3780251367 learned_addr learned my addr 172.21.15.28:0/3780251367 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:40.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.557+0000 7f803de00640 1 --2- 172.21.15.28:0/3780251367 >> v2:172.21.15.28:3300/0 conn(0x7f80400cd500 0x7f80400af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:40.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.557+0000 7f803d400640 1 -- 172.21.15.28:0/3780251367 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1841365170 0 0) 0x7f804006f8b0 con 0x7f80400ceac0 2024-04-26T21:06:40.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.557+0000 7f803d400640 1 -- 172.21.15.28:0/3780251367 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8024003520 con 0x7f80400ceac0 2024-04-26T21:06:40.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.557+0000 7f803de00640 1 -- 172.21.15.28:0/3780251367 >> v1:172.21.15.103:6789/0 conn(0x7f80400d01a0 legacy=0x7f8040162b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:40.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.557+0000 7f803de00640 1 -- 172.21.15.28:0/3780251367 >> v1:172.21.15.28:6789/0 conn(0x7f80400ceac0 legacy=0x7f80400cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:40.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.557+0000 7f803de00640 1 -- 172.21.15.28:0/3780251367 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80400aed40 con 0x7f80400cd500 2024-04-26T21:06:40.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.559+0000 7f803de00640 1 --2- 172.21.15.28:0/3780251367 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80400cd500 0x7f80400af260 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f80300097a0 tx=0x7f803002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05d16c33d91b32c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:40.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.559+0000 7f803d400640 1 -- 172.21.15.28:0/3780251367 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030037070 con 0x7f80400cd500 2024-04-26T21:06:40.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.559+0000 7f803d400640 1 -- 172.21.15.28:0/3780251367 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8030032440 con 0x7f80400cd500 2024-04-26T21:06:40.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.559+0000 7f803d400640 1 -- 172.21.15.28:0/3780251367 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8030032810 con 0x7f80400cd500 2024-04-26T21:06:40.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.560+0000 7f803f200640 1 -- 172.21.15.28:0/3780251367 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80400cd500 msgr2=0x7f80400af260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:40.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.560+0000 7f803f200640 1 --2- 172.21.15.28:0/3780251367 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80400cd500 0x7f80400af260 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f80300097a0 tx=0x7f803002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:06:40.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.560+0000 7f803f200640 1 -- 172.21.15.28:0/3780251367 shutdown_connections 2024-04-26T21:06:40.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.560+0000 7f803f200640 1 --2- 172.21.15.28:0/3780251367 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80400cd500 0x7f80400af260 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:40.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.560+0000 7f803f200640 1 -- 172.21.15.28:0/3780251367 >> 172.21.15.28:0/3780251367 conn(0x7f80400b2820 msgr2=0x7f80400b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:40.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.560+0000 7f803f200640 1 -- 172.21.15.28:0/3780251367 shutdown_connections 2024-04-26T21:06:40.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.560+0000 7f803f200640 1 -- 172.21.15.28:0/3780251367 wait complete. 2024-04-26T21:06:40.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.562+0000 7f803f200640 1 Processor -- start 2024-04-26T21:06:40.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.562+0000 7f803f200640 1 -- start start 2024-04-26T21:06:40.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.563+0000 7f803f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80400cd500 0x7f8040147b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:40.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.563+0000 7f803f200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80400ceac0 0x7f8040144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.563+0000 7f803f200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80400d01a0 0x7f8040145130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.563+0000 7f803f200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f80400d0790 con 0x7f80400d01a0 2024-04-26T21:06:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.563+0000 7f803f200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f80400d0610 con 0x7f80400cd500 2024-04-26T21:06:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.563+0000 7f803f200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f80401632a0 con 0x7f80400ceac0 2024-04-26T21:06:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.563+0000 7f803e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80400d01a0 0x7f8040145130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.563+0000 7f803e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80400d01a0 0x7f8040145130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33976/0 (socket says 172.21.15.28:33976) 2024-04-26T21:06:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.563+0000 7f803e800640 1 -- 172.21.15.28:0/204885455 learned_addr learned my addr 172.21.15.28:0/204885455 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.564+0000 7f8037e00640 1 --2- 172.21.15.28:0/204885455 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80400ceac0 0x7f8040144bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.564+0000 7f803de00640 1 --2- 172.21.15.28:0/204885455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80400cd500 0x7f8040147b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.564+0000 7f803e800640 1 -- 172.21.15.28:0/204885455 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80400ceac0 msgr2=0x7f8040144bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.564+0000 7f803e800640 1 --2- 172.21.15.28:0/204885455 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80400ceac0 0x7f8040144bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.564+0000 7f803e800640 1 -- 172.21.15.28:0/204885455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80400cd500 msgr2=0x7f8040147b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.564+0000 7f803e800640 1 --2- 172.21.15.28:0/204885455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80400cd500 0x7f8040147b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.564+0000 7f803e800640 1 -- 172.21.15.28:0/204885455 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8040147610 con 0x7f80400d01a0 2024-04-26T21:06:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.564+0000 7f803de00640 1 --2- 172.21.15.28:0/204885455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80400cd500 0x7f8040147b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.564+0000 7f803e800640 1 --2- 172.21.15.28:0/204885455 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80400d01a0 0x7f8040145130 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f803800c6e0 tx=0x7f803800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.564+0000 7f8036000640 1 -- 172.21.15.28:0/204885455 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038003b70 con 0x7f80400d01a0 2024-04-26T21:06:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.564+0000 7f8036000640 1 -- 172.21.15.28:0/204885455 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8038003d30 con 0x7f80400d01a0 2024-04-26T21:06:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.564+0000 7f8036000640 1 -- 172.21.15.28:0/204885455 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8038005310 con 0x7f80400d01a0 2024-04-26T21:06:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.564+0000 7f803f200640 1 -- 172.21.15.28:0/204885455 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8040146570 con 0x7f80400d01a0 2024-04-26T21:06:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.565+0000 7f803f200640 1 -- 172.21.15.28:0/204885455 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8040146b30 con 0x7f80400d01a0 2024-04-26T21:06:40.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.566+0000 7f803f200640 1 -- 172.21.15.28:0/204885455 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f804006a3d0 con 0x7f80400d01a0 2024-04-26T21:06:40.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.567+0000 7f8036000640 1 -- 172.21.15.28:0/204885455 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f80380054b0 con 0x7f80400d01a0 2024-04-26T21:06:40.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.568+0000 7f8036000640 1 --2- 172.21.15.28:0/204885455 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f801c06c940 0x7f801c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:40.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.568+0000 7f8036000640 1 -- 172.21.15.28:0/204885455 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5415+0+0 (secure 0 0 0) 0x7f803808b7b0 con 0x7f80400d01a0 2024-04-26T21:06:40.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.568+0000 7f803de00640 1 --2- 172.21.15.28:0/204885455 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f801c06c940 0x7f801c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:40.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.569+0000 7f803de00640 1 --2- 172.21.15.28:0/204885455 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f801c06c940 0x7f801c06ee00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8030034000 tx=0x7f803002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:40.573 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.572+0000 7f8036000640 1 -- 172.21.15.28:0/204885455 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8038059de0 con 0x7f80400d01a0 2024-04-26T21:06:40.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:40.777+0000 7f803f200640 1 -- 172.21.15.28:0/204885455 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8040072d40 con 0x7f80400d01a0 2024-04-26T21:06:41.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.314+0000 7f8036000640 1 -- 172.21.15.28:0/204885455 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) ==== 169+0+0 (secure 0 0 0) 0x7f80380596e0 con 0x7f80400d01a0 2024-04-26T21:06:41.316 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' removed 2024-04-26T21:06:41.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.320+0000 7f803f200640 1 -- 172.21.15.28:0/204885455 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f801c06c940 msgr2=0x7f801c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:41.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.320+0000 7f803f200640 1 --2- 172.21.15.28:0/204885455 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f801c06c940 0x7f801c06ee00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8030034000 tx=0x7f803002e000 comp rx=0 tx=0).stop 2024-04-26T21:06:41.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.320+0000 7f803f200640 1 -- 172.21.15.28:0/204885455 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80400d01a0 msgr2=0x7f8040145130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:41.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.320+0000 7f803f200640 1 --2- 172.21.15.28:0/204885455 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80400d01a0 0x7f8040145130 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f803800c6e0 tx=0x7f803800cba0 comp rx=0 tx=0).stop 2024-04-26T21:06:41.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.320+0000 7f803f200640 1 -- 172.21.15.28:0/204885455 shutdown_connections 2024-04-26T21:06:41.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.320+0000 7f803f200640 1 --2- 172.21.15.28:0/204885455 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f801c06c940 0x7f801c06ee00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:41.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.320+0000 7f803f200640 1 --2- 172.21.15.28:0/204885455 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80400d01a0 0x7f8040145130 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:41.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.320+0000 7f803f200640 1 --2- 172.21.15.28:0/204885455 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f80400ceac0 0x7f8040144bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:41.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.320+0000 7f803f200640 1 --2- 172.21.15.28:0/204885455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f80400cd500 0x7f8040147b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:41.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.320+0000 7f803f200640 1 -- 172.21.15.28:0/204885455 >> 172.21.15.28:0/204885455 conn(0x7f80400b2820 msgr2=0x7f804015d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:41.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.322+0000 7f803f200640 1 -- 172.21.15.28:0/204885455 shutdown_connections 2024-04-26T21:06:41.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.323+0000 7f803f200640 1 -- 172.21.15.28:0/204885455 wait complete. 2024-04-26T21:06:41.352 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T21:06:41.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.551+0000 7f7ef3c00640 1 Processor -- start 2024-04-26T21:06:41.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.551+0000 7f7ef3c00640 1 -- start start 2024-04-26T21:06:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.553+0000 7f7ef3c00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f7eec146860 0x7f7eec06f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.553+0000 7f7ef3c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eec0b4d90 con 0x7f7eec06f810 2024-04-26T21:06:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.553+0000 7f7ef3c00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7eec15d850 con 0x7f7eec0b4730 2024-04-26T21:06:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.553+0000 7f7ef3c00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f7eec06fc70 con 0x7f7eec146860 2024-04-26T21:06:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.553+0000 7f7eebe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7eec06f810 0x7f7eec0b4020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40184/0 (socket says 172.21.15.28:40184) 2024-04-26T21:06:41.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.553+0000 7f7eebe00640 1 -- 172.21.15.28:0/2463556312 learned_addr learned my addr 172.21.15.28:0/2463556312 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:41.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.553+0000 7f7eeb400640 1 -- 172.21.15.28:0/2463556312 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1445465146 0 0) 0x7f7eec0b4d90 con 0x7f7eec06f810 2024-04-26T21:06:41.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.554+0000 7f7eeb400640 1 -- 172.21.15.28:0/2463556312 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ed4003520 con 0x7f7eec06f810 2024-04-26T21:06:41.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.554+0000 7f7ef1200640 1 --2- 172.21.15.28:0/2463556312 >> v2:172.21.15.103:3300/0 conn(0x7f7eec146860 0x7f7eec06f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:41.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.554+0000 7f7eeb400640 1 -- 172.21.15.28:0/2463556312 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 676913810 0 0) 0x7f7ed4003520 con 0x7f7eec06f810 2024-04-26T21:06:41.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.554+0000 7f7eeb400640 1 -- 172.21.15.28:0/2463556312 >> v2:172.21.15.103:3300/0 conn(0x7f7eec146860 msgr2=0x7f7eec06f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.554+0000 7f7eeb400640 1 --2- 172.21.15.28:0/2463556312 >> v2:172.21.15.103:3300/0 conn(0x7f7eec146860 0x7f7eec06f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.554+0000 7f7eeb400640 1 -- 172.21.15.28:0/2463556312 >> v1:172.21.15.103:6789/0 conn(0x7f7eec0b4730 legacy=0x7f7eec15e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.554+0000 7f7eeb400640 1 -- 172.21.15.28:0/2463556312 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eec15db30 con 0x7f7eec06f810 2024-04-26T21:06:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.554+0000 7f7eeb400640 1 -- 172.21.15.28:0/2463556312 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f7ed8001670 con 0x7f7eec06f810 2024-04-26T21:06:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.554+0000 7f7eeb400640 1 -- 172.21.15.28:0/2463556312 >> v1:172.21.15.28:6789/0 conn(0x7f7eec06f810 legacy=0x7f7eec0b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.554+0000 7f7eeb400640 1 --2- 172.21.15.28:0/2463556312 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ed4003a00 0x7f7ed4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.554+0000 7f7eeb400640 1 --2- 172.21.15.28:0/2463556312 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ed4004800 0x7f7ed4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.554+0000 7f7eeb400640 1 --2- 172.21.15.28:0/2463556312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed4005210 0x7f7ed400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.554+0000 7f7eeb400640 1 -- 172.21.15.28:0/2463556312 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed400c130 con 0x7f7ed4004800 2024-04-26T21:06:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.554+0000 7f7eeb400640 1 -- 172.21.15.28:0/2463556312 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed400c2e0 con 0x7f7ed4005210 2024-04-26T21:06:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.554+0000 7f7eeb400640 1 -- 172.21.15.28:0/2463556312 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed400c4e0 con 0x7f7ed4003a00 2024-04-26T21:06:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.554+0000 7f7eebe00640 1 --2- 172.21.15.28:0/2463556312 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ed4004800 0x7f7ed4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.555+0000 7f7ef1c00640 1 --2- 172.21.15.28:0/2463556312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed4005210 0x7f7ed400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.555+0000 7f7ef1200640 1 --2- 172.21.15.28:0/2463556312 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ed4003a00 0x7f7ed4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:41.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.555+0000 7f7eebe00640 1 -- 172.21.15.28:0/2463556312 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ed4003a00 msgr2=0x7f7ed4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:41.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.555+0000 7f7eebe00640 1 --2- 172.21.15.28:0/2463556312 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ed4003a00 0x7f7ed4003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:41.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.555+0000 7f7eebe00640 1 -- 172.21.15.28:0/2463556312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed4005210 msgr2=0x7f7ed400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:41.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.555+0000 7f7eebe00640 1 --2- 172.21.15.28:0/2463556312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed4005210 0x7f7ed400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:41.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.555+0000 7f7eebe00640 1 -- 172.21.15.28:0/2463556312 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed4003520 con 0x7f7ed4004800 2024-04-26T21:06:41.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.555+0000 7f7ef1c00640 1 --2- 172.21.15.28:0/2463556312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed4005210 0x7f7ed400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:41.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.555+0000 7f7ef1c00640 1 -- 172.21.15.28:0/2463556312 reap_dead start 2024-04-26T21:06:41.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.556+0000 7f7eebe00640 1 --2- 172.21.15.28:0/2463556312 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ed4004800 0x7f7ed4004c80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800c0e0 tx=0x7f7ed4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1408b7704099c87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:41.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.556+0000 7f7eeb400640 1 -- 172.21.15.28:0/2463556312 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8007c20 con 0x7f7ed4004800 2024-04-26T21:06:41.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.556+0000 7f7eeb400640 1 -- 172.21.15.28:0/2463556312 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed8005af0 con 0x7f7ed4004800 2024-04-26T21:06:41.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.556+0000 7f7eeb400640 1 -- 172.21.15.28:0/2463556312 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8007070 con 0x7f7ed4004800 2024-04-26T21:06:41.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.557+0000 7f7ef3c00640 1 -- 172.21.15.28:0/2463556312 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ed4004800 msgr2=0x7f7ed4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:41.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.557+0000 7f7ef3c00640 1 --2- 172.21.15.28:0/2463556312 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ed4004800 0x7f7ed4004c80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800c0e0 tx=0x7f7ed4005830 comp rx=0 tx=0).stop 2024-04-26T21:06:41.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.557+0000 7f7ef3c00640 1 -- 172.21.15.28:0/2463556312 shutdown_connections 2024-04-26T21:06:41.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.557+0000 7f7ef3c00640 1 --2- 172.21.15.28:0/2463556312 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ed4004800 0x7f7ed4004c80 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:41.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.557+0000 7f7ef3c00640 1 -- 172.21.15.28:0/2463556312 >> 172.21.15.28:0/2463556312 conn(0x7f7eec0b2b60 msgr2=0x7f7eec0b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:41.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.557+0000 7f7ef3c00640 1 -- 172.21.15.28:0/2463556312 shutdown_connections 2024-04-26T21:06:41.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.557+0000 7f7ef3c00640 1 -- 172.21.15.28:0/2463556312 wait complete. 2024-04-26T21:06:41.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.559+0000 7f7ef3c00640 1 Processor -- start 2024-04-26T21:06:41.560 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.559+0000 7f7ef3c00640 1 -- start start 2024-04-26T21:06:41.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.560+0000 7f7ef3c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ed4004800 0x7f7eec142240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:41.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.560+0000 7f7ef3c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eec0ca460 0x7f7eec073310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:41.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.560+0000 7f7ef3c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eec0cba00 0x7f7eec0cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:41.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.560+0000 7f7ef3c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec06f450 con 0x7f7eec0ca460 2024-04-26T21:06:41.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.560+0000 7f7ef3c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7eec06f2d0 con 0x7f7eec0cba00 2024-04-26T21:06:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.560+0000 7f7ef3c00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f7eec06f5d0 con 0x7f7ed4004800 2024-04-26T21:06:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.560+0000 7f7eebe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eec0ca460 0x7f7eec073310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.560+0000 7f7eebe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eec0ca460 0x7f7eec073310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:33996/0 (socket says 172.21.15.28:33996) 2024-04-26T21:06:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.560+0000 7f7eebe00640 1 -- 172.21.15.28:0/1208408942 learned_addr learned my addr 172.21.15.28:0/1208408942 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.561+0000 7f7ef1200640 1 --2- 172.21.15.28:0/1208408942 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ed4004800 0x7f7eec142240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.561+0000 7f7ef1c00640 1 --2- 172.21.15.28:0/1208408942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eec0cba00 0x7f7eec0cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.561+0000 7f7eebe00640 1 -- 172.21.15.28:0/1208408942 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ed4004800 msgr2=0x7f7eec142240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.561+0000 7f7eebe00640 1 --2- 172.21.15.28:0/1208408942 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ed4004800 0x7f7eec142240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.561+0000 7f7eebe00640 1 -- 172.21.15.28:0/1208408942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eec0cba00 msgr2=0x7f7eec0cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.561+0000 7f7eebe00640 1 --2- 172.21.15.28:0/1208408942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eec0cba00 0x7f7eec0cbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.561+0000 7f7eebe00640 1 -- 172.21.15.28:0/1208408942 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eec0ca960 con 0x7f7eec0ca460 2024-04-26T21:06:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.561+0000 7f7ef1c00640 1 --2- 172.21.15.28:0/1208408942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eec0cba00 0x7f7eec0cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:41.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.561+0000 7f7eebe00640 1 --2- 172.21.15.28:0/1208408942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eec0ca460 0x7f7eec073310 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800c200 tx=0x7f7eec0b4f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:41.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.562+0000 7f7ee8c00640 1 -- 172.21.15.28:0/1208408942 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8037040 con 0x7f7eec0ca460 2024-04-26T21:06:41.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.562+0000 7f7ee8c00640 1 -- 172.21.15.28:0/1208408942 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed8041070 con 0x7f7eec0ca460 2024-04-26T21:06:41.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.562+0000 7f7ee8c00640 1 -- 172.21.15.28:0/1208408942 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed80016f0 con 0x7f7eec0ca460 2024-04-26T21:06:41.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.562+0000 7f7ef3c00640 1 -- 172.21.15.28:0/1208408942 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eec0cac40 con 0x7f7eec0ca460 2024-04-26T21:06:41.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.562+0000 7f7ef3c00640 1 -- 172.21.15.28:0/1208408942 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eec0cb200 con 0x7f7eec0ca460 2024-04-26T21:06:41.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.563+0000 7f7ef3c00640 1 -- 172.21.15.28:0/1208408942 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7eec0ce320 con 0x7f7eec0ca460 2024-04-26T21:06:41.565 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.564+0000 7f7ee8c00640 1 -- 172.21.15.28:0/1208408942 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ed8004a20 con 0x7f7eec0ca460 2024-04-26T21:06:41.566 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.565+0000 7f7ee8c00640 1 --2- 172.21.15.28:0/1208408942 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7ecc06c8f0 0x7f7ecc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:41.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.566+0000 7f7ee8c00640 1 -- 172.21.15.28:0/1208408942 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f7ed80ab690 con 0x7f7eec0ca460 2024-04-26T21:06:41.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.566+0000 7f7ef1200640 1 --2- 172.21.15.28:0/1208408942 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7ecc06c8f0 0x7f7ecc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:41.567 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.567+0000 7f7ef1200640 1 --2- 172.21.15.28:0/1208408942 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7ecc06c8f0 0x7f7ecc06edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0006fd0 tx=0x7f7ee0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:41.572 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.571+0000 7f7ee8c00640 1 -- 172.21.15.28:0/1208408942 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7eec0ce320 con 0x7f7eec0ca460 2024-04-26T21:06:41.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.801+0000 7f7ef3c00640 1 -- 172.21.15.28:0/1208408942 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f7eec076520 con 0x7f7eec0ca460 2024-04-26T21:06:41.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.806+0000 7f7ee8c00640 1 -- 172.21.15.28:0/1208408942 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f7ed8079d20 con 0x7f7eec0ca460 2024-04-26T21:06:41.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.809+0000 7f7ef3c00640 1 -- 172.21.15.28:0/1208408942 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7ecc06c8f0 msgr2=0x7f7ecc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:41.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.809+0000 7f7ef3c00640 1 --2- 172.21.15.28:0/1208408942 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7ecc06c8f0 0x7f7ecc06edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0006fd0 tx=0x7f7ee0008040 comp rx=0 tx=0).stop 2024-04-26T21:06:41.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.809+0000 7f7ef3c00640 1 -- 172.21.15.28:0/1208408942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eec0ca460 msgr2=0x7f7eec073310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:41.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.809+0000 7f7ef3c00640 1 --2- 172.21.15.28:0/1208408942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eec0ca460 0x7f7eec073310 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800c200 tx=0x7f7eec0b4f60 comp rx=0 tx=0).stop 2024-04-26T21:06:41.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.809+0000 7f7ef3c00640 1 -- 172.21.15.28:0/1208408942 shutdown_connections 2024-04-26T21:06:41.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.809+0000 7f7ef3c00640 1 --2- 172.21.15.28:0/1208408942 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f7ecc06c8f0 0x7f7ecc06edb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:41.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.809+0000 7f7ef3c00640 1 --2- 172.21.15.28:0/1208408942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7eec0cba00 0x7f7eec0cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:41.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.809+0000 7f7ef3c00640 1 --2- 172.21.15.28:0/1208408942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7eec0ca460 0x7f7eec073310 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:41.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.809+0000 7f7ef3c00640 1 --2- 172.21.15.28:0/1208408942 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f7ed4004800 0x7f7eec142240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:41.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.809+0000 7f7ef3c00640 1 -- 172.21.15.28:0/1208408942 >> 172.21.15.28:0/1208408942 conn(0x7f7eec0b2b60 msgr2=0x7f7eec1449a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:41.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.809+0000 7f7ef3c00640 1 -- 172.21.15.28:0/1208408942 shutdown_connections 2024-04-26T21:06:41.811 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:41.809+0000 7f7ef3c00640 1 -- 172.21.15.28:0/1208408942 wait complete. 2024-04-26T21:06:41.829 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_dump_tree.TestDumpTree.test_random' 2024-04-26T21:06:42.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.019+0000 7f9c8aa00640 1 Processor -- start 2024-04-26T21:06:42.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.019+0000 7f9c8aa00640 1 -- start start 2024-04-26T21:06:42.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.021+0000 7f9c8aa00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9c840bac60 0x7f9c840cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:42.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.021+0000 7f9c8aa00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c840b8240 con 0x7f9c840d03b0 2024-04-26T21:06:42.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.021+0000 7f9c8aa00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c84163760 con 0x7f9c8415f490 2024-04-26T21:06:42.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.021+0000 7f9c8aa00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9c840bb0c0 con 0x7f9c840bac60 2024-04-26T21:06:42.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.021+0000 7f9c88a00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f9c840d03b0 0x7f9c840d3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40198/0 (socket says 172.21.15.28:40198) 2024-04-26T21:06:42.022 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.021+0000 7f9c88a00640 1 -- 172.21.15.28:0/849064590 learned_addr learned my addr 172.21.15.28:0/849064590 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:42.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.022+0000 7f9c83400640 1 --2- 172.21.15.28:0/849064590 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9c840bac60 0x7f9c840cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:42.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.022+0000 7f9c82a00640 1 -- 172.21.15.28:0/849064590 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4239716940 0 0) 0x7f9c7c001110 con 0x7f9c840d03b0 2024-04-26T21:06:42.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.022+0000 7f9c82a00640 1 -- 172.21.15.28:0/849064590 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c68003520 con 0x7f9c840d03b0 2024-04-26T21:06:42.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.022+0000 7f9c82a00640 1 -- 172.21.15.28:0/849064590 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3783864779 0 0) 0x7f9c68003520 con 0x7f9c840d03b0 2024-04-26T21:06:42.024 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.022+0000 7f9c82a00640 1 -- 172.21.15.28:0/849064590 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9c840bac60 msgr2=0x7f9c840cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.022+0000 7f9c82a00640 1 --2- 172.21.15.28:0/849064590 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9c840bac60 0x7f9c840cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.022+0000 7f9c82a00640 1 -- 172.21.15.28:0/849064590 >> v1:172.21.15.103:6789/0 conn(0x7f9c8415f490 legacy=0x7f9c84162f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.022+0000 7f9c82a00640 1 -- 172.21.15.28:0/849064590 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c84163a40 con 0x7f9c840d03b0 2024-04-26T21:06:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.022+0000 7f9c82a00640 1 -- 172.21.15.28:0/849064590 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f9c7c001840 con 0x7f9c840d03b0 2024-04-26T21:06:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.022+0000 7f9c82a00640 1 -- 172.21.15.28:0/849064590 >> v1:172.21.15.28:6789/0 conn(0x7f9c840d03b0 legacy=0x7f9c840d3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.022+0000 7f9c82a00640 1 --2- 172.21.15.28:0/849064590 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9c840bac60 0x7f9c680039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.022+0000 7f9c82a00640 1 --2- 172.21.15.28:0/849064590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c68003fb0 0x7f9c68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.022+0000 7f9c82a00640 1 --2- 172.21.15.28:0/849064590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c68004d40 0x7f9c680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.022+0000 7f9c82a00640 1 -- 172.21.15.28:0/849064590 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9c840bb0c0 con 0x7f9c68004d40 2024-04-26T21:06:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.022+0000 7f9c82a00640 1 -- 172.21.15.28:0/849064590 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6800bb00 con 0x7f9c68003fb0 2024-04-26T21:06:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.022+0000 7f9c82a00640 1 -- 172.21.15.28:0/849064590 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9c6800bd00 con 0x7f9c840bac60 2024-04-26T21:06:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.023+0000 7f9c88a00640 1 --2- 172.21.15.28:0/849064590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c68004d40 0x7f9c680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.023+0000 7f9c83e00640 1 --2- 172.21.15.28:0/849064590 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9c840bac60 0x7f9c680039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:42.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.023+0000 7f9c83400640 1 --2- 172.21.15.28:0/849064590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c68003fb0 0x7f9c68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:42.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.023+0000 7f9c88a00640 1 -- 172.21.15.28:0/849064590 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9c840bac60 msgr2=0x7f9c680039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:42.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.023+0000 7f9c88a00640 1 --2- 172.21.15.28:0/849064590 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9c840bac60 0x7f9c680039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:42.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.023+0000 7f9c88a00640 1 -- 172.21.15.28:0/849064590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c68003fb0 msgr2=0x7f9c68004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:42.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.023+0000 7f9c88a00640 1 --2- 172.21.15.28:0/849064590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c68003fb0 0x7f9c68004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:42.026 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.023+0000 7f9c88a00640 1 -- 172.21.15.28:0/849064590 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c68003520 con 0x7f9c68004d40 2024-04-26T21:06:42.027 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.024+0000 7f9c88a00640 1 --2- 172.21.15.28:0/849064590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c68004d40 0x7f9c680051c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c007fd0 tx=0x7f9c68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537463a82ea522c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:42.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.024+0000 7f9c82a00640 1 -- 172.21.15.28:0/849064590 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c007e30 con 0x7f9c68004d40 2024-04-26T21:06:42.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.025+0000 7f9c82a00640 1 -- 172.21.15.28:0/849064590 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7c005af0 con 0x7f9c68004d40 2024-04-26T21:06:42.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.025+0000 7f9c82a00640 1 -- 172.21.15.28:0/849064590 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c007070 con 0x7f9c68004d40 2024-04-26T21:06:42.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.025+0000 7f9c8aa00640 1 -- 172.21.15.28:0/849064590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c68004d40 msgr2=0x7f9c680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:42.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.025+0000 7f9c8aa00640 1 --2- 172.21.15.28:0/849064590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c68004d40 0x7f9c680051c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c007fd0 tx=0x7f9c68005830 comp rx=0 tx=0).stop 2024-04-26T21:06:42.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.025+0000 7f9c88a00640 1 -- 172.21.15.28:0/849064590 reap_dead start 2024-04-26T21:06:42.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.025+0000 7f9c8aa00640 1 -- 172.21.15.28:0/849064590 shutdown_connections 2024-04-26T21:06:42.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.025+0000 7f9c8aa00640 1 -- 172.21.15.28:0/849064590 >> 172.21.15.28:0/849064590 conn(0x7f9c840b6b90 msgr2=0x7f9c840b6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:42.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.025+0000 7f9c8aa00640 1 -- 172.21.15.28:0/849064590 shutdown_connections 2024-04-26T21:06:42.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.025+0000 7f9c8aa00640 1 -- 172.21.15.28:0/849064590 wait complete. 2024-04-26T21:06:42.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.027+0000 7f9c8aa00640 1 Processor -- start 2024-04-26T21:06:42.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.027+0000 7f9c8aa00640 1 -- start start 2024-04-26T21:06:42.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.028+0000 7f9c8aa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c68004d40 0x7f9c841ea010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:42.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.028+0000 7f9c8aa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c840cdd00 0x7f9c841500a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:42.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.028+0000 7f9c8aa00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9c84161df0 0x7f9c84162280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:42.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.028+0000 7f9c8aa00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9c840d3810 con 0x7f9c68004d40 2024-04-26T21:06:42.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.028+0000 7f9c8aa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8415f870 con 0x7f9c840cdd00 2024-04-26T21:06:42.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.028+0000 7f9c8aa00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9c840d34a0 con 0x7f9c84161df0 2024-04-26T21:06:42.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.029+0000 7f9c83e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c68004d40 0x7f9c841ea010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:42.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.029+0000 7f9c83e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c68004d40 0x7f9c841ea010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34006/0 (socket says 172.21.15.28:34006) 2024-04-26T21:06:42.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.029+0000 7f9c83e00640 1 -- 172.21.15.28:0/1770660025 learned_addr learned my addr 172.21.15.28:0/1770660025 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:42.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.029+0000 7f9c83400640 1 --2- 172.21.15.28:0/1770660025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c840cdd00 0x7f9c841500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:42.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.029+0000 7f9c88a00640 1 --2- 172.21.15.28:0/1770660025 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9c84161df0 0x7f9c84162280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:42.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.029+0000 7f9c83e00640 1 -- 172.21.15.28:0/1770660025 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9c84161df0 msgr2=0x7f9c84162280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:42.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.029+0000 7f9c83e00640 1 --2- 172.21.15.28:0/1770660025 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9c84161df0 0x7f9c84162280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:42.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.029+0000 7f9c83e00640 1 -- 172.21.15.28:0/1770660025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c840cdd00 msgr2=0x7f9c841500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:42.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.029+0000 7f9c83e00640 1 --2- 172.21.15.28:0/1770660025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c840cdd00 0x7f9c841500a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:42.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.029+0000 7f9c83e00640 1 -- 172.21.15.28:0/1770660025 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c840ce1e0 con 0x7f9c68004d40 2024-04-26T21:06:42.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.029+0000 7f9c83e00640 1 --2- 172.21.15.28:0/1770660025 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c68004d40 0x7f9c841ea010 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400e550 tx=0x7f9c7400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:42.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.030+0000 7f9c7be00640 1 -- 172.21.15.28:0/1770660025 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c740052e0 con 0x7f9c68004d40 2024-04-26T21:06:42.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.030+0000 7f9c7be00640 1 -- 172.21.15.28:0/1770660025 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c74018070 con 0x7f9c68004d40 2024-04-26T21:06:42.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.030+0000 7f9c8aa00640 1 -- 172.21.15.28:0/1770660025 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c840ce4c0 con 0x7f9c68004d40 2024-04-26T21:06:42.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.030+0000 7f9c7be00640 1 -- 172.21.15.28:0/1770660025 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c74003810 con 0x7f9c68004d40 2024-04-26T21:06:42.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.030+0000 7f9c8aa00640 1 -- 172.21.15.28:0/1770660025 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c840cea80 con 0x7f9c68004d40 2024-04-26T21:06:42.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.031+0000 7f9c8aa00640 1 -- 172.21.15.28:0/1770660025 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8406a400 con 0x7f9c68004d40 2024-04-26T21:06:42.033 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.032+0000 7f9c7be00640 1 -- 172.21.15.28:0/1770660025 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c740039b0 con 0x7f9c68004d40 2024-04-26T21:06:42.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.033+0000 7f9c7be00640 1 --2- 172.21.15.28:0/1770660025 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9c6006c8f0 0x7f9c6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:42.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.033+0000 7f9c7be00640 1 -- 172.21.15.28:0/1770660025 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f9c7408b590 con 0x7f9c68004d40 2024-04-26T21:06:42.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.033+0000 7f9c83400640 1 --2- 172.21.15.28:0/1770660025 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9c6006c8f0 0x7f9c6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:42.035 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.034+0000 7f9c83400640 1 --2- 172.21.15.28:0/1770660025 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9c6006c8f0 0x7f9c6006edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9c8415eba0 tx=0x7f9c6c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:42.039 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.038+0000 7f9c7be00640 1 -- 172.21.15.28:0/1770660025 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9c74059c20 con 0x7f9c68004d40 2024-04-26T21:06:42.325 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.324+0000 7f9c8aa00640 1 -- 172.21.15.28:0/1770660025 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_dump_tree.TestDumpTree.test_random"]} v 0) -- 0x7f9c840d2d70 con 0x7f9c68004d40 2024-04-26T21:06:42.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.333+0000 7f9c7be00640 1 -- 172.21.15.28:0/1770660025 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_dump_tree.TestDumpTree.test_random"]}]=0 v380) ==== 131+0+0 (secure 0 0 0) 0x7f9c74059520 con 0x7f9c68004d40 2024-04-26T21:06:42.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.335+0000 7f9c8aa00640 1 -- 172.21.15.28:0/1770660025 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9c6006c8f0 msgr2=0x7f9c6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:42.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.335+0000 7f9c8aa00640 1 --2- 172.21.15.28:0/1770660025 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9c6006c8f0 0x7f9c6006edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9c8415eba0 tx=0x7f9c6c00b040 comp rx=0 tx=0).stop 2024-04-26T21:06:42.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.335+0000 7f9c8aa00640 1 -- 172.21.15.28:0/1770660025 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c68004d40 msgr2=0x7f9c841ea010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:42.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.335+0000 7f9c8aa00640 1 --2- 172.21.15.28:0/1770660025 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c68004d40 0x7f9c841ea010 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400e550 tx=0x7f9c7400ea10 comp rx=0 tx=0).stop 2024-04-26T21:06:42.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.336+0000 7f9c8aa00640 1 -- 172.21.15.28:0/1770660025 shutdown_connections 2024-04-26T21:06:42.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.336+0000 7f9c8aa00640 1 --2- 172.21.15.28:0/1770660025 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9c6006c8f0 0x7f9c6006edb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:42.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.336+0000 7f9c8aa00640 1 --2- 172.21.15.28:0/1770660025 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9c84161df0 0x7f9c84162280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:42.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.336+0000 7f9c8aa00640 1 --2- 172.21.15.28:0/1770660025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9c840cdd00 0x7f9c841500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:42.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.336+0000 7f9c8aa00640 1 --2- 172.21.15.28:0/1770660025 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c68004d40 0x7f9c841ea010 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:42.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.336+0000 7f9c8aa00640 1 -- 172.21.15.28:0/1770660025 >> 172.21.15.28:0/1770660025 conn(0x7f9c840b6b90 msgr2=0x7f9c840d5270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:42.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.336+0000 7f9c8aa00640 1 -- 172.21.15.28:0/1770660025 shutdown_connections 2024-04-26T21:06:42.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:42.336+0000 7f9c8aa00640 1 -- 172.21.15.28:0/1770660025 wait complete. 2024-04-26T21:06:42.356 INFO:tasks.cephfs_test_runner:test_random (tasks.cephfs.test_dump_tree.TestDumpTree) ... ok 2024-04-26T21:06:42.357 INFO:tasks.cephfs_test_runner: 2024-04-26T21:06:42.357 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T21:06:42.357 INFO:tasks.cephfs_test_runner:Ran 2 tests in 553.633s 2024-04-26T21:06:42.357 INFO:tasks.cephfs_test_runner: 2024-04-26T21:06:42.357 INFO:tasks.cephfs_test_runner:OK 2024-04-26T21:06:42.357 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-26T21:06:42.367 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-26T21:06:42.385 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-26T21:06:42.386 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T21:06:42.386 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:06:42.441 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:06:42.441 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2024-04-26T21:06:42.442 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:06:42.494 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T21:06:42.494 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T21:06:42.495 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:06:42.555 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052734,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:06:42.555 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T21:06:42.555 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T21:06:42.608 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052734,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T21:06:42.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:06:42.608 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2024-04-26T21:06:42.670 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T21:06:42.670 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T21:06:42.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:06:42.671 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T21:06:42.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:06:42.765 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T21:06:42.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:06:42.840 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link delete ceph-brx 2024-04-26T21:06:42.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:06:42.968 DEBUG:teuthology.orchestra.run.smithi103:> ip netns list 2024-04-26T21:06:43.029 INFO:teuthology.orchestra.run.smithi103.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T21:06:43.029 INFO:teuthology.orchestra.run.smithi103.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T21:06:43.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:06:43.030 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T21:06:43.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:06:43.116 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T21:06:43.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T21:06:43.190 DEBUG:teuthology.orchestra.run.smithi103:> sudo ip link delete ceph-brx 2024-04-26T21:06:43.322 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-26T21:06:43.331 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T21:06:43.331 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:06:43.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.550+0000 7f2607200640 1 Processor -- start 2024-04-26T21:06:43.551 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.550+0000 7f2607200640 1 -- start start 2024-04-26T21:06:43.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.551+0000 7f2607200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f26080711f0 0x7f26080cf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:43.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.551+0000 7f2607200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f26080d2c10 0x7f26080d30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:43.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.551+0000 7f2607200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f260806c640 con 0x7f260815f0d0 2024-04-26T21:06:43.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.551+0000 7f2607200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f26080cfe90 con 0x7f26080711f0 2024-04-26T21:06:43.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.551+0000 7f2607200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f26080d0170 con 0x7f26080d2c10 2024-04-26T21:06:43.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.552+0000 7f2605e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f260815f0d0 0x7f26080d0e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40202/0 (socket says 172.21.15.28:40202) 2024-04-26T21:06:43.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.552+0000 7f2605e00640 1 -- 172.21.15.28:0/2048818185 learned_addr learned my addr 172.21.15.28:0/2048818185 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:43.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.552+0000 7f2605400640 1 --2- 172.21.15.28:0/2048818185 >> v2:172.21.15.28:3300/0 conn(0x7f26080711f0 0x7f26080cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:43.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.552+0000 7f2606800640 1 --2- 172.21.15.28:0/2048818185 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f26080d2c10 0x7f26080d30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:43.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.552+0000 7f2605400640 1 -- 172.21.15.28:0/2048818185 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f26080d2c10 msgr2=0x7f26080d30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:43.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.552+0000 7f2605400640 1 --2- 172.21.15.28:0/2048818185 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f26080d2c10 0x7f26080d30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:43.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.552+0000 7f2604a00640 1 -- 172.21.15.28:0/2048818185 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2968458380 0 0) 0x7f260806c640 con 0x7f260815f0d0 2024-04-26T21:06:43.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.553+0000 7f2605400640 1 -- 172.21.15.28:0/2048818185 >> v1:172.21.15.28:6789/0 conn(0x7f260815f0d0 legacy=0x7f26080d0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:43.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.553+0000 7f2605400640 1 -- 172.21.15.28:0/2048818185 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26080d0450 con 0x7f26080711f0 2024-04-26T21:06:43.554 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.553+0000 7f2606800640 1 --2- 172.21.15.28:0/2048818185 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f26080d2c10 0x7f26080d30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:43.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.554+0000 7f2605400640 1 --2- 172.21.15.28:0/2048818185 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26080711f0 0x7f26080cf520 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009d90 tx=0x7f25f002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d60a42f51ffdf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:43.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.554+0000 7f2604a00640 1 -- 172.21.15.28:0/2048818185 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0007740 con 0x7f26080711f0 2024-04-26T21:06:43.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.554+0000 7f2604a00640 1 -- 172.21.15.28:0/2048818185 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f00078e0 con 0x7f26080711f0 2024-04-26T21:06:43.555 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.554+0000 7f2604a00640 1 -- 172.21.15.28:0/2048818185 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0007cb0 con 0x7f26080711f0 2024-04-26T21:06:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.554+0000 7f2607200640 1 -- 172.21.15.28:0/2048818185 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26080711f0 msgr2=0x7f26080cf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.554+0000 7f2607200640 1 --2- 172.21.15.28:0/2048818185 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26080711f0 0x7f26080cf520 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009d90 tx=0x7f25f002a660 comp rx=0 tx=0).stop 2024-04-26T21:06:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.555+0000 7f2607200640 1 -- 172.21.15.28:0/2048818185 shutdown_connections 2024-04-26T21:06:43.556 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.555+0000 7f2607200640 1 --2- 172.21.15.28:0/2048818185 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f26080d2c10 0x7f26080d30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:43.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.555+0000 7f2607200640 1 --2- 172.21.15.28:0/2048818185 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26080711f0 0x7f26080cf520 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:43.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.555+0000 7f2607200640 1 -- 172.21.15.28:0/2048818185 >> 172.21.15.28:0/2048818185 conn(0x7f260809ba50 msgr2=0x7f260809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:43.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.555+0000 7f2607200640 1 -- 172.21.15.28:0/2048818185 shutdown_connections 2024-04-26T21:06:43.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.555+0000 7f2607200640 1 -- 172.21.15.28:0/2048818185 wait complete. 2024-04-26T21:06:43.557 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.556+0000 7f2607200640 1 Processor -- start 2024-04-26T21:06:43.558 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.557+0000 7f2607200640 1 -- start start 2024-04-26T21:06:43.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.558+0000 7f2607200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26080711f0 0x7f26081f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:43.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.558+0000 7f2607200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26080d2c10 0x7f2608153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:43.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.558+0000 7f2607200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f260815f0d0 0x7f26080b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:43.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.558+0000 7f2607200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f26080cfa60 con 0x7f26080711f0 2024-04-26T21:06:43.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.558+0000 7f2607200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f260815f4b0 con 0x7f26080d2c10 2024-04-26T21:06:43.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.558+0000 7f2607200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f26080cfbe0 con 0x7f260815f0d0 2024-04-26T21:06:43.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.558+0000 7f2605e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26080711f0 0x7f26081f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:43.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.558+0000 7f2605e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26080711f0 0x7f26081f4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34034/0 (socket says 172.21.15.28:34034) 2024-04-26T21:06:43.559 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.558+0000 7f2605e00640 1 -- 172.21.15.28:0/1088196411 learned_addr learned my addr 172.21.15.28:0/1088196411 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:43.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.558+0000 7f2606800640 1 --2- 172.21.15.28:0/1088196411 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f260815f0d0 0x7f26080b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:43.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.558+0000 7f2605e00640 1 -- 172.21.15.28:0/1088196411 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f260815f0d0 msgr2=0x7f26080b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:43.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.558+0000 7f2605e00640 1 --2- 172.21.15.28:0/1088196411 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f260815f0d0 0x7f26080b65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:43.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.558+0000 7f2605e00640 1 -- 172.21.15.28:0/1088196411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26080d2c10 msgr2=0x7f2608153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:43.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.558+0000 7f2605e00640 1 --2- 172.21.15.28:0/1088196411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26080d2c10 0x7f2608153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:43.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.558+0000 7f2605e00640 1 -- 172.21.15.28:0/1088196411 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26081f7f40 con 0x7f26080711f0 2024-04-26T21:06:43.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.558+0000 7f2606800640 1 --2- 172.21.15.28:0/1088196411 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f260815f0d0 0x7f26080b65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:43.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.559+0000 7f2605e00640 1 --2- 172.21.15.28:0/1088196411 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26080711f0 0x7f26081f4bf0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f25f800c430 tx=0x7f25f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:43.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.559+0000 7f25fe000640 1 -- 172.21.15.28:0/1088196411 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f80074e0 con 0x7f26080711f0 2024-04-26T21:06:43.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.559+0000 7f25fe000640 1 -- 172.21.15.28:0/1088196411 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f80047e0 con 0x7f26080711f0 2024-04-26T21:06:43.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.559+0000 7f25fe000640 1 -- 172.21.15.28:0/1088196411 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f8004bd0 con 0x7f26080711f0 2024-04-26T21:06:43.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.559+0000 7f2607200640 1 -- 172.21.15.28:0/1088196411 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2608160520 con 0x7f26080711f0 2024-04-26T21:06:43.561 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.559+0000 7f2607200640 1 -- 172.21.15.28:0/1088196411 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26081607a0 con 0x7f26080711f0 2024-04-26T21:06:43.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.560+0000 7f2607200640 1 -- 172.21.15.28:0/1088196411 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f260806a360 con 0x7f26080711f0 2024-04-26T21:06:43.562 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.562+0000 7f25fe000640 1 -- 172.21.15.28:0/1088196411 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f25f8011070 con 0x7f26080711f0 2024-04-26T21:06:43.563 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.562+0000 7f25fe000640 1 --2- 172.21.15.28:0/1088196411 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f25e406c8f0 0x7f25e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:43.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.563+0000 7f25fe000640 1 -- 172.21.15.28:0/1088196411 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f25f808b3f0 con 0x7f26080711f0 2024-04-26T21:06:43.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.563+0000 7f2605400640 1 --2- 172.21.15.28:0/1088196411 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f25e406c8f0 0x7f25e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:43.564 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.564+0000 7f2605400640 1 --2- 172.21.15.28:0/1088196411 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f25e406c8f0 0x7f25e406edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f25f0034000 tx=0x7f25f002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:43.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.567+0000 7f25fe000640 1 -- 172.21.15.28:0/1088196411 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2608160520 con 0x7f26080711f0 2024-04-26T21:06:43.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.766+0000 7f2607200640 1 -- 172.21.15.28:0/1088196411 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f260806c420 con 0x7f25e406c8f0 2024-04-26T21:06:43.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.767+0000 7f25fe000640 1 -- 172.21.15.28:0/1088196411 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13471 (secure 0 0 0) 0x7f260806c420 con 0x7f25e406c8f0 2024-04-26T21:06:43.769 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T21:06:43.769 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T21:06:43.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.770+0000 7f2607200640 1 -- 172.21.15.28:0/1088196411 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f25e406c8f0 msgr2=0x7f25e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:43.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.770+0000 7f2607200640 1 --2- 172.21.15.28:0/1088196411 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f25e406c8f0 0x7f25e406edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f25f0034000 tx=0x7f25f002e000 comp rx=0 tx=0).stop 2024-04-26T21:06:43.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.770+0000 7f2607200640 1 -- 172.21.15.28:0/1088196411 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26080711f0 msgr2=0x7f26081f4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:43.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.770+0000 7f2607200640 1 --2- 172.21.15.28:0/1088196411 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26080711f0 0x7f26081f4bf0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f25f800c430 tx=0x7f25f800c8f0 comp rx=0 tx=0).stop 2024-04-26T21:06:43.771 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.770+0000 7f2607200640 1 -- 172.21.15.28:0/1088196411 shutdown_connections 2024-04-26T21:06:43.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.770+0000 7f2607200640 1 --2- 172.21.15.28:0/1088196411 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f25e406c8f0 0x7f25e406edb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:43.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.770+0000 7f2607200640 1 --2- 172.21.15.28:0/1088196411 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f260815f0d0 0x7f26080b65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:43.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.770+0000 7f2607200640 1 --2- 172.21.15.28:0/1088196411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f26080d2c10 0x7f2608153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:43.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.770+0000 7f2607200640 1 --2- 172.21.15.28:0/1088196411 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f26080711f0 0x7f26081f4bf0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:43.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.770+0000 7f2607200640 1 -- 172.21.15.28:0/1088196411 >> 172.21.15.28:0/1088196411 conn(0x7f260809ba50 msgr2=0x7f26080d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:43.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.771+0000 7f2607200640 1 -- 172.21.15.28:0/1088196411 shutdown_connections 2024-04-26T21:06:43.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.771+0000 7f2607200640 1 -- 172.21.15.28:0/1088196411 wait complete. 2024-04-26T21:06:43.791 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-04-26T21:06:42.911492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":296184,"kb_used_data":46768,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754678536,"statfs":{"total":773094113280,"available":772790820864,"internally_reserved":0,"allocated":47890432,"data_stored":38256522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17625806,"num_objects":-993,"num_object_clones":0,"num_object_copies":-1986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-993,"num_whiteouts":0,"num_read":-6,"num_read_kb":-6,"num_write":-1410,"num_write_kb":-18300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-36,"num_bytes_recovered":-103097,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001931"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":223,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:02:48.819329+0000","last_change":"2024-04-26T20:56:46.848603+0000","last_active":"2024-04-26T21:02:48.819329+0000","last_peered":"2024-04-26T21:02:48.819329+0000","last_clean":"2024-04-26T21:02:48.819329+0000","last_became_active":"2024-04-26T20:56:46.846091+0000","last_became_peered":"2024-04-26T20:56:46.846091+0000","last_unstale":"2024-04-26T21:02:48.819329+0000","last_undegraded":"2024-04-26T21:02:48.819329+0000","last_fullsized":"2024-04-26T21:02:48.819329+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_clean_scrub_stamp":"2024-04-26T20:56:45.594041+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:00.232387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574970,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36112,"kb_used_data":4936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335728,"statfs":{"total":96636764160,"available":96599785472,"internally_reserved":0,"allocated":5054464,"data_stored":3882500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574969,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38732,"kb_used_data":7556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333108,"statfs":{"total":96636764160,"available":96597102592,"internally_reserved":0,"allocated":7737344,"data_stored":6193821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1662976,"data_stored":687090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37060,"kb_used_data":5884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334780,"statfs":{"total":96636764160,"available":96598814720,"internally_reserved":0,"allocated":6025216,"data_stored":4957950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574970,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38856,"kb_used_data":7680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332984,"statfs":{"total":96636764160,"available":96596975616,"internally_reserved":0,"allocated":7864320,"data_stored":6657722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574970,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40360,"kb_used_data":9180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331480,"statfs":{"total":96636764160,"available":96595435520,"internally_reserved":0,"allocated":9400320,"data_stored":8314706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38248,"kb_used_data":7068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333592,"statfs":{"total":96636764160,"available":96597598208,"internally_reserved":0,"allocated":7237632,"data_stored":6130317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574969,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34016,"kb_used_data":2840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337824,"statfs":{"total":96636764160,"available":96601931776,"internally_reserved":0,"allocated":2908160,"data_stored":1432416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:06:43.792 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:06:43.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.989+0000 7f1aa7200640 1 Processor -- start 2024-04-26T21:06:43.990 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.989+0000 7f1aa7200640 1 -- start start 2024-04-26T21:06:43.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.991+0000 7f1aa7200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1aa8073f10 0x7f1aa80cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:43.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.991+0000 7f1aa7200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1aa8072760 con 0x7f1aa8144b90 2024-04-26T21:06:43.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.991+0000 7f1aa7200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1aa8072a40 con 0x7f1aa8145080 2024-04-26T21:06:43.992 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.991+0000 7f1aa7200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f1aa8072d20 con 0x7f1aa8073f10 2024-04-26T21:06:43.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.991+0000 7f1aa6800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1aa8073f10 0x7f1aa80cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:43.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.991+0000 7f1aa6800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1aa8073f10 0x7f1aa80cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34036/0 (socket says 172.21.15.28:34036) 2024-04-26T21:06:43.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.991+0000 7f1aa5e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1aa8144b90 0x7f1aa80ce620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40212/0 (socket says 172.21.15.28:40212) 2024-04-26T21:06:43.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.991+0000 7f1aa6800640 1 -- 172.21.15.28:0/640329028 learned_addr learned my addr 172.21.15.28:0/640329028 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:43.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.992+0000 7f1aa6800640 1 -- 172.21.15.28:0/640329028 >> v1:172.21.15.103:6789/0 conn(0x7f1aa8145080 legacy=0x7f1aa8073800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:43.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.992+0000 7f1aa6800640 1 -- 172.21.15.28:0/640329028 >> v1:172.21.15.28:6789/0 conn(0x7f1aa8144b90 legacy=0x7f1aa80ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:43.993 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.992+0000 7f1aa6800640 1 -- 172.21.15.28:0/640329028 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa8073000 con 0x7f1aa8073f10 2024-04-26T21:06:43.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.993+0000 7f1aa6800640 1 --2- 172.21.15.28:0/640329028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aa8073f10 0x7f1aa80cb0a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009e50 tx=0x7f1aa002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94987864f9b0e029 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:43.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.993+0000 7f1aa4a00640 1 -- 172.21.15.28:0/640329028 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0036070 con 0x7f1aa8073f10 2024-04-26T21:06:43.994 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.993+0000 7f1aa4a00640 1 -- 172.21.15.28:0/640329028 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aa0031600 con 0x7f1aa8073f10 2024-04-26T21:06:43.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.993+0000 7f1aa4a00640 1 -- 172.21.15.28:0/640329028 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa00319d0 con 0x7f1aa8073f10 2024-04-26T21:06:43.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.994+0000 7f1aa7200640 1 -- 172.21.15.28:0/640329028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aa8073f10 msgr2=0x7f1aa80cb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:43.995 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.994+0000 7f1aa7200640 1 --2- 172.21.15.28:0/640329028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aa8073f10 0x7f1aa80cb0a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009e50 tx=0x7f1aa002a6d0 comp rx=0 tx=0).stop 2024-04-26T21:06:43.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.994+0000 7f1aa7200640 1 -- 172.21.15.28:0/640329028 shutdown_connections 2024-04-26T21:06:43.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.994+0000 7f1aa7200640 1 --2- 172.21.15.28:0/640329028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aa8073f10 0x7f1aa80cb0a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:43.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.994+0000 7f1aa7200640 1 -- 172.21.15.28:0/640329028 >> 172.21.15.28:0/640329028 conn(0x7f1aa80d0a50 msgr2=0x7f1aa80d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:43.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.994+0000 7f1aa7200640 1 -- 172.21.15.28:0/640329028 shutdown_connections 2024-04-26T21:06:43.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.994+0000 7f1aa7200640 1 -- 172.21.15.28:0/640329028 wait complete. 2024-04-26T21:06:43.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.996+0000 7f1aa7200640 1 Processor -- start 2024-04-26T21:06:43.997 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.996+0000 7f1aa7200640 1 -- start start 2024-04-26T21:06:43.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.997+0000 7f1aa7200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aa8144b90 0x7f1aa80cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:43.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.997+0000 7f1aa7200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1aa8145080 0x7f1aa81563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:43.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.997+0000 7f1aa7200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1aa80c64b0 0x7f1aa8156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:43.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.997+0000 7f1aa7200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa80ced30 con 0x7f1aa8144b90 2024-04-26T21:06:43.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.997+0000 7f1aa7200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa806e710 con 0x7f1aa8145080 2024-04-26T21:06:43.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.997+0000 7f1aa7200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f1aa81572a0 con 0x7f1aa80c64b0 2024-04-26T21:06:43.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.997+0000 7f1aa5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aa8144b90 0x7f1aa80cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:43.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.997+0000 7f1aa5e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aa8144b90 0x7f1aa80cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34046/0 (socket says 172.21.15.28:34046) 2024-04-26T21:06:43.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.997+0000 7f1aa5e00640 1 -- 172.21.15.28:0/1529705335 learned_addr learned my addr 172.21.15.28:0/1529705335 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:44.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.998+0000 7f1aa6800640 1 --2- 172.21.15.28:0/1529705335 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1aa80c64b0 0x7f1aa8156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:44.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.998+0000 7f1aa5400640 1 --2- 172.21.15.28:0/1529705335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1aa8145080 0x7f1aa81563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:44.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.998+0000 7f1aa5e00640 1 -- 172.21.15.28:0/1529705335 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1aa80c64b0 msgr2=0x7f1aa8156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:44.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.998+0000 7f1aa5e00640 1 --2- 172.21.15.28:0/1529705335 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1aa80c64b0 0x7f1aa8156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.998+0000 7f1aa5e00640 1 -- 172.21.15.28:0/1529705335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1aa8145080 msgr2=0x7f1aa81563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:44.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.998+0000 7f1aa5e00640 1 --2- 172.21.15.28:0/1529705335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1aa8145080 0x7f1aa81563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.998+0000 7f1aa5e00640 1 -- 172.21.15.28:0/1529705335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa8157580 con 0x7f1aa8144b90 2024-04-26T21:06:44.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.998+0000 7f1aa5e00640 1 --2- 172.21.15.28:0/1529705335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aa8144b90 0x7f1aa80cf2b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800c400 tx=0x7f1a9800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:44.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.999+0000 7f1a9e000640 1 -- 172.21.15.28:0/1529705335 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a980074e0 con 0x7f1aa8144b90 2024-04-26T21:06:44.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.999+0000 7f1a9e000640 1 -- 172.21.15.28:0/1529705335 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a98007680 con 0x7f1aa8144b90 2024-04-26T21:06:44.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.999+0000 7f1a9e000640 1 -- 172.21.15.28:0/1529705335 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a98004a00 con 0x7f1aa8144b90 2024-04-26T21:06:44.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.999+0000 7f1aa7200640 1 -- 172.21.15.28:0/1529705335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa8157860 con 0x7f1aa8144b90 2024-04-26T21:06:44.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:43.999+0000 7f1aa7200640 1 -- 172.21.15.28:0/1529705335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa8129cc0 con 0x7f1aa8144b90 2024-04-26T21:06:44.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.000+0000 7f1aa7200640 1 -- 172.21.15.28:0/1529705335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa806a360 con 0x7f1aa8144b90 2024-04-26T21:06:44.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.001+0000 7f1a9e000640 1 -- 172.21.15.28:0/1529705335 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a9801a020 con 0x7f1aa8144b90 2024-04-26T21:06:44.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.002+0000 7f1a9e000640 1 --2- 172.21.15.28:0/1529705335 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1a8806c8f0 0x7f1a8806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:44.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.002+0000 7f1a9e000640 1 -- 172.21.15.28:0/1529705335 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f1a9808b150 con 0x7f1aa8144b90 2024-04-26T21:06:44.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.002+0000 7f1aa5400640 1 --2- 172.21.15.28:0/1529705335 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1a8806c8f0 0x7f1a8806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:44.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.003+0000 7f1aa5400640 1 --2- 172.21.15.28:0/1529705335 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1a8806c8f0 0x7f1a8806edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1a94006fd0 tx=0x7f1a94008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:44.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.007+0000 7f1a9e000640 1 -- 172.21.15.28:0/1529705335 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1a980597e0 con 0x7f1aa8144b90 2024-04-26T21:06:44.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.208+0000 7f1aa7200640 1 -- 172.21.15.28:0/1529705335 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1aa8157c40 con 0x7f1a8806c8f0 2024-04-26T21:06:44.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.210+0000 7f1a9e000640 1 -- 172.21.15.28:0/1529705335 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13471 (secure 0 0 0) 0x7f1aa8157c40 con 0x7f1a8806c8f0 2024-04-26T21:06:44.212 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T21:06:44.212 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T21:06:44.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.213+0000 7f1aa7200640 1 -- 172.21.15.28:0/1529705335 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1a8806c8f0 msgr2=0x7f1a8806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:44.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.213+0000 7f1aa7200640 1 --2- 172.21.15.28:0/1529705335 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1a8806c8f0 0x7f1a8806edb0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1a94006fd0 tx=0x7f1a94008040 comp rx=0 tx=0).stop 2024-04-26T21:06:44.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.213+0000 7f1aa7200640 1 -- 172.21.15.28:0/1529705335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aa8144b90 msgr2=0x7f1aa80cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:44.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.213+0000 7f1aa7200640 1 --2- 172.21.15.28:0/1529705335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aa8144b90 0x7f1aa80cf2b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f1a9800c400 tx=0x7f1a9800c8c0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.213+0000 7f1aa7200640 1 -- 172.21.15.28:0/1529705335 shutdown_connections 2024-04-26T21:06:44.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.213+0000 7f1aa7200640 1 --2- 172.21.15.28:0/1529705335 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f1a8806c8f0 0x7f1a8806edb0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.213+0000 7f1aa7200640 1 --2- 172.21.15.28:0/1529705335 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f1aa80c64b0 0x7f1aa8156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.213+0000 7f1aa7200640 1 --2- 172.21.15.28:0/1529705335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1aa8145080 0x7f1aa81563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.213+0000 7f1aa7200640 1 --2- 172.21.15.28:0/1529705335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aa8144b90 0x7f1aa80cf2b0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.213+0000 7f1aa7200640 1 -- 172.21.15.28:0/1529705335 >> 172.21.15.28:0/1529705335 conn(0x7f1aa80d0a50 msgr2=0x7f1aa806eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:44.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.213+0000 7f1aa7200640 1 -- 172.21.15.28:0/1529705335 shutdown_connections 2024-04-26T21:06:44.215 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.213+0000 7f1aa7200640 1 -- 172.21.15.28:0/1529705335 wait complete. 2024-04-26T21:06:44.236 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-04-26T21:06:42.911492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":296184,"kb_used_data":46768,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754678536,"statfs":{"total":773094113280,"available":772790820864,"internally_reserved":0,"allocated":47890432,"data_stored":38256522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17625806,"num_objects":-993,"num_object_clones":0,"num_object_copies":-1986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-993,"num_whiteouts":0,"num_read":-6,"num_read_kb":-6,"num_write":-1410,"num_write_kb":-18300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-36,"num_bytes_recovered":-103097,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001931"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":223,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:02:48.819329+0000","last_change":"2024-04-26T20:56:46.848603+0000","last_active":"2024-04-26T21:02:48.819329+0000","last_peered":"2024-04-26T21:02:48.819329+0000","last_clean":"2024-04-26T21:02:48.819329+0000","last_became_active":"2024-04-26T20:56:46.846091+0000","last_became_peered":"2024-04-26T20:56:46.846091+0000","last_unstale":"2024-04-26T21:02:48.819329+0000","last_undegraded":"2024-04-26T21:02:48.819329+0000","last_fullsized":"2024-04-26T21:02:48.819329+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_clean_scrub_stamp":"2024-04-26T20:56:45.594041+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:00.232387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574970,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36112,"kb_used_data":4936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335728,"statfs":{"total":96636764160,"available":96599785472,"internally_reserved":0,"allocated":5054464,"data_stored":3882500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574969,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38732,"kb_used_data":7556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333108,"statfs":{"total":96636764160,"available":96597102592,"internally_reserved":0,"allocated":7737344,"data_stored":6193821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1662976,"data_stored":687090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37060,"kb_used_data":5884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334780,"statfs":{"total":96636764160,"available":96598814720,"internally_reserved":0,"allocated":6025216,"data_stored":4957950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574970,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38856,"kb_used_data":7680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332984,"statfs":{"total":96636764160,"available":96596975616,"internally_reserved":0,"allocated":7864320,"data_stored":6657722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574970,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40360,"kb_used_data":9180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331480,"statfs":{"total":96636764160,"available":96595435520,"internally_reserved":0,"allocated":9400320,"data_stored":8314706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38248,"kb_used_data":7068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333592,"statfs":{"total":96636764160,"available":96597598208,"internally_reserved":0,"allocated":7237632,"data_stored":6130317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574969,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34016,"kb_used_data":2840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337824,"statfs":{"total":96636764160,"available":96601931776,"internally_reserved":0,"allocated":2908160,"data_stored":1432416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:06:44.236 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T21:06:44.237 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:06:44.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.435+0000 7f86ca800640 1 Processor -- start 2024-04-26T21:06:44.436 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.435+0000 7f86ca800640 1 -- start start 2024-04-26T21:06:44.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.436+0000 7f86ca800640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f86c415f1b0 0x7f86c40d4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:44.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.436+0000 7f86ca800640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f86c40cf700 0x7f86c40d4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:44.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.437+0000 7f86ca800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86c40d0710 con 0x7f86c40d5370 2024-04-26T21:06:44.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.437+0000 7f86ca800640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f86c40d57e0 con 0x7f86c415f1b0 2024-04-26T21:06:44.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.437+0000 7f86ca800640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f86c40cff00 con 0x7f86c40cf700 2024-04-26T21:06:44.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.437+0000 7f86c9400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f86c415f1b0 0x7f86c40d4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:44.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.437+0000 7f86c9e00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f86c40d5370 0x7f86c40b80b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40218/0 (socket says 172.21.15.28:40218) 2024-04-26T21:06:44.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.437+0000 7f86c9400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f86c415f1b0 0x7f86c40d4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34052/0 (socket says 172.21.15.28:34052) 2024-04-26T21:06:44.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.437+0000 7f86c9e00640 1 -- 172.21.15.28:0/835704022 learned_addr learned my addr 172.21.15.28:0/835704022 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:44.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.437+0000 7f86c8a00640 1 --2- 172.21.15.28:0/835704022 >> v2:172.21.15.103:3300/0 conn(0x7f86c40cf700 0x7f86c40d4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:44.439 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.438+0000 7f86c9400640 1 -- 172.21.15.28:0/835704022 >> v2:172.21.15.103:3300/0 conn(0x7f86c40cf700 msgr2=0x7f86c40d4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:44.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.438+0000 7f86c9400640 1 --2- 172.21.15.28:0/835704022 >> v2:172.21.15.103:3300/0 conn(0x7f86c40cf700 0x7f86c40d4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.438+0000 7f86c9400640 1 -- 172.21.15.28:0/835704022 >> v1:172.21.15.28:6789/0 conn(0x7f86c40d5370 legacy=0x7f86c40b80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:44.440 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.438+0000 7f86c9400640 1 -- 172.21.15.28:0/835704022 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c40d0150 con 0x7f86c415f1b0 2024-04-26T21:06:44.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.439+0000 7f86c9400640 1 --2- 172.21.15.28:0/835704022 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86c415f1b0 0x7f86c40d4700 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f86c40d0080 tx=0x7f86b802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7db50b6ffb65764 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:44.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.439+0000 7f86c3e00640 1 -- 172.21.15.28:0/835704022 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b8037070 con 0x7f86c415f1b0 2024-04-26T21:06:44.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.439+0000 7f86c3e00640 1 -- 172.21.15.28:0/835704022 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86b8032440 con 0x7f86c415f1b0 2024-04-26T21:06:44.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.439+0000 7f86c3e00640 1 -- 172.21.15.28:0/835704022 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b8032810 con 0x7f86c415f1b0 2024-04-26T21:06:44.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.440+0000 7f86ca800640 1 -- 172.21.15.28:0/835704022 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86c415f1b0 msgr2=0x7f86c40d4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:44.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.440+0000 7f86ca800640 1 --2- 172.21.15.28:0/835704022 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86c415f1b0 0x7f86c40d4700 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f86c40d0080 tx=0x7f86b802a230 comp rx=0 tx=0).stop 2024-04-26T21:06:44.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.440+0000 7f86ca800640 1 -- 172.21.15.28:0/835704022 shutdown_connections 2024-04-26T21:06:44.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.440+0000 7f86ca800640 1 --2- 172.21.15.28:0/835704022 >> v2:172.21.15.103:3300/0 conn(0x7f86c40cf700 0x7f86c40d4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.441 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.440+0000 7f86ca800640 1 --2- 172.21.15.28:0/835704022 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86c415f1b0 0x7f86c40d4700 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.440+0000 7f86ca800640 1 -- 172.21.15.28:0/835704022 >> 172.21.15.28:0/835704022 conn(0x7f86c406c770 msgr2=0x7f86c406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:44.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.440+0000 7f86ca800640 1 -- 172.21.15.28:0/835704022 shutdown_connections 2024-04-26T21:06:44.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.440+0000 7f86ca800640 1 -- 172.21.15.28:0/835704022 wait complete. 2024-04-26T21:06:44.442 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.442+0000 7f86ca800640 1 Processor -- start 2024-04-26T21:06:44.443 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.442+0000 7f86ca800640 1 -- start start 2024-04-26T21:06:44.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.443+0000 7f86ca800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86c40cf700 0x7f86c41546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:44.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.443+0000 7f86ca800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86c40d5370 0x7f86c4156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:44.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.443+0000 7f86ca800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86c415f1b0 0x7f86c40c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:44.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.443+0000 7f86ca800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f86c40d3f00 con 0x7f86c40cf700 2024-04-26T21:06:44.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.443+0000 7f86ca800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f86c40d4270 con 0x7f86c415f1b0 2024-04-26T21:06:44.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.443+0000 7f86ca800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f86c415f590 con 0x7f86c40d5370 2024-04-26T21:06:44.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.443+0000 7f86c9400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86c40cf700 0x7f86c41546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:44.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.443+0000 7f86c9400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86c40cf700 0x7f86c41546f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34060/0 (socket says 172.21.15.28:34060) 2024-04-26T21:06:44.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.443+0000 7f86c9400640 1 -- 172.21.15.28:0/3202620959 learned_addr learned my addr 172.21.15.28:0/3202620959 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:44.446 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.443+0000 7f86c9400640 1 -- 172.21.15.28:0/3202620959 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86c40d5370 msgr2=0x7f86c4156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:44.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.443+0000 7f86c9400640 1 --2- 172.21.15.28:0/3202620959 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86c40d5370 0x7f86c4156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.443+0000 7f86c9400640 1 -- 172.21.15.28:0/3202620959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86c415f1b0 msgr2=0x7f86c40c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:44.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.443+0000 7f86c9400640 1 --2- 172.21.15.28:0/3202620959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86c415f1b0 0x7f86c40c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.443+0000 7f86c9400640 1 -- 172.21.15.28:0/3202620959 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c40cc010 con 0x7f86c40cf700 2024-04-26T21:06:44.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.444+0000 7f86c9400640 1 --2- 172.21.15.28:0/3202620959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86c40cf700 0x7f86c41546f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f86b8000c00 tx=0x7f86b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:44.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.444+0000 7f86c1600640 1 -- 172.21.15.28:0/3202620959 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b8037040 con 0x7f86c40cf700 2024-04-26T21:06:44.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.444+0000 7f86c1600640 1 -- 172.21.15.28:0/3202620959 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86b8042070 con 0x7f86c40cf700 2024-04-26T21:06:44.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.444+0000 7f86c1600640 1 -- 172.21.15.28:0/3202620959 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b8030bd0 con 0x7f86c40cf700 2024-04-26T21:06:44.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.444+0000 7f86ca800640 1 -- 172.21.15.28:0/3202620959 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c40cc2f0 con 0x7f86c40cf700 2024-04-26T21:06:44.447 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.444+0000 7f86ca800640 1 -- 172.21.15.28:0/3202620959 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c40c9dc0 con 0x7f86c40cf700 2024-04-26T21:06:44.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.445+0000 7f86ca800640 1 -- 172.21.15.28:0/3202620959 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c406a360 con 0x7f86c40cf700 2024-04-26T21:06:44.448 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.447+0000 7f86c1600640 1 -- 172.21.15.28:0/3202620959 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86b8047460 con 0x7f86c40cf700 2024-04-26T21:06:44.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.448+0000 7f86c1600640 1 --2- 172.21.15.28:0/3202620959 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f86a806c8f0 0x7f86a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:44.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.448+0000 7f86c1600640 1 -- 172.21.15.28:0/3202620959 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f86b80ac390 con 0x7f86c40cf700 2024-04-26T21:06:44.449 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.448+0000 7f86c8a00640 1 --2- 172.21.15.28:0/3202620959 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f86a806c8f0 0x7f86a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:44.450 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.449+0000 7f86c8a00640 1 --2- 172.21.15.28:0/3202620959 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f86a806c8f0 0x7f86a806edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f86b4006fd0 tx=0x7f86b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:44.453 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.453+0000 7f86c1600640 1 -- 172.21.15.28:0/3202620959 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f86b8032e20 con 0x7f86c40cf700 2024-04-26T21:06:44.656 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.655+0000 7f86ca800640 1 -- 172.21.15.28:0/3202620959 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f86c40b3ae0 con 0x7f86a806c8f0 2024-04-26T21:06:44.658 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.657+0000 7f86c1600640 1 -- 172.21.15.28:0/3202620959 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13471 (secure 0 0 0) 0x7f86c40b3ae0 con 0x7f86a806c8f0 2024-04-26T21:06:44.658 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T21:06:44.658 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T21:06:44.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.660+0000 7f86ca800640 1 -- 172.21.15.28:0/3202620959 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f86a806c8f0 msgr2=0x7f86a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:44.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.660+0000 7f86ca800640 1 --2- 172.21.15.28:0/3202620959 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f86a806c8f0 0x7f86a806edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f86b4006fd0 tx=0x7f86b4008040 comp rx=0 tx=0).stop 2024-04-26T21:06:44.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.660+0000 7f86ca800640 1 -- 172.21.15.28:0/3202620959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86c40cf700 msgr2=0x7f86c41546f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:44.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.660+0000 7f86ca800640 1 --2- 172.21.15.28:0/3202620959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86c40cf700 0x7f86c41546f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f86b8000c00 tx=0x7f86b80046c0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.660+0000 7f86ca800640 1 -- 172.21.15.28:0/3202620959 shutdown_connections 2024-04-26T21:06:44.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.660+0000 7f86ca800640 1 --2- 172.21.15.28:0/3202620959 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f86a806c8f0 0x7f86a806edb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.661 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.660+0000 7f86ca800640 1 --2- 172.21.15.28:0/3202620959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86c415f1b0 0x7f86c40c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.660+0000 7f86ca800640 1 --2- 172.21.15.28:0/3202620959 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f86c40d5370 0x7f86c4156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.660+0000 7f86ca800640 1 --2- 172.21.15.28:0/3202620959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86c40cf700 0x7f86c41546f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.660+0000 7f86ca800640 1 -- 172.21.15.28:0/3202620959 >> 172.21.15.28:0/3202620959 conn(0x7f86c406c770 msgr2=0x7f86c4073150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:44.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.660+0000 7f86ca800640 1 -- 172.21.15.28:0/3202620959 shutdown_connections 2024-04-26T21:06:44.662 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.660+0000 7f86ca800640 1 -- 172.21.15.28:0/3202620959 wait complete. 2024-04-26T21:06:44.681 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-04-26T21:06:42.911492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":296184,"kb_used_data":46768,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754678536,"statfs":{"total":773094113280,"available":772790820864,"internally_reserved":0,"allocated":47890432,"data_stored":38256522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17625806,"num_objects":-993,"num_object_clones":0,"num_object_copies":-1986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-993,"num_whiteouts":0,"num_read":-6,"num_read_kb":-6,"num_write":-1410,"num_write_kb":-18300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-36,"num_bytes_recovered":-103097,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001931"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":223,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-26T21:02:48.819329+0000","last_change":"2024-04-26T20:56:46.848603+0000","last_active":"2024-04-26T21:02:48.819329+0000","last_peered":"2024-04-26T21:02:48.819329+0000","last_clean":"2024-04-26T21:02:48.819329+0000","last_became_active":"2024-04-26T20:56:46.846091+0000","last_became_peered":"2024-04-26T20:56:46.846091+0000","last_unstale":"2024-04-26T21:02:48.819329+0000","last_undegraded":"2024-04-26T21:02:48.819329+0000","last_fullsized":"2024-04-26T21:02:48.819329+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_clean_scrub_stamp":"2024-04-26T20:56:45.594041+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:00.232387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574970,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36112,"kb_used_data":4936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335728,"statfs":{"total":96636764160,"available":96599785472,"internally_reserved":0,"allocated":5054464,"data_stored":3882500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574969,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38732,"kb_used_data":7556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333108,"statfs":{"total":96636764160,"available":96597102592,"internally_reserved":0,"allocated":7737344,"data_stored":6193821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574969,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1662976,"data_stored":687090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574969,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37060,"kb_used_data":5884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94334780,"statfs":{"total":96636764160,"available":96598814720,"internally_reserved":0,"allocated":6025216,"data_stored":4957950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574970,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38856,"kb_used_data":7680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332984,"statfs":{"total":96636764160,"available":96596975616,"internally_reserved":0,"allocated":7864320,"data_stored":6657722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574970,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40360,"kb_used_data":9180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331480,"statfs":{"total":96636764160,"available":96595435520,"internally_reserved":0,"allocated":9400320,"data_stored":8314706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38248,"kb_used_data":7068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333592,"statfs":{"total":96636764160,"available":96597598208,"internally_reserved":0,"allocated":7237632,"data_stored":6130317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574969,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34016,"kb_used_data":2840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337824,"statfs":{"total":96636764160,"available":96601931776,"internally_reserved":0,"allocated":2908160,"data_stored":1432416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:06:44.681 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T21:06:44.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.864+0000 7fdeede00640 1 Processor -- start 2024-04-26T21:06:44.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.864+0000 7fdeede00640 1 -- start start 2024-04-26T21:06:44.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.865+0000 7fdeede00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdee80cf7e0 0x7fdee80b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:44.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.865+0000 7fdeede00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdee809c570 0x7fdee80cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:44.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.866+0000 7fdeede00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdee80d0fe0 con 0x7fdee80d0100 2024-04-26T21:06:44.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.866+0000 7fdeede00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fdee80b4640 con 0x7fdee809c570 2024-04-26T21:06:44.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.866+0000 7fdeede00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fdee80d3220 con 0x7fdee80cf7e0 2024-04-26T21:06:44.867 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.866+0000 7fdee6000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdee809c570 0x7fdee80cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:44.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.866+0000 7fdee6000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdee809c570 0x7fdee80cfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34072/0 (socket says 172.21.15.28:34072) 2024-04-26T21:06:44.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.866+0000 7fdee6000640 1 -- 172.21.15.28:0/2678355868 learned_addr learned my addr 172.21.15.28:0/2678355868 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:44.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.866+0000 7fdee7400640 1 --1- 172.21.15.28:0/2678355868 >> v1:172.21.15.28:6789/0 conn(0x7fdee80d0100 0x7fdee80b2440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40220/0 (socket says 172.21.15.28:40220) 2024-04-26T21:06:44.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.866+0000 7fdee6a00640 1 --2- 172.21.15.28:0/2678355868 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdee80cf7e0 0x7fdee80b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:44.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.867+0000 7fdee6000640 1 -- 172.21.15.28:0/2678355868 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdee80cf7e0 msgr2=0x7fdee80b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:44.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.867+0000 7fdee6000640 1 --2- 172.21.15.28:0/2678355868 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdee80cf7e0 0x7fdee80b4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.868 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.867+0000 7fdee6000640 1 -- 172.21.15.28:0/2678355868 >> v1:172.21.15.28:6789/0 conn(0x7fdee80d0100 legacy=0x7fdee80b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:44.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.867+0000 7fdee6000640 1 -- 172.21.15.28:0/2678355868 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee80d3500 con 0x7fdee809c570 2024-04-26T21:06:44.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.867+0000 7fdee5600640 1 -- 172.21.15.28:0/2678355868 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1019102030 0 0) 0x7fdee80d0fe0 con 0x7fdee80d0100 2024-04-26T21:06:44.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.868+0000 7fdee6000640 1 --2- 172.21.15.28:0/2678355868 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee809c570 0x7fdee80cfbc0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fded4009870 tx=0x7fded402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636523847c6ac000 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:44.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.868+0000 7fdee5600640 1 -- 172.21.15.28:0/2678355868 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4037070 con 0x7fdee809c570 2024-04-26T21:06:44.869 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.868+0000 7fdee5600640 1 -- 172.21.15.28:0/2678355868 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded4032440 con 0x7fdee809c570 2024-04-26T21:06:44.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.868+0000 7fdee5600640 1 -- 172.21.15.28:0/2678355868 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4032810 con 0x7fdee809c570 2024-04-26T21:06:44.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.869+0000 7fdeede00640 1 -- 172.21.15.28:0/2678355868 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee809c570 msgr2=0x7fdee80cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:44.870 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.869+0000 7fdeede00640 1 --2- 172.21.15.28:0/2678355868 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee809c570 0x7fdee80cfbc0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fded4009870 tx=0x7fded402a270 comp rx=0 tx=0).stop 2024-04-26T21:06:44.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.869+0000 7fdeede00640 1 -- 172.21.15.28:0/2678355868 shutdown_connections 2024-04-26T21:06:44.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.869+0000 7fdeede00640 1 --2- 172.21.15.28:0/2678355868 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee809c570 0x7fdee80cfbc0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.869+0000 7fdeede00640 1 --2- 172.21.15.28:0/2678355868 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdee80cf7e0 0x7fdee80b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.869+0000 7fdeede00640 1 -- 172.21.15.28:0/2678355868 >> 172.21.15.28:0/2678355868 conn(0x7fdee809ba50 msgr2=0x7fdee809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:44.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.869+0000 7fdeede00640 1 -- 172.21.15.28:0/2678355868 shutdown_connections 2024-04-26T21:06:44.871 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.869+0000 7fdeede00640 1 -- 172.21.15.28:0/2678355868 wait complete. 2024-04-26T21:06:44.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.871+0000 7fdeede00640 1 Processor -- start 2024-04-26T21:06:44.872 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.871+0000 7fdeede00640 1 -- start start 2024-04-26T21:06:44.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.872+0000 7fdeede00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdee809c570 0x7fdee815a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:44.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.872+0000 7fdeede00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee80cf7e0 0x7fdee8157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:44.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.872+0000 7fdeede00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdee80d0100 0x7fdee81f7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:44.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.872+0000 7fdeede00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdee80d0570 con 0x7fdee80cf7e0 2024-04-26T21:06:44.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.872+0000 7fdeede00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdee80d3d00 con 0x7fdee80d0100 2024-04-26T21:06:44.873 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.872+0000 7fdeede00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fdee80b2b50 con 0x7fdee809c570 2024-04-26T21:06:44.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.872+0000 7fdee6000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee80cf7e0 0x7fdee8157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:44.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.872+0000 7fdee6000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee80cf7e0 0x7fdee8157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34082/0 (socket says 172.21.15.28:34082) 2024-04-26T21:06:44.874 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.872+0000 7fdee6000640 1 -- 172.21.15.28:0/1081070605 learned_addr learned my addr 172.21.15.28:0/1081070605 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:44.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.873+0000 7fdee7400640 1 --2- 172.21.15.28:0/1081070605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdee80d0100 0x7fdee81f7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:44.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.873+0000 7fdee6000640 1 -- 172.21.15.28:0/1081070605 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdee809c570 msgr2=0x7fdee815a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:44.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.873+0000 7fdee6000640 1 --2- 172.21.15.28:0/1081070605 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdee809c570 0x7fdee815a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.873+0000 7fdee6000640 1 -- 172.21.15.28:0/1081070605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdee80d0100 msgr2=0x7fdee81f7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:44.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.873+0000 7fdee6000640 1 --2- 172.21.15.28:0/1081070605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdee80d0100 0x7fdee81f7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:44.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.873+0000 7fdee6000640 1 -- 172.21.15.28:0/1081070605 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee81f8910 con 0x7fdee80cf7e0 2024-04-26T21:06:44.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.873+0000 7fdee6000640 1 --2- 172.21.15.28:0/1081070605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee80cf7e0 0x7fdee8157d80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fded4000c00 tx=0x7fded40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:44.875 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.873+0000 7fdedf400640 1 -- 172.21.15.28:0/1081070605 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4041020 con 0x7fdee80cf7e0 2024-04-26T21:06:44.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.873+0000 7fdedf400640 1 -- 172.21.15.28:0/1081070605 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded40306c0 con 0x7fdee80cf7e0 2024-04-26T21:06:44.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.873+0000 7fdedf400640 1 -- 172.21.15.28:0/1081070605 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded4037040 con 0x7fdee80cf7e0 2024-04-26T21:06:44.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.873+0000 7fdeede00640 1 -- 172.21.15.28:0/1081070605 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee81f8bf0 con 0x7fdee80cf7e0 2024-04-26T21:06:44.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.874+0000 7fdeede00640 1 -- 172.21.15.28:0/1081070605 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee8150cf0 con 0x7fdee80cf7e0 2024-04-26T21:06:44.876 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.875+0000 7fdeede00640 1 -- 172.21.15.28:0/1081070605 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee806a400 con 0x7fdee80cf7e0 2024-04-26T21:06:44.877 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.876+0000 7fdedf400640 1 -- 172.21.15.28:0/1081070605 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fded4044020 con 0x7fdee80cf7e0 2024-04-26T21:06:44.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.877+0000 7fdedf400640 1 --2- 172.21.15.28:0/1081070605 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fdec806c8f0 0x7fdec806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:44.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.877+0000 7fdedf400640 1 -- 172.21.15.28:0/1081070605 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fded40ac060 con 0x7fdee80cf7e0 2024-04-26T21:06:44.878 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.877+0000 7fdee6a00640 1 --2- 172.21.15.28:0/1081070605 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fdec806c8f0 0x7fdec806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:44.879 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.878+0000 7fdee6a00640 1 --2- 172.21.15.28:0/1081070605 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fdec806c8f0 0x7fdec806edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fded80098f0 tx=0x7fded8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:44.882 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:44.881+0000 7fdedf400640 1 -- 172.21.15.28:0/1081070605 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fded404b030 con 0x7fdee80cf7e0 2024-04-26T21:06:45.083 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:45.082+0000 7fdeede00640 1 -- 172.21.15.28:0/1081070605 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdee8068e00 con 0x7fdee80cf7e0 2024-04-26T21:06:45.085 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:45.084+0000 7fdedf400640 1 -- 172.21.15.28:0/1081070605 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+12996 (secure 0 0 0) 0x7fded407a780 con 0x7fdee80cf7e0 2024-04-26T21:06:45.085 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T21:06:45.085 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":50,"fsid":"79845bd7-2c94-4fa6-8082-292285d530c9","created":"2024-04-26T20:56:31.441671+0000","modified":"2024-04-26T21:06:41.311501+0000","last_up_change":"2024-04-26T20:56:43.392886+0000","last_in_change":"2024-04-26T20:56:34.747682+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T20:56:44.737461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"ae2c534d-cc92-4d2a-8e08-d341decd44ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6809","nonce":993399689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6811","nonce":993399689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6815","nonce":993399689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":993399689},{"type":"v1","addr":"172.21.15.28:6813","nonce":993399689}]},"public_addr":"172.21.15.28:6809/993399689","cluster_addr":"172.21.15.28:6811/993399689","heartbeat_back_addr":"172.21.15.28:6815/993399689","heartbeat_front_addr":"172.21.15.28:6813/993399689","state":["exists","up"]},{"osd":1,"uuid":"db8ba836-8a8b-49d2-9e2f-30a0b1e537a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6801","nonce":3928805442}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6803","nonce":3928805442}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6807","nonce":3928805442}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":3928805442},{"type":"v1","addr":"172.21.15.28:6805","nonce":3928805442}]},"public_addr":"172.21.15.28:6801/3928805442","cluster_addr":"172.21.15.28:6803/3928805442","heartbeat_back_addr":"172.21.15.28:6807/3928805442","heartbeat_front_addr":"172.21.15.28:6805/3928805442","state":["exists","up"]},{"osd":2,"uuid":"3e379b23-d512-4825-a8e8-8fa26f754094","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6817","nonce":2056655204}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6819","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6821","nonce":2056655204}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6827","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6829","nonce":2056655204}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6823","nonce":2056655204},{"type":"v1","addr":"172.21.15.28:6825","nonce":2056655204}]},"public_addr":"172.21.15.28:6817/2056655204","cluster_addr":"172.21.15.28:6821/2056655204","heartbeat_back_addr":"172.21.15.28:6829/2056655204","heartbeat_front_addr":"172.21.15.28:6825/2056655204","state":["exists","up"]},{"osd":3,"uuid":"85e64a2e-c12e-44cb-a4ef-74229ad82940","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6820","nonce":2569592417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6824","nonce":2569592417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6831","nonce":2569592417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":2569592417},{"type":"v1","addr":"172.21.15.28:6828","nonce":2569592417}]},"public_addr":"172.21.15.28:6820/2569592417","cluster_addr":"172.21.15.28:6824/2569592417","heartbeat_back_addr":"172.21.15.28:6831/2569592417","heartbeat_front_addr":"172.21.15.28:6828/2569592417","state":["exists","up"]},{"osd":4,"uuid":"5945cd04-1c07-4d65-8642-6f5ab2bfe237","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6817","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6819","nonce":472173438}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6821","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6822","nonce":472173438}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6827","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6829","nonce":472173438}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":472173438},{"type":"v1","addr":"172.21.15.103:6826","nonce":472173438}]},"public_addr":"172.21.15.103:6819/472173438","cluster_addr":"172.21.15.103:6822/472173438","heartbeat_back_addr":"172.21.15.103:6829/472173438","heartbeat_front_addr":"172.21.15.103:6826/472173438","state":["exists","up"]},{"osd":5,"uuid":"76406cbb-cc40-48c5-8698-5269d0ba54a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6801","nonce":4239183489}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6803","nonce":4239183489}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6807","nonce":4239183489}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":4239183489},{"type":"v1","addr":"172.21.15.103:6805","nonce":4239183489}]},"public_addr":"172.21.15.103:6801/4239183489","cluster_addr":"172.21.15.103:6803/4239183489","heartbeat_back_addr":"172.21.15.103:6807/4239183489","heartbeat_front_addr":"172.21.15.103:6805/4239183489","state":["exists","up"]},{"osd":6,"uuid":"23ff5c7d-1a9b-4e13-96b3-c399a9d22142","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6818","nonce":1745016180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6823","nonce":1745016180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6831","nonce":1745016180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6825","nonce":1745016180},{"type":"v1","addr":"172.21.15.103:6828","nonce":1745016180}]},"public_addr":"172.21.15.103:6818/1745016180","cluster_addr":"172.21.15.103:6823/1745016180","heartbeat_back_addr":"172.21.15.103:6831/1745016180","heartbeat_front_addr":"172.21.15.103:6828/1745016180","state":["exists","up"]},{"osd":7,"uuid":"26e56afd-dfe1-498a-bced-06e368861524","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6809","nonce":583391321}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6811","nonce":583391321}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6815","nonce":583391321}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":583391321},{"type":"v1","addr":"172.21.15.103:6813","nonce":583391321}]},"public_addr":"172.21.15.103:6809/583391321","cluster_addr":"172.21.15.103:6811/583391321","heartbeat_back_addr":"172.21.15.103:6815/583391321","heartbeat_front_addr":"172.21.15.103:6813/583391321","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.143586+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.302644+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.093024+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.459083+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.121041+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.019902+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.153485+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T20:56:40.040775+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.28:6837/2997047727":"2024-04-27T21:06:38.292886+0000","172.21.15.28:6836/2997047727":"2024-04-27T21:06:38.292886+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T21:06:45.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:45.086+0000 7fdeede00640 1 -- 172.21.15.28:0/1081070605 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fdec806c8f0 msgr2=0x7fdec806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:45.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:45.086+0000 7fdeede00640 1 --2- 172.21.15.28:0/1081070605 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fdec806c8f0 0x7fdec806edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fded80098f0 tx=0x7fded8008040 comp rx=0 tx=0).stop 2024-04-26T21:06:45.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:45.086+0000 7fdeede00640 1 -- 172.21.15.28:0/1081070605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee80cf7e0 msgr2=0x7fdee8157d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:45.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:45.086+0000 7fdeede00640 1 --2- 172.21.15.28:0/1081070605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee80cf7e0 0x7fdee8157d80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fded4000c00 tx=0x7fded40046c0 comp rx=0 tx=0).stop 2024-04-26T21:06:45.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:45.086+0000 7fdeede00640 1 -- 172.21.15.28:0/1081070605 shutdown_connections 2024-04-26T21:06:45.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:45.086+0000 7fdeede00640 1 --2- 172.21.15.28:0/1081070605 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fdec806c8f0 0x7fdec806edb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:45.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:45.086+0000 7fdeede00640 1 --2- 172.21.15.28:0/1081070605 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdee80d0100 0x7fdee81f7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:45.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:45.086+0000 7fdeede00640 1 --2- 172.21.15.28:0/1081070605 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee80cf7e0 0x7fdee8157d80 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:45.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:45.086+0000 7fdeede00640 1 --2- 172.21.15.28:0/1081070605 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdee809c570 0x7fdee815a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:45.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:45.086+0000 7fdeede00640 1 -- 172.21.15.28:0/1081070605 >> 172.21.15.28:0/1081070605 conn(0x7fdee809ba50 msgr2=0x7fdee8071990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:45.086+0000 7fdeede00640 1 -- 172.21.15.28:0/1081070605 shutdown_connections 2024-04-26T21:06:45.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:45.087+0000 7fdeede00640 1 -- 172.21.15.28:0/1081070605 wait complete. 2024-04-26T21:06:46.106 INFO:tasks.ceph:Scrubbing osd.0 2024-04-26T21:06:46.106 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:06:46.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.333+0000 7f60d9e00640 1 Processor -- start 2024-04-26T21:06:46.334 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.333+0000 7f60d9e00640 1 -- start start 2024-04-26T21:06:46.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.334+0000 7f60d9e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f60d40705d0 0x7f60d4163870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:46.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.334+0000 7f60d9e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f60d40b5f80 0x7f60d40709b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:46.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.334+0000 7f60d9e00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60d4163db0 con 0x7f60d415f890 2024-04-26T21:06:46.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.334+0000 7f60d9e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f60d4070ef0 con 0x7f60d40b5f80 2024-04-26T21:06:46.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.334+0000 7f60d9e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f60d4160330 con 0x7f60d40705d0 2024-04-26T21:06:46.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.335+0000 7f60d2000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f60d40b5f80 0x7f60d40709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:46.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.335+0000 7f60d2000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f60d40b5f80 0x7f60d40709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34090/0 (socket says 172.21.15.28:34090) 2024-04-26T21:06:46.336 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.335+0000 7f60d2000640 1 -- 172.21.15.28:0/2780957774 learned_addr learned my addr 172.21.15.28:0/2780957774 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:46.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.335+0000 7f60d2000640 1 -- 172.21.15.28:0/2780957774 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f60d40705d0 msgr2=0x7f60d4163870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:46.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.335+0000 7f60d2000640 1 --2- 172.21.15.28:0/2780957774 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f60d40705d0 0x7f60d4163870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:46.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.335+0000 7f60d2000640 1 -- 172.21.15.28:0/2780957774 >> v1:172.21.15.103:6789/0 conn(0x7f60d415f890 legacy=0x7f60d406c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:46.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.335+0000 7f60d2000640 1 -- 172.21.15.28:0/2780957774 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d4160610 con 0x7f60d40b5f80 2024-04-26T21:06:46.337 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.337+0000 7f60d2000640 1 --2- 172.21.15.28:0/2780957774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60d40b5f80 0x7f60d40709b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009870 tx=0x7f60bc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807dde7f0d3f9d0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:46.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.337+0000 7f60d1600640 1 -- 172.21.15.28:0/2780957774 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc037070 con 0x7f60d40b5f80 2024-04-26T21:06:46.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.337+0000 7f60d1600640 1 -- 172.21.15.28:0/2780957774 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60bc032440 con 0x7f60d40b5f80 2024-04-26T21:06:46.338 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.337+0000 7f60d1600640 1 -- 172.21.15.28:0/2780957774 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc032810 con 0x7f60d40b5f80 2024-04-26T21:06:46.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.338+0000 7f60d9e00640 1 -- 172.21.15.28:0/2780957774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60d40b5f80 msgr2=0x7f60d40709b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:46.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.338+0000 7f60d9e00640 1 --2- 172.21.15.28:0/2780957774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60d40b5f80 0x7f60d40709b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009870 tx=0x7f60bc02a210 comp rx=0 tx=0).stop 2024-04-26T21:06:46.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.338+0000 7f60d9e00640 1 -- 172.21.15.28:0/2780957774 shutdown_connections 2024-04-26T21:06:46.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.338+0000 7f60d9e00640 1 --2- 172.21.15.28:0/2780957774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60d40b5f80 0x7f60d40709b0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:46.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.338+0000 7f60d9e00640 1 --2- 172.21.15.28:0/2780957774 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f60d40705d0 0x7f60d4163870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:46.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.338+0000 7f60d9e00640 1 -- 172.21.15.28:0/2780957774 >> 172.21.15.28:0/2780957774 conn(0x7f60d40b4b80 msgr2=0x7f60d40b4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:46.339 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.338+0000 7f60d9e00640 1 -- 172.21.15.28:0/2780957774 shutdown_connections 2024-04-26T21:06:46.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.338+0000 7f60d9e00640 1 -- 172.21.15.28:0/2780957774 wait complete. 2024-04-26T21:06:46.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.339+0000 7f60d9e00640 1 Processor -- start 2024-04-26T21:06:46.340 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.340+0000 7f60d9e00640 1 -- start start 2024-04-26T21:06:46.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.341+0000 7f60d9e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60d40705d0 0x7f60d4142b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:46.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.341+0000 7f60d9e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60d40b5f80 0x7f60d413f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:46.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.341+0000 7f60d9e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f60d415f890 0x7f60d4155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:46.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.341+0000 7f60d9e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f60d415d290 con 0x7f60d40b5f80 2024-04-26T21:06:46.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.341+0000 7f60d9e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f60d415d110 con 0x7f60d40705d0 2024-04-26T21:06:46.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.341+0000 7f60d9e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f60d415fd00 con 0x7f60d415f890 2024-04-26T21:06:46.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.341+0000 7f60d2000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60d40b5f80 0x7f60d413f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:46.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.341+0000 7f60d2000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60d40b5f80 0x7f60d413f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34106/0 (socket says 172.21.15.28:34106) 2024-04-26T21:06:46.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.341+0000 7f60d2000640 1 -- 172.21.15.28:0/1218130966 learned_addr learned my addr 172.21.15.28:0/1218130966 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:46.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.341+0000 7f60d2a00640 1 --2- 172.21.15.28:0/1218130966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60d40705d0 0x7f60d4142b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:46.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.341+0000 7f60d2000640 1 -- 172.21.15.28:0/1218130966 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f60d415f890 msgr2=0x7f60d4155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:46.343 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.341+0000 7f60d2000640 1 --2- 172.21.15.28:0/1218130966 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f60d415f890 0x7f60d4155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:46.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.341+0000 7f60d2000640 1 -- 172.21.15.28:0/1218130966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60d40705d0 msgr2=0x7f60d4142b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:46.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.341+0000 7f60d2000640 1 --2- 172.21.15.28:0/1218130966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60d40705d0 0x7f60d4142b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:46.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.341+0000 7f60d2000640 1 -- 172.21.15.28:0/1218130966 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60d413fca0 con 0x7f60d40b5f80 2024-04-26T21:06:46.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.341+0000 7f60d2a00640 1 --2- 172.21.15.28:0/1218130966 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f60d40705d0 0x7f60d4142b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:46.344 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.342+0000 7f60d2000640 1 --2- 172.21.15.28:0/1218130966 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60d40b5f80 0x7f60d413f760 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009870 tx=0x7f60bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:46.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.342+0000 7f60cb400640 1 -- 172.21.15.28:0/1218130966 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc041070 con 0x7f60d40b5f80 2024-04-26T21:06:46.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.342+0000 7f60cb400640 1 -- 172.21.15.28:0/1218130966 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60bc03c930 con 0x7f60d40b5f80 2024-04-26T21:06:46.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.342+0000 7f60cb400640 1 -- 172.21.15.28:0/1218130966 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60bc037040 con 0x7f60d40b5f80 2024-04-26T21:06:46.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.342+0000 7f60d9e00640 1 -- 172.21.15.28:0/1218130966 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60d4156590 con 0x7f60d40b5f80 2024-04-26T21:06:46.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.342+0000 7f60d9e00640 1 -- 172.21.15.28:0/1218130966 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60d4156b50 con 0x7f60d40b5f80 2024-04-26T21:06:46.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.343+0000 7f60d9e00640 1 -- 172.21.15.28:0/1218130966 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f60d4076520 con 0x7f60d40b5f80 2024-04-26T21:06:46.345 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.344+0000 7f60cb400640 1 -- 172.21.15.28:0/1218130966 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f60bc044070 con 0x7f60d40b5f80 2024-04-26T21:06:46.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.345+0000 7f60cb400640 1 --2- 172.21.15.28:0/1218130966 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f60b006c8f0 0x7f60b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:46.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.345+0000 7f60cb400640 1 -- 172.21.15.28:0/1218130966 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f60bc0ac330 con 0x7f60d40b5f80 2024-04-26T21:06:46.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.346+0000 7f60d2a00640 1 --2- 172.21.15.28:0/1218130966 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f60b006c8f0 0x7f60b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:46.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.346+0000 7f60cb400640 1 --2- 172.21.15.28:0/1218130966 >> [v2:172.21.15.28:6808/993399689,v1:172.21.15.28:6809/993399689] conn(0x7f60b00727b0 0x7f60b0074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:46.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.346+0000 7f60cb400640 1 -- 172.21.15.28:0/1218130966 --> [v2:172.21.15.28:6808/993399689,v1:172.21.15.28:6809/993399689] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f60bc04be80 con 0x7f60b00727b0 2024-04-26T21:06:46.347 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.346+0000 7f60cb400640 1 -- 172.21.15.28:0/1218130966 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f60bc0b3050 con 0x7f60d40b5f80 2024-04-26T21:06:46.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.346+0000 7f60d3400640 1 --2- 172.21.15.28:0/1218130966 >> [v2:172.21.15.28:6808/993399689,v1:172.21.15.28:6809/993399689] conn(0x7f60b00727b0 0x7f60b0074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:46.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.346+0000 7f60d2a00640 1 --2- 172.21.15.28:0/1218130966 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f60b006c8f0 0x7f60b006edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f60c4006fd0 tx=0x7f60c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:46.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.347+0000 7f60d3400640 1 --2- 172.21.15.28:0/1218130966 >> [v2:172.21.15.28:6808/993399689,v1:172.21.15.28:6809/993399689] conn(0x7f60b00727b0 0x7f60b0074bd0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:46.348 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.347+0000 7f60cb400640 1 -- 172.21.15.28:0/1218130966 <== osd.0 v2:172.21.15.28:6808/993399689 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f60bc04be80 con 0x7f60b00727b0 2024-04-26T21:06:46.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.363+0000 7f60d9e00640 1 -- 172.21.15.28:0/1218130966 --> [v2:172.21.15.28:6808/993399689,v1:172.21.15.28:6809/993399689] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f60d41e9fe0 con 0x7f60b00727b0 2024-04-26T21:06:46.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.363+0000 7f60cb400640 1 -- 172.21.15.28:0/1218130966 <== osd.0 v2:172.21.15.28:6808/993399689 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f60d41e9fe0 con 0x7f60b00727b0 2024-04-26T21:06:46.364 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T21:06:46.364 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:06:46.365 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T21:06:46.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.363+0000 7f60d9e00640 1 -- 172.21.15.28:0/1218130966 >> [v2:172.21.15.28:6808/993399689,v1:172.21.15.28:6809/993399689] conn(0x7f60b00727b0 msgr2=0x7f60b0074bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:46.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.364+0000 7f60d9e00640 1 --2- 172.21.15.28:0/1218130966 >> [v2:172.21.15.28:6808/993399689,v1:172.21.15.28:6809/993399689] conn(0x7f60b00727b0 0x7f60b0074bd0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:46.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.364+0000 7f60d9e00640 1 -- 172.21.15.28:0/1218130966 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f60b006c8f0 msgr2=0x7f60b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:46.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.364+0000 7f60d9e00640 1 --2- 172.21.15.28:0/1218130966 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f60b006c8f0 0x7f60b006edb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f60c4006fd0 tx=0x7f60c4008040 comp rx=0 tx=0).stop 2024-04-26T21:06:46.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.364+0000 7f60d9e00640 1 -- 172.21.15.28:0/1218130966 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60d40b5f80 msgr2=0x7f60d413f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:46.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.364+0000 7f60d9e00640 1 --2- 172.21.15.28:0/1218130966 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60d40b5f80 0x7f60d413f760 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f60bc009870 tx=0x7f60bc0046c0 comp rx=0 tx=0).stop 2024-04-26T21:06:46.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.364+0000 7f60d3400640 1 -- 172.21.15.28:0/1218130966 reap_dead start 2024-04-26T21:06:46.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.364+0000 7f60d9e00640 1 -- 172.21.15.28:0/1218130966 shutdown_connections 2024-04-26T21:06:46.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.364+0000 7f60d9e00640 1 -- 172.21.15.28:0/1218130966 >> 172.21.15.28:0/1218130966 conn(0x7f60d40b4b80 msgr2=0x7f60d415edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:46.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.364+0000 7f60d9e00640 1 -- 172.21.15.28:0/1218130966 shutdown_connections 2024-04-26T21:06:46.366 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.364+0000 7f60d9e00640 1 -- 172.21.15.28:0/1218130966 wait complete. 2024-04-26T21:06:46.382 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-26T21:06:46.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.592+0000 7f9975000640 1 Processor -- start 2024-04-26T21:06:46.593 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.592+0000 7f9975000640 1 -- start start 2024-04-26T21:06:46.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.594+0000 7f9975000640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f9970146670 0x7f9970071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:46.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.594+0000 7f9975000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9970146440 con 0x7f99701610c0 2024-04-26T21:06:46.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.594+0000 7f9975000640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f997006f010 con 0x7f9970146060 2024-04-26T21:06:46.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.594+0000 7f9975000640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f997006f510 con 0x7f9970146670 2024-04-26T21:06:46.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.594+0000 7f996fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f99701610c0 0x7f99700cb0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40230/0 (socket says 172.21.15.28:40230) 2024-04-26T21:06:46.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.594+0000 7f996fe00640 1 -- 172.21.15.28:0/2326770364 learned_addr learned my addr 172.21.15.28:0/2326770364 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:46.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.595+0000 7f996ea00640 1 --2- 172.21.15.28:0/2326770364 >> v2:172.21.15.103:3300/0 conn(0x7f9970146670 0x7f9970071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:46.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.595+0000 7f996e000640 1 -- 172.21.15.28:0/2326770364 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2030369544 0 0) 0x7f9970146440 con 0x7f99701610c0 2024-04-26T21:06:46.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.595+0000 7f996e000640 1 -- 172.21.15.28:0/2326770364 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9954003520 con 0x7f99701610c0 2024-04-26T21:06:46.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.595+0000 7f996e000640 1 -- 172.21.15.28:0/2326770364 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 744623341 0 0) 0x7f9954003520 con 0x7f99701610c0 2024-04-26T21:06:46.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.595+0000 7f996e000640 1 -- 172.21.15.28:0/2326770364 >> v2:172.21.15.103:3300/0 conn(0x7f9970146670 msgr2=0x7f9970071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:46.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.595+0000 7f996e000640 1 --2- 172.21.15.28:0/2326770364 >> v2:172.21.15.103:3300/0 conn(0x7f9970146670 0x7f9970071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:46.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.595+0000 7f996e000640 1 -- 172.21.15.28:0/2326770364 >> v1:172.21.15.103:6789/0 conn(0x7f9970146060 legacy=0x7f99701609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:46.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.595+0000 7f996e000640 1 -- 172.21.15.28:0/2326770364 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99701614d0 con 0x7f99701610c0 2024-04-26T21:06:46.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.595+0000 7f996e000640 1 -- 172.21.15.28:0/2326770364 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f99680019f0 con 0x7f99701610c0 2024-04-26T21:06:46.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.595+0000 7f996e000640 1 -- 172.21.15.28:0/2326770364 >> v1:172.21.15.28:6789/0 conn(0x7f99701610c0 legacy=0x7f99700cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:46.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.595+0000 7f996e000640 1 --2- 172.21.15.28:0/2326770364 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f99540039b0 0x7f9954003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:46.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.595+0000 7f996e000640 1 --2- 172.21.15.28:0/2326770364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9954004800 0x7f9954004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:46.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.595+0000 7f996e000640 1 --2- 172.21.15.28:0/2326770364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99540051c0 0x7f995400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:46.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.595+0000 7f996e000640 1 -- 172.21.15.28:0/2326770364 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f997006f510 con 0x7f9954004800 2024-04-26T21:06:46.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.595+0000 7f996e000640 1 -- 172.21.15.28:0/2326770364 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f995400c160 con 0x7f99540051c0 2024-04-26T21:06:46.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.595+0000 7f996e000640 1 -- 172.21.15.28:0/2326770364 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f995400c360 con 0x7f99540039b0 2024-04-26T21:06:46.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.596+0000 7f996ea00640 1 --2- 172.21.15.28:0/2326770364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9954004800 0x7f9954004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:46.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.596+0000 7f996ea00640 1 -- 172.21.15.28:0/2326770364 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f99540039b0 msgr2=0x7f9954003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:46.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.596+0000 7f996fe00640 1 --2- 172.21.15.28:0/2326770364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99540051c0 0x7f995400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:46.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.596+0000 7f996f400640 1 --2- 172.21.15.28:0/2326770364 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f99540039b0 0x7f9954003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:46.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.596+0000 7f996ea00640 1 --2- 172.21.15.28:0/2326770364 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f99540039b0 0x7f9954003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:46.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.596+0000 7f996ea00640 1 -- 172.21.15.28:0/2326770364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99540051c0 msgr2=0x7f995400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:46.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.596+0000 7f996ea00640 1 --2- 172.21.15.28:0/2326770364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99540051c0 0x7f995400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:46.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.596+0000 7f996ea00640 1 -- 172.21.15.28:0/2326770364 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9954003520 con 0x7f9954004800 2024-04-26T21:06:46.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.596+0000 7f996fe00640 1 --2- 172.21.15.28:0/2326770364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99540051c0 0x7f995400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:46.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.596+0000 7f996fe00640 1 -- 172.21.15.28:0/2326770364 reap_dead start 2024-04-26T21:06:46.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.597+0000 7f996ea00640 1 --2- 172.21.15.28:0/2326770364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9954004800 0x7f9954004c80 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f995c009ac0 tx=0x7f9954005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68864d6df0735f42 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:46.600 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.597+0000 7f996e000640 1 -- 172.21.15.28:0/2326770364 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c036070 con 0x7f9954004800 2024-04-26T21:06:46.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.597+0000 7f996e000640 1 -- 172.21.15.28:0/2326770364 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f995c031400 con 0x7f9954004800 2024-04-26T21:06:46.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.597+0000 7f996e000640 1 -- 172.21.15.28:0/2326770364 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995c009690 con 0x7f9954004800 2024-04-26T21:06:46.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.598+0000 7f9975000640 1 -- 172.21.15.28:0/2326770364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9954004800 msgr2=0x7f9954004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:46.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.598+0000 7f9975000640 1 --2- 172.21.15.28:0/2326770364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9954004800 0x7f9954004c80 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f995c009ac0 tx=0x7f9954005830 comp rx=0 tx=0).stop 2024-04-26T21:06:46.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.598+0000 7f9975000640 1 -- 172.21.15.28:0/2326770364 shutdown_connections 2024-04-26T21:06:46.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.598+0000 7f9975000640 1 --2- 172.21.15.28:0/2326770364 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9954004800 0x7f9954004c80 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:46.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.598+0000 7f9975000640 1 -- 172.21.15.28:0/2326770364 >> 172.21.15.28:0/2326770364 conn(0x7f99700d0a50 msgr2=0x7f99700d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:46.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.598+0000 7f9975000640 1 -- 172.21.15.28:0/2326770364 shutdown_connections 2024-04-26T21:06:46.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.598+0000 7f9975000640 1 -- 172.21.15.28:0/2326770364 wait complete. 2024-04-26T21:06:46.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.600+0000 7f9975000640 1 Processor -- start 2024-04-26T21:06:46.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.600+0000 7f9975000640 1 -- start start 2024-04-26T21:06:46.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.601+0000 7f9975000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9954004800 0x7f9970158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:46.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.601+0000 7f9975000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f99701f7b70 0x7f99700723b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:46.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.601+0000 7f9975000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99700728f0 0x7f99701572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:46.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.601+0000 7f9975000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f997006ff60 con 0x7f9954004800 2024-04-26T21:06:46.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.601+0000 7f9975000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f997006fc80 con 0x7f99700728f0 2024-04-26T21:06:46.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.601+0000 7f9975000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9970071e20 con 0x7f99701f7b70 2024-04-26T21:06:46.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.601+0000 7f996f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9954004800 0x7f9970158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:46.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.601+0000 7f996f400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9954004800 0x7f9970158810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34118/0 (socket says 172.21.15.28:34118) 2024-04-26T21:06:46.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.601+0000 7f996f400640 1 -- 172.21.15.28:0/2212859094 learned_addr learned my addr 172.21.15.28:0/2212859094 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:46.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.602+0000 7f996f400640 1 -- 172.21.15.28:0/2212859094 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f99701f7b70 msgr2=0x7f99700723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:46.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.602+0000 7f996ea00640 1 --2- 172.21.15.28:0/2212859094 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f99701f7b70 0x7f99700723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:46.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.602+0000 7f996fe00640 1 --2- 172.21.15.28:0/2212859094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99700728f0 0x7f99701572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:46.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.602+0000 7f996f400640 1 --2- 172.21.15.28:0/2212859094 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f99701f7b70 0x7f99700723b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:46.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.602+0000 7f996f400640 1 -- 172.21.15.28:0/2212859094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99700728f0 msgr2=0x7f99701572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:46.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.602+0000 7f996f400640 1 --2- 172.21.15.28:0/2212859094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99700728f0 0x7f99701572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:46.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.602+0000 7f996f400640 1 -- 172.21.15.28:0/2212859094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99700ce6c0 con 0x7f9954004800 2024-04-26T21:06:46.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.602+0000 7f996f400640 1 --2- 172.21.15.28:0/2212859094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9954004800 0x7f9970158810 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f996000d4b0 tx=0x7f996000d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:46.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.602+0000 7f9967400640 1 -- 172.21.15.28:0/2212859094 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960005480 con 0x7f9954004800 2024-04-26T21:06:46.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.602+0000 7f9967400640 1 -- 172.21.15.28:0/2212859094 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9960017070 con 0x7f9954004800 2024-04-26T21:06:46.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.602+0000 7f9967400640 1 -- 172.21.15.28:0/2212859094 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960003810 con 0x7f9954004800 2024-04-26T21:06:46.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.602+0000 7f9975000640 1 -- 172.21.15.28:0/2212859094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9970157fb0 con 0x7f9954004800 2024-04-26T21:06:46.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.603+0000 7f9975000640 1 -- 172.21.15.28:0/2212859094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99700ce960 con 0x7f9954004800 2024-04-26T21:06:46.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.604+0000 7f9975000640 1 -- 172.21.15.28:0/2212859094 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f997006a360 con 0x7f9954004800 2024-04-26T21:06:46.606 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.605+0000 7f9967400640 1 -- 172.21.15.28:0/2212859094 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f99600039b0 con 0x7f9954004800 2024-04-26T21:06:46.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.606+0000 7f9967400640 1 --2- 172.21.15.28:0/2212859094 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f995006c920 0x7f995006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:46.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.606+0000 7f9967400640 1 -- 172.21.15.28:0/2212859094 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f996008b600 con 0x7f9954004800 2024-04-26T21:06:46.607 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.606+0000 7f996ea00640 1 --2- 172.21.15.28:0/2212859094 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f995006c920 0x7f995006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:46.608 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.607+0000 7f996ea00640 1 --2- 172.21.15.28:0/2212859094 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f995006c920 0x7f995006ede0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f995c033000 tx=0x7f995c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:46.612 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.611+0000 7f9967400640 1 -- 172.21.15.28:0/2212859094 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9960059df0 con 0x7f9954004800 2024-04-26T21:06:46.817 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.816+0000 7f9975000640 1 -- 172.21.15.28:0/2212859094 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f9970072cd0 con 0x7f995006c920 2024-04-26T21:06:46.818 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.817+0000 7f9967400640 1 -- 172.21.15.28:0/2212859094 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9970072cd0 con 0x7f995006c920 2024-04-26T21:06:46.818 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 0 to deep-scrub 2024-04-26T21:06:46.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.819+0000 7f9975000640 1 -- 172.21.15.28:0/2212859094 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f995006c920 msgr2=0x7f995006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:46.820 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.819+0000 7f9975000640 1 --2- 172.21.15.28:0/2212859094 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f995006c920 0x7f995006ede0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f995c033000 tx=0x7f995c008030 comp rx=0 tx=0).stop 2024-04-26T21:06:46.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.819+0000 7f9975000640 1 -- 172.21.15.28:0/2212859094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9954004800 msgr2=0x7f9970158810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:46.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.819+0000 7f9975000640 1 --2- 172.21.15.28:0/2212859094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9954004800 0x7f9970158810 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f996000d4b0 tx=0x7f996000d970 comp rx=0 tx=0).stop 2024-04-26T21:06:46.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.819+0000 7f9975000640 1 -- 172.21.15.28:0/2212859094 shutdown_connections 2024-04-26T21:06:46.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.819+0000 7f9975000640 1 --2- 172.21.15.28:0/2212859094 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f995006c920 0x7f995006ede0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:46.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.819+0000 7f9975000640 1 --2- 172.21.15.28:0/2212859094 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99700728f0 0x7f99701572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:46.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.819+0000 7f9975000640 1 --2- 172.21.15.28:0/2212859094 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f99701f7b70 0x7f99700723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:46.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.819+0000 7f9975000640 1 --2- 172.21.15.28:0/2212859094 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9954004800 0x7f9970158810 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:46.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.819+0000 7f9975000640 1 -- 172.21.15.28:0/2212859094 >> 172.21.15.28:0/2212859094 conn(0x7f99700d0a50 msgr2=0x7f99700cc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:46.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.819+0000 7f9975000640 1 -- 172.21.15.28:0/2212859094 shutdown_connections 2024-04-26T21:06:46.821 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:46.819+0000 7f9975000640 1 -- 172.21.15.28:0/2212859094 wait complete. 2024-04-26T21:06:46.841 INFO:tasks.ceph:Scrubbing osd.1 2024-04-26T21:06:46.841 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:06:47.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.054+0000 7fa349200640 1 Processor -- start 2024-04-26T21:06:47.055 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.054+0000 7fa349200640 1 -- start start 2024-04-26T21:06:47.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.056+0000 7fa349200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3440ceac0 0x7fa3440af260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.056+0000 7fa349200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fa3440d01a0 0x7fa344162b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.056+0000 7fa349200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa34406f8b0 con 0x7fa3440cd500 2024-04-26T21:06:47.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.056+0000 7fa349200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fa34406fb90 con 0x7fa3440d01a0 2024-04-26T21:06:47.057 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.056+0000 7fa349200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa3440aea60 con 0x7fa3440ceac0 2024-04-26T21:06:47.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.056+0000 7fa343400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa3440cd500 0x7fa3440cd8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40244/0 (socket says 172.21.15.28:40244) 2024-04-26T21:06:47.058 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.056+0000 7fa343400640 1 -- 172.21.15.28:0/293709765 learned_addr learned my addr 172.21.15.28:0/293709765 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:47.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.057+0000 7fa343e00640 1 --2- 172.21.15.28:0/293709765 >> v2:172.21.15.103:3300/0 conn(0x7fa3440d01a0 0x7fa344162b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.057+0000 7fa342a00640 1 --2- 172.21.15.28:0/293709765 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3440ceac0 0x7fa3440af260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.057+0000 7fa342000640 1 -- 172.21.15.28:0/293709765 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2764044936 0 0) 0x7fa34406f8b0 con 0x7fa3440cd500 2024-04-26T21:06:47.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.057+0000 7fa342000640 1 -- 172.21.15.28:0/293709765 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa328003520 con 0x7fa3440cd500 2024-04-26T21:06:47.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.057+0000 7fa342000640 1 -- 172.21.15.28:0/293709765 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4082978100 0 0) 0x7fa328003520 con 0x7fa3440cd500 2024-04-26T21:06:47.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.057+0000 7fa342000640 1 -- 172.21.15.28:0/293709765 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3440ceac0 msgr2=0x7fa3440af260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:47.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.057+0000 7fa342000640 1 --2- 172.21.15.28:0/293709765 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3440ceac0 0x7fa3440af260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.057+0000 7fa342000640 1 -- 172.21.15.28:0/293709765 >> v2:172.21.15.103:3300/0 conn(0x7fa3440d01a0 msgr2=0x7fa344162b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:47.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.057+0000 7fa342000640 1 --2- 172.21.15.28:0/293709765 >> v2:172.21.15.103:3300/0 conn(0x7fa3440d01a0 0x7fa344162b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.057+0000 7fa342000640 1 -- 172.21.15.28:0/293709765 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3440aed40 con 0x7fa3440cd500 2024-04-26T21:06:47.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.057+0000 7fa342000640 1 -- 172.21.15.28:0/293709765 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7fa334001670 con 0x7fa3440cd500 2024-04-26T21:06:47.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.057+0000 7fa342000640 1 -- 172.21.15.28:0/293709765 >> v1:172.21.15.28:6789/0 conn(0x7fa3440cd500 legacy=0x7fa3440cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.057+0000 7fa342000640 1 --2- 172.21.15.28:0/293709765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3280039b0 0x7fa328003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.060 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.058+0000 7fa342000640 1 --2- 172.21.15.28:0/293709765 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa328004800 0x7fa328004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.058+0000 7fa342000640 1 --2- 172.21.15.28:0/293709765 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3440ceac0 0x7fa3280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.058+0000 7fa342000640 1 -- 172.21.15.28:0/293709765 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa34406fb90 con 0x7fa328004800 2024-04-26T21:06:47.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.058+0000 7fa342000640 1 -- 172.21.15.28:0/293709765 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3440aea60 con 0x7fa3280039b0 2024-04-26T21:06:47.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.058+0000 7fa342000640 1 -- 172.21.15.28:0/293709765 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa32800b9c0 con 0x7fa3440ceac0 2024-04-26T21:06:47.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.058+0000 7fa342a00640 1 --2- 172.21.15.28:0/293709765 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa328004800 0x7fa328004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.058+0000 7fa343e00640 1 --2- 172.21.15.28:0/293709765 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3440ceac0 0x7fa3280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.058+0000 7fa342a00640 1 -- 172.21.15.28:0/293709765 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3440ceac0 msgr2=0x7fa3280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:47.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.058+0000 7fa342a00640 1 --2- 172.21.15.28:0/293709765 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3440ceac0 0x7fa3280051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.058+0000 7fa342a00640 1 -- 172.21.15.28:0/293709765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3280039b0 msgr2=0x7fa328003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:47.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.058+0000 7fa342a00640 1 --2- 172.21.15.28:0/293709765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3280039b0 0x7fa328003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.061 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.058+0000 7fa342a00640 1 -- 172.21.15.28:0/293709765 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa328003520 con 0x7fa328004800 2024-04-26T21:06:47.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.059+0000 7fa342a00640 1 --2- 172.21.15.28:0/293709765 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa328004800 0x7fa328004c80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009af0 tx=0x7fa328005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cc40ad3023d5be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:47.062 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.060+0000 7fa342000640 1 -- 172.21.15.28:0/293709765 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c036070 con 0x7fa328004800 2024-04-26T21:06:47.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.060+0000 7fa342000640 1 -- 172.21.15.28:0/293709765 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa32c031400 con 0x7fa328004800 2024-04-26T21:06:47.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.060+0000 7fa342000640 1 -- 172.21.15.28:0/293709765 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32c009690 con 0x7fa328004800 2024-04-26T21:06:47.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.060+0000 7fa349200640 1 -- 172.21.15.28:0/293709765 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa328004800 msgr2=0x7fa328004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.060+0000 7fa349200640 1 --2- 172.21.15.28:0/293709765 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa328004800 0x7fa328004c80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009af0 tx=0x7fa328005830 comp rx=0 tx=0).stop 2024-04-26T21:06:47.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.060+0000 7fa343e00640 1 -- 172.21.15.28:0/293709765 reap_dead start 2024-04-26T21:06:47.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.060+0000 7fa349200640 1 -- 172.21.15.28:0/293709765 shutdown_connections 2024-04-26T21:06:47.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.060+0000 7fa349200640 1 -- 172.21.15.28:0/293709765 >> 172.21.15.28:0/293709765 conn(0x7fa3440b2820 msgr2=0x7fa3440b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:47.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.060+0000 7fa349200640 1 -- 172.21.15.28:0/293709765 shutdown_connections 2024-04-26T21:06:47.063 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.060+0000 7fa349200640 1 -- 172.21.15.28:0/293709765 wait complete. 2024-04-26T21:06:47.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.062+0000 7fa349200640 1 Processor -- start 2024-04-26T21:06:47.064 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.062+0000 7fa349200640 1 -- start start 2024-04-26T21:06:47.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.063+0000 7fa349200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa328004800 0x7fa3441e8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.063+0000 7fa349200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3440b3910 0x7fa3440b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.063+0000 7fa349200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3441f79c0 0x7fa3441f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.063+0000 7fa349200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa3440d07d0 con 0x7fa328004800 2024-04-26T21:06:47.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.063+0000 7fa349200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3440d0650 con 0x7fa3441f79c0 2024-04-26T21:06:47.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.063+0000 7fa349200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa3440af260 con 0x7fa3440b3910 2024-04-26T21:06:47.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.064+0000 7fa343400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa328004800 0x7fa3441e8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.064+0000 7fa343400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa328004800 0x7fa3441e8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34134/0 (socket says 172.21.15.28:34134) 2024-04-26T21:06:47.065 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.064+0000 7fa343400640 1 -- 172.21.15.28:0/655245140 learned_addr learned my addr 172.21.15.28:0/655245140 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:47.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.064+0000 7fa342a00640 1 --2- 172.21.15.28:0/655245140 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3440b3910 0x7fa3440b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.066 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.064+0000 7fa343400640 1 -- 172.21.15.28:0/655245140 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3440b3910 msgr2=0x7fa3440b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.064+0000 7fa343400640 1 --2- 172.21.15.28:0/655245140 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3440b3910 0x7fa3440b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.064+0000 7fa343400640 1 -- 172.21.15.28:0/655245140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3441f79c0 msgr2=0x7fa3441f7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.064+0000 7fa343400640 1 --2- 172.21.15.28:0/655245140 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa3441f79c0 0x7fa3441f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.064+0000 7fa343400640 1 -- 172.21.15.28:0/655245140 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3440b43d0 con 0x7fa328004800 2024-04-26T21:06:47.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.064+0000 7fa342a00640 1 --2- 172.21.15.28:0/655245140 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa3440b3910 0x7fa3440b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:47.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.065+0000 7fa343400640 1 --2- 172.21.15.28:0/655245140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa328004800 0x7fa3441e8c40 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa33400c710 tx=0x7fa33400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:47.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.065+0000 7fa33b400640 1 -- 172.21.15.28:0/655245140 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334018070 con 0x7fa328004800 2024-04-26T21:06:47.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.065+0000 7fa33b400640 1 -- 172.21.15.28:0/655245140 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3340074e0 con 0x7fa328004800 2024-04-26T21:06:47.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.065+0000 7fa33b400640 1 -- 172.21.15.28:0/655245140 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334004a20 con 0x7fa328004800 2024-04-26T21:06:47.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.065+0000 7fa349200640 1 -- 172.21.15.28:0/655245140 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3440b46b0 con 0x7fa328004800 2024-04-26T21:06:47.067 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.065+0000 7fa349200640 1 -- 172.21.15.28:0/655245140 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3440b4c70 con 0x7fa328004800 2024-04-26T21:06:47.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.066+0000 7fa349200640 1 -- 172.21.15.28:0/655245140 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa34406a3d0 con 0x7fa328004800 2024-04-26T21:06:47.068 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.067+0000 7fa33b400640 1 -- 172.21.15.28:0/655245140 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa33401b070 con 0x7fa328004800 2024-04-26T21:06:47.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.068+0000 7fa33b400640 1 --2- 172.21.15.28:0/655245140 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa32006c920 0x7fa32006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.069 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.068+0000 7fa33b400640 1 -- 172.21.15.28:0/655245140 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fa33408ae10 con 0x7fa328004800 2024-04-26T21:06:47.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.068+0000 7fa33b400640 1 --2- 172.21.15.28:0/655245140 >> [v2:172.21.15.28:6800/3928805442,v1:172.21.15.28:6801/3928805442] conn(0x7fa3200727e0 0x7fa320074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.069+0000 7fa33b400640 1 -- 172.21.15.28:0/655245140 --> [v2:172.21.15.28:6800/3928805442,v1:172.21.15.28:6801/3928805442] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3200752b0 con 0x7fa3200727e0 2024-04-26T21:06:47.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.069+0000 7fa33b400640 1 -- 172.21.15.28:0/655245140 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa33408d430 con 0x7fa328004800 2024-04-26T21:06:47.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.069+0000 7fa342a00640 1 --2- 172.21.15.28:0/655245140 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa32006c920 0x7fa32006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.070 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.069+0000 7fa343e00640 1 --2- 172.21.15.28:0/655245140 >> [v2:172.21.15.28:6800/3928805442,v1:172.21.15.28:6801/3928805442] conn(0x7fa3200727e0 0x7fa320074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.069+0000 7fa342a00640 1 --2- 172.21.15.28:0/655245140 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa32006c920 0x7fa32006ede0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa32c033000 tx=0x7fa32c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:47.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.070+0000 7fa343e00640 1 --2- 172.21.15.28:0/655245140 >> [v2:172.21.15.28:6800/3928805442,v1:172.21.15.28:6801/3928805442] conn(0x7fa3200727e0 0x7fa320074c00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:47.071 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.070+0000 7fa33b400640 1 -- 172.21.15.28:0/655245140 <== osd.1 v2:172.21.15.28:6800/3928805442 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa3200752b0 con 0x7fa3200727e0 2024-04-26T21:06:47.086 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.085+0000 7fa349200640 1 -- 172.21.15.28:0/655245140 --> [v2:172.21.15.28:6800/3928805442,v1:172.21.15.28:6801/3928805442] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa344073d30 con 0x7fa3200727e0 2024-04-26T21:06:47.087 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.086+0000 7fa33b400640 1 -- 172.21.15.28:0/655245140 <== osd.1 v2:172.21.15.28:6800/3928805442 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa344073d30 con 0x7fa3200727e0 2024-04-26T21:06:47.087 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T21:06:47.087 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:06:47.087 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T21:06:47.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.086+0000 7fa349200640 1 -- 172.21.15.28:0/655245140 >> [v2:172.21.15.28:6800/3928805442,v1:172.21.15.28:6801/3928805442] conn(0x7fa3200727e0 msgr2=0x7fa320074c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.086+0000 7fa349200640 1 --2- 172.21.15.28:0/655245140 >> [v2:172.21.15.28:6800/3928805442,v1:172.21.15.28:6801/3928805442] conn(0x7fa3200727e0 0x7fa320074c00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.086+0000 7fa349200640 1 -- 172.21.15.28:0/655245140 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa32006c920 msgr2=0x7fa32006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.086+0000 7fa349200640 1 --2- 172.21.15.28:0/655245140 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa32006c920 0x7fa32006ede0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa32c033000 tx=0x7fa32c008030 comp rx=0 tx=0).stop 2024-04-26T21:06:47.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.086+0000 7fa349200640 1 -- 172.21.15.28:0/655245140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa328004800 msgr2=0x7fa3441e8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.086+0000 7fa349200640 1 --2- 172.21.15.28:0/655245140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa328004800 0x7fa3441e8c40 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa33400c710 tx=0x7fa33400cbd0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.086+0000 7fa343e00640 1 -- 172.21.15.28:0/655245140 reap_dead start 2024-04-26T21:06:47.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.086+0000 7fa349200640 1 -- 172.21.15.28:0/655245140 shutdown_connections 2024-04-26T21:06:47.088 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.086+0000 7fa349200640 1 -- 172.21.15.28:0/655245140 >> 172.21.15.28:0/655245140 conn(0x7fa3440b2820 msgr2=0x7fa34415e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:47.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.087+0000 7fa349200640 1 -- 172.21.15.28:0/655245140 shutdown_connections 2024-04-26T21:06:47.089 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.087+0000 7fa349200640 1 -- 172.21.15.28:0/655245140 wait complete. 2024-04-26T21:06:47.105 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-26T21:06:47.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.308+0000 7fdcc7000640 1 Processor -- start 2024-04-26T21:06:47.310 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.309+0000 7fdcc7000640 1 -- start start 2024-04-26T21:06:47.311 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.310+0000 7fdcc7000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdcc0071e20 0x7fdcc01610c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.310+0000 7fdcc7000640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcc0146440 con 0x7fdcc0146670 2024-04-26T21:06:47.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.310+0000 7fdcc7000640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcc006f010 con 0x7fdcc0146060 2024-04-26T21:06:47.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.310+0000 7fdcc7000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fdcc0072220 con 0x7fdcc0071e20 2024-04-26T21:06:47.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.311+0000 7fdcc5200640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fdcc0146670 0x7fdcc00cb0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40248/0 (socket says 172.21.15.28:40248) 2024-04-26T21:06:47.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.311+0000 7fdcc5200640 1 -- 172.21.15.28:0/2760853407 learned_addr learned my addr 172.21.15.28:0/2760853407 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:47.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.311+0000 7fdcc6600640 1 --2- 172.21.15.28:0/2760853407 >> v2:172.21.15.28:3300/0 conn(0x7fdcc0071e20 0x7fdcc01610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.311+0000 7fdcc6600640 1 -- 172.21.15.28:0/2760853407 >> v1:172.21.15.103:6789/0 conn(0x7fdcc0146060 legacy=0x7fdcc01609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:47.312 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.311+0000 7fdcbfe00640 1 -- 172.21.15.28:0/2760853407 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2411680236 0 0) 0x7fdcc0146440 con 0x7fdcc0146670 2024-04-26T21:06:47.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.311+0000 7fdcc6600640 1 -- 172.21.15.28:0/2760853407 >> v1:172.21.15.28:6789/0 conn(0x7fdcc0146670 legacy=0x7fdcc00cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.311+0000 7fdcc6600640 1 -- 172.21.15.28:0/2760853407 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc01617a0 con 0x7fdcc0071e20 2024-04-26T21:06:47.313 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.312+0000 7fdcc6600640 1 --2- 172.21.15.28:0/2760853407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc0071e20 0x7fdcc01610c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009d90 tx=0x7fdcb802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b644ea3873c965fc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:47.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.313+0000 7fdcbfe00640 1 -- 172.21.15.28:0/2760853407 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8031600 con 0x7fdcc0071e20 2024-04-26T21:06:47.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.313+0000 7fdcbfe00640 1 -- 172.21.15.28:0/2760853407 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb80317a0 con 0x7fdcc0071e20 2024-04-26T21:06:47.314 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.313+0000 7fdcbfe00640 1 -- 172.21.15.28:0/2760853407 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8031b70 con 0x7fdcc0071e20 2024-04-26T21:06:47.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.313+0000 7fdcc7000640 1 -- 172.21.15.28:0/2760853407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc0071e20 msgr2=0x7fdcc01610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.313+0000 7fdcc7000640 1 --2- 172.21.15.28:0/2760853407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc0071e20 0x7fdcc01610c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009d90 tx=0x7fdcb802a610 comp rx=0 tx=0).stop 2024-04-26T21:06:47.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.313+0000 7fdcc7000640 1 -- 172.21.15.28:0/2760853407 shutdown_connections 2024-04-26T21:06:47.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.313+0000 7fdcc7000640 1 --2- 172.21.15.28:0/2760853407 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc0071e20 0x7fdcc01610c0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.314+0000 7fdcc7000640 1 -- 172.21.15.28:0/2760853407 >> 172.21.15.28:0/2760853407 conn(0x7fdcc00d0a50 msgr2=0x7fdcc00d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:47.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.314+0000 7fdcc7000640 1 -- 172.21.15.28:0/2760853407 shutdown_connections 2024-04-26T21:06:47.315 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.314+0000 7fdcc7000640 1 -- 172.21.15.28:0/2760853407 wait complete. 2024-04-26T21:06:47.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.315+0000 7fdcc7000640 1 Processor -- start 2024-04-26T21:06:47.316 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.315+0000 7fdcc7000640 1 -- start start 2024-04-26T21:06:47.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.316+0000 7fdcc7000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdcc0071e20 0x7fdcc00c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.316+0000 7fdcc7000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdcc0146060 0x7fdcc0137890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.316+0000 7fdcc7000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc0146670 0x7fdcc01f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.316+0000 7fdcc7000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc006ff60 con 0x7fdcc0146670 2024-04-26T21:06:47.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.316+0000 7fdcc7000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc006f510 con 0x7fdcc0071e20 2024-04-26T21:06:47.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.316+0000 7fdcc7000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc00cb7b0 con 0x7fdcc0146060 2024-04-26T21:06:47.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.317+0000 7fdcc6600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc0146670 0x7fdcc01f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.317+0000 7fdcc6600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc0146670 0x7fdcc01f5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34152/0 (socket says 172.21.15.28:34152) 2024-04-26T21:06:47.318 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.317+0000 7fdcc6600640 1 -- 172.21.15.28:0/3182207381 learned_addr learned my addr 172.21.15.28:0/3182207381 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:47.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.317+0000 7fdcc5c00640 1 --2- 172.21.15.28:0/3182207381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdcc0071e20 0x7fdcc00c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.317+0000 7fdcc6600640 1 -- 172.21.15.28:0/3182207381 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdcc0146060 msgr2=0x7fdcc0137890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.317+0000 7fdcc6600640 1 --2- 172.21.15.28:0/3182207381 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdcc0146060 0x7fdcc0137890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.317+0000 7fdcc6600640 1 -- 172.21.15.28:0/3182207381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdcc0071e20 msgr2=0x7fdcc00c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.317+0000 7fdcc6600640 1 --2- 172.21.15.28:0/3182207381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdcc0071e20 0x7fdcc00c99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.317+0000 7fdcc6600640 1 -- 172.21.15.28:0/3182207381 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc0137dd0 con 0x7fdcc0146670 2024-04-26T21:06:47.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.318+0000 7fdcc6600640 1 --2- 172.21.15.28:0/3182207381 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc0146670 0x7fdcc01f5d90 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009eb0 tx=0x7fdcb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:47.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.318+0000 7fdcbd600640 1 -- 172.21.15.28:0/3182207381 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8040070 con 0x7fdcc0146670 2024-04-26T21:06:47.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.318+0000 7fdcbd600640 1 -- 172.21.15.28:0/3182207381 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb8030b60 con 0x7fdcc0146670 2024-04-26T21:06:47.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.318+0000 7fdcbd600640 1 -- 172.21.15.28:0/3182207381 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8045460 con 0x7fdcc0146670 2024-04-26T21:06:47.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.318+0000 7fdcc7000640 1 -- 172.21.15.28:0/3182207381 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc0138080 con 0x7fdcc0146670 2024-04-26T21:06:47.320 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.318+0000 7fdcc7000640 1 -- 172.21.15.28:0/3182207381 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc0138640 con 0x7fdcc0146670 2024-04-26T21:06:47.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.320+0000 7fdcc7000640 1 -- 172.21.15.28:0/3182207381 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc006a360 con 0x7fdcc0146670 2024-04-26T21:06:47.321 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.320+0000 7fdcbd600640 1 -- 172.21.15.28:0/3182207381 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdcb8039040 con 0x7fdcc0146670 2024-04-26T21:06:47.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.321+0000 7fdcbd600640 1 --2- 172.21.15.28:0/3182207381 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fdca006c940 0x7fdca006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.322 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.321+0000 7fdcbd600640 1 -- 172.21.15.28:0/3182207381 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fdcb80acf40 con 0x7fdcc0146670 2024-04-26T21:06:47.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.322+0000 7fdcc5c00640 1 --2- 172.21.15.28:0/3182207381 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fdca006c940 0x7fdca006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.323 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.322+0000 7fdcc5c00640 1 --2- 172.21.15.28:0/3182207381 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fdca006c940 0x7fdca006ee00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0003d90 tx=0x7fdcb0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:47.327 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.326+0000 7fdcbd600640 1 -- 172.21.15.28:0/3182207381 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdcb807b810 con 0x7fdcc0146670 2024-04-26T21:06:47.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.529+0000 7fdcc7000640 1 -- 172.21.15.28:0/3182207381 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fdcc00cafd0 con 0x7fdca006c940 2024-04-26T21:06:47.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.530+0000 7fdcbd600640 1 -- 172.21.15.28:0/3182207381 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdcc00cafd0 con 0x7fdca006c940 2024-04-26T21:06:47.531 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 1 to deep-scrub 2024-04-26T21:06:47.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.532+0000 7fdcc7000640 1 -- 172.21.15.28:0/3182207381 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fdca006c940 msgr2=0x7fdca006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.532+0000 7fdcc7000640 1 --2- 172.21.15.28:0/3182207381 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fdca006c940 0x7fdca006ee00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0003d90 tx=0x7fdcb0009040 comp rx=0 tx=0).stop 2024-04-26T21:06:47.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.532+0000 7fdcc7000640 1 -- 172.21.15.28:0/3182207381 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc0146670 msgr2=0x7fdcc01f5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.533+0000 7fdcc7000640 1 --2- 172.21.15.28:0/3182207381 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc0146670 0x7fdcc01f5d90 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fdcb8009eb0 tx=0x7fdcb8004a40 comp rx=0 tx=0).stop 2024-04-26T21:06:47.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.533+0000 7fdcc7000640 1 -- 172.21.15.28:0/3182207381 shutdown_connections 2024-04-26T21:06:47.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.533+0000 7fdcc7000640 1 --2- 172.21.15.28:0/3182207381 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fdca006c940 0x7fdca006ee00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.533+0000 7fdcc7000640 1 --2- 172.21.15.28:0/3182207381 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdcc0146670 0x7fdcc01f5d90 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.533+0000 7fdcc7000640 1 --2- 172.21.15.28:0/3182207381 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fdcc0146060 0x7fdcc0137890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.533+0000 7fdcc7000640 1 --2- 172.21.15.28:0/3182207381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdcc0071e20 0x7fdcc00c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.533+0000 7fdcc7000640 1 -- 172.21.15.28:0/3182207381 >> 172.21.15.28:0/3182207381 conn(0x7fdcc00d0a50 msgr2=0x7fdcc0160e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:47.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.533+0000 7fdcc7000640 1 -- 172.21.15.28:0/3182207381 shutdown_connections 2024-04-26T21:06:47.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.533+0000 7fdcc7000640 1 -- 172.21.15.28:0/3182207381 wait complete. 2024-04-26T21:06:47.553 INFO:tasks.ceph:Scrubbing osd.2 2024-04-26T21:06:47.554 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:06:47.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.775+0000 7f0a8fa00640 1 Processor -- start 2024-04-26T21:06:47.776 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.775+0000 7f0a8fa00640 1 -- start start 2024-04-26T21:06:47.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.777+0000 7f0a8fa00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f0a880cee70 0x7f0a88167f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.777+0000 7f0a8fa00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a88073630 con 0x7f0a880b6100 2024-04-26T21:06:47.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.777+0000 7f0a8fa00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a88073910 con 0x7f0a88071fb0 2024-04-26T21:06:47.778 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.777+0000 7f0a8fa00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f0a88073bf0 con 0x7f0a880cee70 2024-04-26T21:06:47.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.777+0000 7f0a8d000640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f0a880b6100 0x7f0a880b64e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40264/0 (socket says 172.21.15.28:40264) 2024-04-26T21:06:47.779 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.777+0000 7f0a8d000640 1 -- 172.21.15.28:0/4198023151 learned_addr learned my addr 172.21.15.28:0/4198023151 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:47.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.778+0000 7f0a87400640 1 -- 172.21.15.28:0/4198023151 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1665977105 0 0) 0x7f0a88073630 con 0x7f0a880b6100 2024-04-26T21:06:47.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.778+0000 7f0a87400640 1 -- 172.21.15.28:0/4198023151 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a70003520 con 0x7f0a880b6100 2024-04-26T21:06:47.781 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.778+0000 7f0a8da00640 1 --2- 172.21.15.28:0/4198023151 >> v2:172.21.15.103:3300/0 conn(0x7f0a880cee70 0x7f0a88167f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.778+0000 7f0a87400640 1 -- 172.21.15.28:0/4198023151 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3611446056 0 0) 0x7f0a70003520 con 0x7f0a880b6100 2024-04-26T21:06:47.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.778+0000 7f0a87400640 1 -- 172.21.15.28:0/4198023151 >> v2:172.21.15.103:3300/0 conn(0x7f0a880cee70 msgr2=0x7f0a88167f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:47.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.778+0000 7f0a87400640 1 --2- 172.21.15.28:0/4198023151 >> v2:172.21.15.103:3300/0 conn(0x7f0a880cee70 0x7f0a88167f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.778+0000 7f0a87400640 1 -- 172.21.15.28:0/4198023151 >> v1:172.21.15.103:6789/0 conn(0x7f0a88071fb0 legacy=0x7f0a880ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.778+0000 7f0a87400640 1 -- 172.21.15.28:0/4198023151 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a88073ed0 con 0x7f0a880b6100 2024-04-26T21:06:47.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.778+0000 7f0a87400640 1 -- 172.21.15.28:0/4198023151 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f0a7c001670 con 0x7f0a880b6100 2024-04-26T21:06:47.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.778+0000 7f0a87400640 1 -- 172.21.15.28:0/4198023151 >> v1:172.21.15.28:6789/0 conn(0x7f0a880b6100 legacy=0x7f0a880b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.778+0000 7f0a87400640 1 --2- 172.21.15.28:0/4198023151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a70003a00 0x7f0a70003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.778+0000 7f0a87400640 1 --2- 172.21.15.28:0/4198023151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a70004800 0x7f0a70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.778+0000 7f0a87400640 1 --2- 172.21.15.28:0/4198023151 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a70005210 0x7f0a7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.782 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.778+0000 7f0a87400640 1 -- 172.21.15.28:0/4198023151 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0a88073bf0 con 0x7f0a70004800 2024-04-26T21:06:47.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.778+0000 7f0a87400640 1 -- 172.21.15.28:0/4198023151 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a7000c160 con 0x7f0a70003a00 2024-04-26T21:06:47.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.778+0000 7f0a87400640 1 -- 172.21.15.28:0/4198023151 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f0a7000c360 con 0x7f0a70005210 2024-04-26T21:06:47.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.779+0000 7f0a87e00640 1 --2- 172.21.15.28:0/4198023151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a70004800 0x7f0a70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.779+0000 7f0a8d000640 1 --2- 172.21.15.28:0/4198023151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a70003a00 0x7f0a70003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.779+0000 7f0a8da00640 1 --2- 172.21.15.28:0/4198023151 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a70005210 0x7f0a7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.779+0000 7f0a87e00640 1 -- 172.21.15.28:0/4198023151 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a70005210 msgr2=0x7f0a7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:47.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.779+0000 7f0a87e00640 1 --2- 172.21.15.28:0/4198023151 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a70005210 0x7f0a7000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.779+0000 7f0a87e00640 1 -- 172.21.15.28:0/4198023151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a70003a00 msgr2=0x7f0a70003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:47.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.779+0000 7f0a87e00640 1 --2- 172.21.15.28:0/4198023151 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a70003a00 0x7f0a70003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.779+0000 7f0a87e00640 1 -- 172.21.15.28:0/4198023151 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a70003520 con 0x7f0a70004800 2024-04-26T21:06:47.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.779+0000 7f0a8da00640 1 -- 172.21.15.28:0/4198023151 reap_dead start 2024-04-26T21:06:47.783 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.780+0000 7f0a87e00640 1 --2- 172.21.15.28:0/4198023151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a70004800 0x7f0a70004c80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0a74001750 tx=0x7f0a70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeaafe2d7d818202 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:47.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.781+0000 7f0a87400640 1 -- 172.21.15.28:0/4198023151 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a74001370 con 0x7f0a70004800 2024-04-26T21:06:47.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.781+0000 7f0a87400640 1 -- 172.21.15.28:0/4198023151 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a740052d0 con 0x7f0a70004800 2024-04-26T21:06:47.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.781+0000 7f0a87400640 1 -- 172.21.15.28:0/4198023151 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7402abc0 con 0x7f0a70004800 2024-04-26T21:06:47.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.781+0000 7f0a8fa00640 1 -- 172.21.15.28:0/4198023151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a70004800 msgr2=0x7f0a70004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.781+0000 7f0a8fa00640 1 --2- 172.21.15.28:0/4198023151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a70004800 0x7f0a70004c80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f0a74001750 tx=0x7f0a70005830 comp rx=0 tx=0).stop 2024-04-26T21:06:47.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.781+0000 7f0a8fa00640 1 -- 172.21.15.28:0/4198023151 shutdown_connections 2024-04-26T21:06:47.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.781+0000 7f0a8fa00640 1 --2- 172.21.15.28:0/4198023151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a70004800 0x7f0a70004c80 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.785 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.781+0000 7f0a8fa00640 1 -- 172.21.15.28:0/4198023151 >> 172.21.15.28:0/4198023151 conn(0x7f0a8806e220 msgr2=0x7f0a8806e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:47.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.782+0000 7f0a8fa00640 1 -- 172.21.15.28:0/4198023151 shutdown_connections 2024-04-26T21:06:47.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.782+0000 7f0a8fa00640 1 -- 172.21.15.28:0/4198023151 wait complete. 2024-04-26T21:06:47.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.783+0000 7f0a8fa00640 1 Processor -- start 2024-04-26T21:06:47.786 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.784+0000 7f0a8fa00640 1 -- start start 2024-04-26T21:06:47.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.784+0000 7f0a8fa00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a70004800 0x7f0a8813a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.788 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.784+0000 7f0a8fa00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a8814e600 0x7f0a8814ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.785+0000 7f0a8fa00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a8815f050 0x7f0a8815f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.785+0000 7f0a8fa00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f0a88146160 con 0x7f0a70004800 2024-04-26T21:06:47.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.785+0000 7f0a8fa00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0a88145fe0 con 0x7f0a8814e600 2024-04-26T21:06:47.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.785+0000 7f0a8fa00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f0a880ce760 con 0x7f0a8815f050 2024-04-26T21:06:47.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.785+0000 7f0a8d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a70004800 0x7f0a8813a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.785+0000 7f0a8d000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a70004800 0x7f0a8813a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34166/0 (socket says 172.21.15.28:34166) 2024-04-26T21:06:47.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.785+0000 7f0a8d000640 1 -- 172.21.15.28:0/486026919 learned_addr learned my addr 172.21.15.28:0/486026919 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:47.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.785+0000 7f0a8da00640 1 --2- 172.21.15.28:0/486026919 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a8815f050 0x7f0a8815f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.785+0000 7f0a8d000640 1 -- 172.21.15.28:0/486026919 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a8815f050 msgr2=0x7f0a8815f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.785+0000 7f0a8d000640 1 --2- 172.21.15.28:0/486026919 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0a8815f050 0x7f0a8815f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.789 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.785+0000 7f0a8d000640 1 -- 172.21.15.28:0/486026919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a8814e600 msgr2=0x7f0a8814ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.785+0000 7f0a87e00640 1 --2- 172.21.15.28:0/486026919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a8814e600 0x7f0a8814ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.785+0000 7f0a8d000640 1 --2- 172.21.15.28:0/486026919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0a8814e600 0x7f0a8814ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.785+0000 7f0a8d000640 1 -- 172.21.15.28:0/486026919 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a8814f460 con 0x7f0a70004800 2024-04-26T21:06:47.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.786+0000 7f0a8d000640 1 --2- 172.21.15.28:0/486026919 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a70004800 0x7f0a8813a990 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00e760 tx=0x7f0a7c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:47.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.786+0000 7f0a84c00640 1 -- 172.21.15.28:0/486026919 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c00c660 con 0x7f0a70004800 2024-04-26T21:06:47.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.786+0000 7f0a84c00640 1 -- 172.21.15.28:0/486026919 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a7c018070 con 0x7f0a70004800 2024-04-26T21:06:47.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.786+0000 7f0a84c00640 1 -- 172.21.15.28:0/486026919 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c00ca00 con 0x7f0a70004800 2024-04-26T21:06:47.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.786+0000 7f0a8fa00640 1 -- 172.21.15.28:0/486026919 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a8813cd20 con 0x7f0a70004800 2024-04-26T21:06:47.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.786+0000 7f0a8fa00640 1 -- 172.21.15.28:0/486026919 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a8813d2e0 con 0x7f0a70004800 2024-04-26T21:06:47.790 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.787+0000 7f0a8fa00640 1 -- 172.21.15.28:0/486026919 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0a88076520 con 0x7f0a70004800 2024-04-26T21:06:47.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.788+0000 7f0a84c00640 1 -- 172.21.15.28:0/486026919 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0a7c00cba0 con 0x7f0a70004800 2024-04-26T21:06:47.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.789+0000 7f0a84c00640 1 --2- 172.21.15.28:0/486026919 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0a6806c920 0x7f0a6806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.790+0000 7f0a84c00640 1 -- 172.21.15.28:0/486026919 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f0a7c08b300 con 0x7f0a70004800 2024-04-26T21:06:47.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.790+0000 7f0a87e00640 1 --2- 172.21.15.28:0/486026919 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0a6806c920 0x7f0a6806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.790+0000 7f0a84c00640 1 --2- 172.21.15.28:0/486026919 >> [v2:172.21.15.28:6816/2056655204,v1:172.21.15.28:6817/2056655204] conn(0x7f0a680727e0 0x7f0a68074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:47.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.790+0000 7f0a84c00640 1 -- 172.21.15.28:0/486026919 --> [v2:172.21.15.28:6816/2056655204,v1:172.21.15.28:6817/2056655204] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a7c019ec0 con 0x7f0a680727e0 2024-04-26T21:06:47.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.790+0000 7f0a84c00640 1 -- 172.21.15.28:0/486026919 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f0a7c092050 con 0x7f0a70004800 2024-04-26T21:06:47.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.790+0000 7f0a8da00640 1 --2- 172.21.15.28:0/486026919 >> [v2:172.21.15.28:6816/2056655204,v1:172.21.15.28:6817/2056655204] conn(0x7f0a680727e0 0x7f0a68074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:47.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.791+0000 7f0a87e00640 1 --2- 172.21.15.28:0/486026919 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0a6806c920 0x7f0a6806ede0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0a74032000 tx=0x7f0a7402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:47.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.791+0000 7f0a8da00640 1 --2- 172.21.15.28:0/486026919 >> [v2:172.21.15.28:6816/2056655204,v1:172.21.15.28:6817/2056655204] conn(0x7f0a680727e0 0x7f0a68074c00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:47.793 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.792+0000 7f0a84c00640 1 -- 172.21.15.28:0/486026919 <== osd.2 v2:172.21.15.28:6816/2056655204 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0a7c019ec0 con 0x7f0a680727e0 2024-04-26T21:06:47.807 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.806+0000 7f0a8fa00640 1 -- 172.21.15.28:0/486026919 --> [v2:172.21.15.28:6816/2056655204,v1:172.21.15.28:6817/2056655204] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0a8806ff90 con 0x7f0a680727e0 2024-04-26T21:06:47.808 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.807+0000 7f0a84c00640 1 -- 172.21.15.28:0/486026919 <== osd.2 v2:172.21.15.28:6816/2056655204 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0a8806ff90 con 0x7f0a680727e0 2024-04-26T21:06:47.808 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T21:06:47.808 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:06:47.809 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T21:06:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.807+0000 7f0a8fa00640 1 -- 172.21.15.28:0/486026919 >> [v2:172.21.15.28:6816/2056655204,v1:172.21.15.28:6817/2056655204] conn(0x7f0a680727e0 msgr2=0x7f0a68074c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.808+0000 7f0a8fa00640 1 --2- 172.21.15.28:0/486026919 >> [v2:172.21.15.28:6816/2056655204,v1:172.21.15.28:6817/2056655204] conn(0x7f0a680727e0 0x7f0a68074c00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.808+0000 7f0a8fa00640 1 -- 172.21.15.28:0/486026919 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0a6806c920 msgr2=0x7f0a6806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.808+0000 7f0a8fa00640 1 --2- 172.21.15.28:0/486026919 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f0a6806c920 0x7f0a6806ede0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0a74032000 tx=0x7f0a7402e000 comp rx=0 tx=0).stop 2024-04-26T21:06:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.808+0000 7f0a8fa00640 1 -- 172.21.15.28:0/486026919 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a70004800 msgr2=0x7f0a8813a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.808+0000 7f0a8fa00640 1 --2- 172.21.15.28:0/486026919 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0a70004800 0x7f0a8813a990 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00e760 tx=0x7f0a7c00ec20 comp rx=0 tx=0).stop 2024-04-26T21:06:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.808+0000 7f0a8da00640 1 -- 172.21.15.28:0/486026919 reap_dead start 2024-04-26T21:06:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.808+0000 7f0a8fa00640 1 -- 172.21.15.28:0/486026919 shutdown_connections 2024-04-26T21:06:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.808+0000 7f0a8fa00640 1 -- 172.21.15.28:0/486026919 >> 172.21.15.28:0/486026919 conn(0x7f0a8806e220 msgr2=0x7f0a88142a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.808+0000 7f0a8fa00640 1 -- 172.21.15.28:0/486026919 shutdown_connections 2024-04-26T21:06:47.810 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:47.808+0000 7f0a8fa00640 1 -- 172.21.15.28:0/486026919 wait complete. 2024-04-26T21:06:47.830 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-26T21:06:48.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.031+0000 7f5933200640 1 Processor -- start 2024-04-26T21:06:48.032 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.031+0000 7f5933200640 1 -- start start 2024-04-26T21:06:48.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.033+0000 7f5933200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f592c146060 0x7f592c071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.033+0000 7f5933200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f592c146440 con 0x7f592c146670 2024-04-26T21:06:48.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.033+0000 7f5933200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f592c06f010 con 0x7f592c1610c0 2024-04-26T21:06:48.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.033+0000 7f5933200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f592c06f510 con 0x7f592c146060 2024-04-26T21:06:48.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.033+0000 7f5931400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f592c146670 0x7f592c1609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40268/0 (socket says 172.21.15.28:40268) 2024-04-26T21:06:48.034 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.033+0000 7f5931400640 1 -- 172.21.15.28:0/961949485 learned_addr learned my addr 172.21.15.28:0/961949485 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:48.036 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.034+0000 7f5931e00640 1 --2- 172.21.15.28:0/961949485 >> v2:172.21.15.103:3300/0 conn(0x7f592c146060 0x7f592c071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.034+0000 7f5930a00640 1 -- 172.21.15.28:0/961949485 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1452450559 0 0) 0x7f592c146440 con 0x7f592c146670 2024-04-26T21:06:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.034+0000 7f5930a00640 1 -- 172.21.15.28:0/961949485 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5914003520 con 0x7f592c146670 2024-04-26T21:06:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.034+0000 7f5930a00640 1 -- 172.21.15.28:0/961949485 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2236389482 0 0) 0x7f5914003520 con 0x7f592c146670 2024-04-26T21:06:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.034+0000 7f5930a00640 1 -- 172.21.15.28:0/961949485 >> v2:172.21.15.103:3300/0 conn(0x7f592c146060 msgr2=0x7f592c071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.034+0000 7f5930a00640 1 --2- 172.21.15.28:0/961949485 >> v2:172.21.15.103:3300/0 conn(0x7f592c146060 0x7f592c071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.034+0000 7f5930a00640 1 -- 172.21.15.28:0/961949485 >> v1:172.21.15.103:6789/0 conn(0x7f592c1610c0 legacy=0x7f592c0cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.034+0000 7f5930a00640 1 -- 172.21.15.28:0/961949485 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592c1614d0 con 0x7f592c146670 2024-04-26T21:06:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.034+0000 7f5930a00640 1 -- 172.21.15.28:0/961949485 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f591c001670 con 0x7f592c146670 2024-04-26T21:06:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.034+0000 7f5930a00640 1 -- 172.21.15.28:0/961949485 >> v1:172.21.15.28:6789/0 conn(0x7f592c146670 legacy=0x7f592c1609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.034+0000 7f5930a00640 1 --2- 172.21.15.28:0/961949485 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5914003a00 0x7f5914003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.034+0000 7f5930a00640 1 --2- 172.21.15.28:0/961949485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5914004800 0x7f5914004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.034+0000 7f5930a00640 1 --2- 172.21.15.28:0/961949485 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5914005210 0x7f591400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.037 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.034+0000 7f5930a00640 1 -- 172.21.15.28:0/961949485 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f592c06f510 con 0x7f5914003a00 2024-04-26T21:06:48.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.034+0000 7f5930a00640 1 -- 172.21.15.28:0/961949485 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f591400c160 con 0x7f5914004800 2024-04-26T21:06:48.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.034+0000 7f5930a00640 1 -- 172.21.15.28:0/961949485 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f591400c360 con 0x7f5914005210 2024-04-26T21:06:48.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.035+0000 7f5931e00640 1 --2- 172.21.15.28:0/961949485 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5914003a00 0x7f5914003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.035+0000 7f5931400640 1 --2- 172.21.15.28:0/961949485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5914004800 0x7f5914004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.035+0000 7f5931e00640 1 -- 172.21.15.28:0/961949485 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5914005210 msgr2=0x7f591400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:48.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.035+0000 7f5931e00640 1 --2- 172.21.15.28:0/961949485 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f5914005210 0x7f591400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.035+0000 7f5931e00640 1 -- 172.21.15.28:0/961949485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5914004800 msgr2=0x7f5914004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:48.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.035+0000 7f5931e00640 1 --2- 172.21.15.28:0/961949485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5914004800 0x7f5914004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.035+0000 7f5931e00640 1 -- 172.21.15.28:0/961949485 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5914003520 con 0x7f5914003a00 2024-04-26T21:06:48.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.035+0000 7f5931400640 1 --2- 172.21.15.28:0/961949485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5914004800 0x7f5914004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:48.038 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.035+0000 7f5932800640 1 -- 172.21.15.28:0/961949485 reap_dead start 2024-04-26T21:06:48.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.036+0000 7f5931e00640 1 --2- 172.21.15.28:0/961949485 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5914003a00 0x7f5914003e20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5920009a90 tx=0x7f5914005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e217bd7380d297f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:48.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.036+0000 7f5930a00640 1 -- 172.21.15.28:0/961949485 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920036070 con 0x7f5914003a00 2024-04-26T21:06:48.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.036+0000 7f5930a00640 1 -- 172.21.15.28:0/961949485 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5920031400 con 0x7f5914003a00 2024-04-26T21:06:48.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.036+0000 7f5930a00640 1 -- 172.21.15.28:0/961949485 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920009690 con 0x7f5914003a00 2024-04-26T21:06:48.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.037+0000 7f5933200640 1 -- 172.21.15.28:0/961949485 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5914003a00 msgr2=0x7f5914003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:48.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.037+0000 7f5933200640 1 --2- 172.21.15.28:0/961949485 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5914003a00 0x7f5914003e20 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5920009a90 tx=0x7f5914005830 comp rx=0 tx=0).stop 2024-04-26T21:06:48.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.037+0000 7f5933200640 1 -- 172.21.15.28:0/961949485 shutdown_connections 2024-04-26T21:06:48.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.037+0000 7f5933200640 1 --2- 172.21.15.28:0/961949485 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5914003a00 0x7f5914003e20 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.037+0000 7f5933200640 1 -- 172.21.15.28:0/961949485 >> 172.21.15.28:0/961949485 conn(0x7f592c0d0a50 msgr2=0x7f592c0d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:48.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.037+0000 7f5933200640 1 -- 172.21.15.28:0/961949485 shutdown_connections 2024-04-26T21:06:48.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.037+0000 7f5933200640 1 -- 172.21.15.28:0/961949485 wait complete. 2024-04-26T21:06:48.040 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.039+0000 7f5933200640 1 Processor -- start 2024-04-26T21:06:48.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.039+0000 7f5933200640 1 -- start start 2024-04-26T21:06:48.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.040+0000 7f5933200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5914003a00 0x7f592c141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.040+0000 7f5933200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f592c155390 0x7f592c0cffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.040+0000 7f5933200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f592c0ca010 0x7f592c140e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.040+0000 7f5933200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f592c06ff60 con 0x7f5914003a00 2024-04-26T21:06:48.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.040+0000 7f5933200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f592c06fc80 con 0x7f592c0ca010 2024-04-26T21:06:48.042 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.040+0000 7f5933200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f592c071e20 con 0x7f592c155390 2024-04-26T21:06:48.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.040+0000 7f5931e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5914003a00 0x7f592c141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.040+0000 7f5931e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5914003a00 0x7f592c141ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34182/0 (socket says 172.21.15.28:34182) 2024-04-26T21:06:48.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.040+0000 7f5931e00640 1 -- 172.21.15.28:0/2179995155 learned_addr learned my addr 172.21.15.28:0/2179995155 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:48.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.041+0000 7f5931e00640 1 -- 172.21.15.28:0/2179995155 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f592c155390 msgr2=0x7f592c0cffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:48.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.041+0000 7f5931e00640 1 --2- 172.21.15.28:0/2179995155 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f592c155390 0x7f592c0cffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.041+0000 7f5932800640 1 --2- 172.21.15.28:0/2179995155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f592c0ca010 0x7f592c140e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.041+0000 7f5931e00640 1 -- 172.21.15.28:0/2179995155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f592c0ca010 msgr2=0x7f592c140e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:48.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.041+0000 7f5931e00640 1 --2- 172.21.15.28:0/2179995155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f592c0ca010 0x7f592c140e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.041+0000 7f5931e00640 1 -- 172.21.15.28:0/2179995155 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f592c0ce7c0 con 0x7f5914003a00 2024-04-26T21:06:48.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.041+0000 7f5932800640 1 --2- 172.21.15.28:0/2179995155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f592c0ca010 0x7f592c140e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:48.043 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.041+0000 7f5931e00640 1 --2- 172.21.15.28:0/2179995155 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5914003a00 0x7f592c141ed0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5920009bb0 tx=0x7f5920004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:48.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.041+0000 7f5926000640 1 -- 172.21.15.28:0/2179995155 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592003f020 con 0x7f5914003a00 2024-04-26T21:06:48.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.041+0000 7f5926000640 1 -- 172.21.15.28:0/2179995155 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5920037040 con 0x7f5914003a00 2024-04-26T21:06:48.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.041+0000 7f5926000640 1 -- 172.21.15.28:0/2179995155 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920036070 con 0x7f5914003a00 2024-04-26T21:06:48.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.042+0000 7f5933200640 1 -- 172.21.15.28:0/2179995155 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f592c156110 con 0x7f5914003a00 2024-04-26T21:06:48.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.042+0000 7f5933200640 1 -- 172.21.15.28:0/2179995155 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f592c155b50 con 0x7f5914003a00 2024-04-26T21:06:48.044 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.043+0000 7f5933200640 1 -- 172.21.15.28:0/2179995155 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f592c06a360 con 0x7f5914003a00 2024-04-26T21:06:48.045 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.044+0000 7f5926000640 1 -- 172.21.15.28:0/2179995155 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5920042020 con 0x7f5914003a00 2024-04-26T21:06:48.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.045+0000 7f5926000640 1 --2- 172.21.15.28:0/2179995155 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f591006c920 0x7f591006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.045+0000 7f5926000640 1 -- 172.21.15.28:0/2179995155 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f59200ab980 con 0x7f5914003a00 2024-04-26T21:06:48.046 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.046+0000 7f5931400640 1 --2- 172.21.15.28:0/2179995155 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f591006c920 0x7f591006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.047 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.046+0000 7f5931400640 1 --2- 172.21.15.28:0/2179995155 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f591006c920 0x7f591006ede0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f591c000c00 tx=0x7f591c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:48.051 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.050+0000 7f5926000640 1 -- 172.21.15.28:0/2179995155 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5920049030 con 0x7f5914003a00 2024-04-26T21:06:48.253 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.252+0000 7f5933200640 1 -- 172.21.15.28:0/2179995155 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f592c0b17c0 con 0x7f591006c920 2024-04-26T21:06:48.254 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.253+0000 7f5926000640 1 -- 172.21.15.28:0/2179995155 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f592c0b17c0 con 0x7f591006c920 2024-04-26T21:06:48.254 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 2 to deep-scrub 2024-04-26T21:06:48.256 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:06:48.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.255+0000 7f5933200640 1 -- 172.21.15.28:0/2179995155 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f591006c920 msgr2=0x7f591006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:48.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.255+0000 7f5933200640 1 --2- 172.21.15.28:0/2179995155 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f591006c920 0x7f591006ede0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f591c000c00 tx=0x7f591c009040 comp rx=0 tx=0).stop 2024-04-26T21:06:48.257 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.255+0000 7f5933200640 1 -- 172.21.15.28:0/2179995155 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5914003a00 msgr2=0x7f592c141ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:48.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.255+0000 7f5933200640 1 --2- 172.21.15.28:0/2179995155 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5914003a00 0x7f592c141ed0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5920009bb0 tx=0x7f5920004730 comp rx=0 tx=0).stop 2024-04-26T21:06:48.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.256+0000 7f5933200640 1 -- 172.21.15.28:0/2179995155 shutdown_connections 2024-04-26T21:06:48.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.256+0000 7f5933200640 1 --2- 172.21.15.28:0/2179995155 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f591006c920 0x7f591006ede0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.256+0000 7f5933200640 1 --2- 172.21.15.28:0/2179995155 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f592c0ca010 0x7f592c140e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.256+0000 7f5933200640 1 --2- 172.21.15.28:0/2179995155 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f592c155390 0x7f592c0cffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.256+0000 7f5933200640 1 --2- 172.21.15.28:0/2179995155 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5914003a00 0x7f592c141ed0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.256+0000 7f5933200640 1 -- 172.21.15.28:0/2179995155 >> 172.21.15.28:0/2179995155 conn(0x7f592c0d0a50 msgr2=0x7f592c1609b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:48.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.256+0000 7f5933200640 1 -- 172.21.15.28:0/2179995155 shutdown_connections 2024-04-26T21:06:48.258 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.256+0000 7f5933200640 1 -- 172.21.15.28:0/2179995155 wait complete. 2024-04-26T21:06:48.263 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:06:48.277 INFO:tasks.ceph:Scrubbing osd.3 2024-04-26T21:06:48.277 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:06:48.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.536+0000 7f19d7200640 1 Processor -- start 2024-04-26T21:06:48.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.536+0000 7f19d7200640 1 -- start start 2024-04-26T21:06:48.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.538+0000 7f19d7200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19d806f0b0 0x7f19d8167e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.538+0000 7f19d7200640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19d806f6d0 con 0x7f19d8161240 2024-04-26T21:06:48.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.538+0000 7f19d7200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19d80ceb10 con 0x7f19d81434c0 2024-04-26T21:06:48.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.538+0000 7f19d7200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f19d8143ce0 con 0x7f19d806f0b0 2024-04-26T21:06:48.539 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.538+0000 7f19d5400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f19d8161240 0x7f19d806e9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40272/0 (socket says 172.21.15.28:40272) 2024-04-26T21:06:48.540 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.538+0000 7f19d5400640 1 -- 172.21.15.28:0/3858373925 learned_addr learned my addr 172.21.15.28:0/3858373925 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:48.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.539+0000 7f19d6800640 1 --2- 172.21.15.28:0/3858373925 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19d806f0b0 0x7f19d8167e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.539+0000 7f19d4a00640 1 -- 172.21.15.28:0/3858373925 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3528727707 0 0) 0x7f19d806f6d0 con 0x7f19d8161240 2024-04-26T21:06:48.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.539+0000 7f19d4a00640 1 -- 172.21.15.28:0/3858373925 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19bc003520 con 0x7f19d8161240 2024-04-26T21:06:48.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.539+0000 7f19d4a00640 1 -- 172.21.15.28:0/3858373925 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2083806016 0 0) 0x7f19bc003520 con 0x7f19d8161240 2024-04-26T21:06:48.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.539+0000 7f19d4a00640 1 -- 172.21.15.28:0/3858373925 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19d806f0b0 msgr2=0x7f19d8167e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:48.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.539+0000 7f19d4a00640 1 --2- 172.21.15.28:0/3858373925 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19d806f0b0 0x7f19d8167e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.539+0000 7f19d4a00640 1 -- 172.21.15.28:0/3858373925 >> v1:172.21.15.103:6789/0 conn(0x7f19d81434c0 legacy=0x7f19d8160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:48.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.539+0000 7f19d4a00640 1 -- 172.21.15.28:0/3858373925 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d80cedf0 con 0x7f19d8161240 2024-04-26T21:06:48.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.539+0000 7f19d4a00640 1 -- 172.21.15.28:0/3858373925 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f19c4001670 con 0x7f19d8161240 2024-04-26T21:06:48.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.539+0000 7f19d4a00640 1 -- 172.21.15.28:0/3858373925 >> v1:172.21.15.28:6789/0 conn(0x7f19d8161240 legacy=0x7f19d806e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:48.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.539+0000 7f19d4a00640 1 --2- 172.21.15.28:0/3858373925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19bc0039e0 0x7f19bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.539+0000 7f19d4a00640 1 --2- 172.21.15.28:0/3858373925 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19d806f0b0 0x7f19bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.539+0000 7f19d4a00640 1 --2- 172.21.15.28:0/3858373925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19bc004d40 0x7f19bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.542 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.539+0000 7f19d4a00640 1 -- 172.21.15.28:0/3858373925 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f19bc00bbe0 con 0x7f19bc0039e0 2024-04-26T21:06:48.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.539+0000 7f19d4a00640 1 -- 172.21.15.28:0/3858373925 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19bc00bd60 con 0x7f19bc004d40 2024-04-26T21:06:48.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.539+0000 7f19d4a00640 1 -- 172.21.15.28:0/3858373925 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f19bc00bf60 con 0x7f19d806f0b0 2024-04-26T21:06:48.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.539+0000 7f19d5e00640 1 --2- 172.21.15.28:0/3858373925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19bc0039e0 0x7f19bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.540+0000 7f19d5400640 1 --2- 172.21.15.28:0/3858373925 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19d806f0b0 0x7f19bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.540+0000 7f19d5e00640 1 -- 172.21.15.28:0/3858373925 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19d806f0b0 msgr2=0x7f19bc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:48.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.540+0000 7f19d5e00640 1 --2- 172.21.15.28:0/3858373925 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19d806f0b0 0x7f19bc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.540+0000 7f19d6800640 1 --2- 172.21.15.28:0/3858373925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19bc004d40 0x7f19bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.540+0000 7f19d5e00640 1 -- 172.21.15.28:0/3858373925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19bc004d40 msgr2=0x7f19bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:48.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.540+0000 7f19d5e00640 1 --2- 172.21.15.28:0/3858373925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19bc004d40 0x7f19bc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.540+0000 7f19d5e00640 1 -- 172.21.15.28:0/3858373925 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19bc003520 con 0x7f19bc0039e0 2024-04-26T21:06:48.543 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.540+0000 7f19d5400640 1 --2- 172.21.15.28:0/3858373925 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19d806f0b0 0x7f19bc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:48.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.541+0000 7f19d5e00640 1 --2- 172.21.15.28:0/3858373925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19bc0039e0 0x7f19bc003e00 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f19c8001750 tx=0x7f19bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a759148d75471240 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:48.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.541+0000 7f19d4a00640 1 -- 172.21.15.28:0/3858373925 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8001370 con 0x7f19bc0039e0 2024-04-26T21:06:48.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.541+0000 7f19d4a00640 1 -- 172.21.15.28:0/3858373925 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19c80052d0 con 0x7f19bc0039e0 2024-04-26T21:06:48.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.541+0000 7f19d4a00640 1 -- 172.21.15.28:0/3858373925 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c802abe0 con 0x7f19bc0039e0 2024-04-26T21:06:48.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.542+0000 7f19d7200640 1 -- 172.21.15.28:0/3858373925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19bc0039e0 msgr2=0x7f19bc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:48.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.542+0000 7f19d7200640 1 --2- 172.21.15.28:0/3858373925 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19bc0039e0 0x7f19bc003e00 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f19c8001750 tx=0x7f19bc005830 comp rx=0 tx=0).stop 2024-04-26T21:06:48.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.542+0000 7f19d6800640 1 -- 172.21.15.28:0/3858373925 reap_dead start 2024-04-26T21:06:48.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.542+0000 7f19d7200640 1 -- 172.21.15.28:0/3858373925 shutdown_connections 2024-04-26T21:06:48.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.542+0000 7f19d7200640 1 -- 172.21.15.28:0/3858373925 >> 172.21.15.28:0/3858373925 conn(0x7f19d80b2820 msgr2=0x7f19d80b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:48.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.542+0000 7f19d7200640 1 -- 172.21.15.28:0/3858373925 shutdown_connections 2024-04-26T21:06:48.545 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.542+0000 7f19d7200640 1 -- 172.21.15.28:0/3858373925 wait complete. 2024-04-26T21:06:48.546 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.544+0000 7f19d7200640 1 Processor -- start 2024-04-26T21:06:48.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.544+0000 7f19d7200640 1 -- start start 2024-04-26T21:06:48.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.545+0000 7f19d7200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19bc0039e0 0x7f19d815c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.545+0000 7f19d7200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19d8144c60 0x7f19d81450e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.545+0000 7f19d7200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d815f830 0x7f19d815fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.545+0000 7f19d7200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f19d8143a20 con 0x7f19d815f830 2024-04-26T21:06:48.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.545+0000 7f19d7200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f19d81438a0 con 0x7f19bc0039e0 2024-04-26T21:06:48.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.545+0000 7f19d7200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f19d80d0600 con 0x7f19d8144c60 2024-04-26T21:06:48.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.546+0000 7f19d6800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d815f830 0x7f19d815fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.547 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.546+0000 7f19d6800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d815f830 0x7f19d815fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34198/0 (socket says 172.21.15.28:34198) 2024-04-26T21:06:48.548 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.546+0000 7f19d6800640 1 -- 172.21.15.28:0/3037530022 learned_addr learned my addr 172.21.15.28:0/3037530022 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:48.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.546+0000 7f19d5e00640 1 --2- 172.21.15.28:0/3037530022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19bc0039e0 0x7f19d815c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.546+0000 7f19d5400640 1 --2- 172.21.15.28:0/3037530022 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19d8144c60 0x7f19d81450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.546+0000 7f19d6800640 1 -- 172.21.15.28:0/3037530022 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19d8144c60 msgr2=0x7f19d81450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:48.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.546+0000 7f19d6800640 1 --2- 172.21.15.28:0/3037530022 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19d8144c60 0x7f19d81450e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.546+0000 7f19d6800640 1 -- 172.21.15.28:0/3037530022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19bc0039e0 msgr2=0x7f19d815c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:48.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.546+0000 7f19d6800640 1 --2- 172.21.15.28:0/3037530022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19bc0039e0 0x7f19d815c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.546+0000 7f19d6800640 1 -- 172.21.15.28:0/3037530022 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d815ca90 con 0x7f19d815f830 2024-04-26T21:06:48.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.546+0000 7f19d5400640 1 --2- 172.21.15.28:0/3037530022 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f19d8144c60 0x7f19d81450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:48.549 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.546+0000 7f19d5e00640 1 --2- 172.21.15.28:0/3037530022 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f19bc0039e0 0x7f19d815c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:48.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.546+0000 7f19d6800640 1 --2- 172.21.15.28:0/3037530022 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d815f830 0x7f19d815fcb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f19d80b6760 tx=0x7f19d0009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:48.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.547+0000 7f19ce000640 1 -- 172.21.15.28:0/3037530022 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0019070 con 0x7f19d815f830 2024-04-26T21:06:48.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.547+0000 7f19ce000640 1 -- 172.21.15.28:0/3037530022 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19d0007ce0 con 0x7f19d815f830 2024-04-26T21:06:48.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.547+0000 7f19ce000640 1 -- 172.21.15.28:0/3037530022 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19d0014640 con 0x7f19d815f830 2024-04-26T21:06:48.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.547+0000 7f19d7200640 1 -- 172.21.15.28:0/3037530022 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d81601f0 con 0x7f19d815f830 2024-04-26T21:06:48.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.547+0000 7f19d7200640 1 -- 172.21.15.28:0/3037530022 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d815d8b0 con 0x7f19d815f830 2024-04-26T21:06:48.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.548+0000 7f19d7200640 1 -- 172.21.15.28:0/3037530022 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f19d806a3d0 con 0x7f19d815f830 2024-04-26T21:06:48.550 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.549+0000 7f19ce000640 1 -- 172.21.15.28:0/3037530022 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19d001c070 con 0x7f19d815f830 2024-04-26T21:06:48.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.550+0000 7f19ce000640 1 --2- 172.21.15.28:0/3037530022 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f19b806c920 0x7f19b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.550+0000 7f19ce000640 1 -- 172.21.15.28:0/3037530022 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f19d008b1f0 con 0x7f19d815f830 2024-04-26T21:06:48.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.551+0000 7f19d5e00640 1 --2- 172.21.15.28:0/3037530022 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f19b806c920 0x7f19b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.551+0000 7f19ce000640 1 --2- 172.21.15.28:0/3037530022 >> [v2:172.21.15.28:6818/2569592417,v1:172.21.15.28:6820/2569592417] conn(0x7f19b80727e0 0x7f19b8074c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.551+0000 7f19ce000640 1 -- 172.21.15.28:0/3037530022 --> [v2:172.21.15.28:6818/2569592417,v1:172.21.15.28:6820/2569592417] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f19b80752b0 con 0x7f19b80727e0 2024-04-26T21:06:48.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.551+0000 7f19ce000640 1 -- 172.21.15.28:0/3037530022 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f19d0092050 con 0x7f19d815f830 2024-04-26T21:06:48.552 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.551+0000 7f19d5400640 1 --2- 172.21.15.28:0/3037530022 >> [v2:172.21.15.28:6818/2569592417,v1:172.21.15.28:6820/2569592417] conn(0x7f19b80727e0 0x7f19b8074c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.551+0000 7f19d5e00640 1 --2- 172.21.15.28:0/3037530022 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f19b806c920 0x7f19b806ede0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f19c8001750 tx=0x7f19c802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:48.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.552+0000 7f19d5400640 1 --2- 172.21.15.28:0/3037530022 >> [v2:172.21.15.28:6818/2569592417,v1:172.21.15.28:6820/2569592417] conn(0x7f19b80727e0 0x7f19b8074c00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:48.553 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.552+0000 7f19ce000640 1 -- 172.21.15.28:0/3037530022 <== osd.3 v2:172.21.15.28:6818/2569592417 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f19b80752b0 con 0x7f19b80727e0 2024-04-26T21:06:48.568 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.567+0000 7f19d7200640 1 -- 172.21.15.28:0/3037530022 --> [v2:172.21.15.28:6818/2569592417,v1:172.21.15.28:6820/2569592417] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f19d80ce320 con 0x7f19b80727e0 2024-04-26T21:06:48.569 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.568+0000 7f19ce000640 1 -- 172.21.15.28:0/3037530022 <== osd.3 v2:172.21.15.28:6818/2569592417 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f19d80ce320 con 0x7f19b80727e0 2024-04-26T21:06:48.569 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T21:06:48.569 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:06:48.569 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T21:06:48.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.568+0000 7f19d7200640 1 -- 172.21.15.28:0/3037530022 >> [v2:172.21.15.28:6818/2569592417,v1:172.21.15.28:6820/2569592417] conn(0x7f19b80727e0 msgr2=0x7f19b8074c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:48.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.568+0000 7f19d7200640 1 --2- 172.21.15.28:0/3037530022 >> [v2:172.21.15.28:6818/2569592417,v1:172.21.15.28:6820/2569592417] conn(0x7f19b80727e0 0x7f19b8074c00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.568+0000 7f19d7200640 1 -- 172.21.15.28:0/3037530022 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f19b806c920 msgr2=0x7f19b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:48.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.568+0000 7f19d7200640 1 --2- 172.21.15.28:0/3037530022 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f19b806c920 0x7f19b806ede0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f19c8001750 tx=0x7f19c802e000 comp rx=0 tx=0).stop 2024-04-26T21:06:48.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.568+0000 7f19d7200640 1 -- 172.21.15.28:0/3037530022 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d815f830 msgr2=0x7f19d815fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:48.570 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.568+0000 7f19d7200640 1 --2- 172.21.15.28:0/3037530022 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19d815f830 0x7f19d815fcb0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f19d80b6760 tx=0x7f19d0009d50 comp rx=0 tx=0).stop 2024-04-26T21:06:48.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.568+0000 7f19d6800640 1 -- 172.21.15.28:0/3037530022 reap_dead start 2024-04-26T21:06:48.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.568+0000 7f19d7200640 1 -- 172.21.15.28:0/3037530022 shutdown_connections 2024-04-26T21:06:48.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.568+0000 7f19d7200640 1 -- 172.21.15.28:0/3037530022 >> 172.21.15.28:0/3037530022 conn(0x7f19d80b2820 msgr2=0x7f19d80b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:48.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.569+0000 7f19d7200640 1 -- 172.21.15.28:0/3037530022 shutdown_connections 2024-04-26T21:06:48.571 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.569+0000 7f19d7200640 1 -- 172.21.15.28:0/3037530022 wait complete. 2024-04-26T21:06:48.586 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-26T21:06:48.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.791+0000 7f4929000640 1 Processor -- start 2024-04-26T21:06:48.792 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.791+0000 7f4929000640 1 -- start start 2024-04-26T21:06:48.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.793+0000 7f4929000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4924146060 0x7f4924071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.793+0000 7f4929000640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f4924146670 0x7f49241609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.793+0000 7f4929000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4924160ef0 0x7f492406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.793+0000 7f4929000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f492406f510 con 0x7f4924160ef0 2024-04-26T21:06:48.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.793+0000 7f4929000640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f4924146440 con 0x7f4924146670 2024-04-26T21:06:48.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.793+0000 7f4929000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f492406f010 con 0x7f4924146060 2024-04-26T21:06:48.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.793+0000 7f4923e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4924160ef0 0x7f492406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.793+0000 7f4923e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4924160ef0 0x7f492406de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34208/0 (socket says 172.21.15.28:34208) 2024-04-26T21:06:48.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.793+0000 7f4923e00640 1 -- 172.21.15.28:0/1560827369 learned_addr learned my addr 172.21.15.28:0/1560827369 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:48.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.793+0000 7f4922a00640 1 --2- 172.21.15.28:0/1560827369 >> v2:172.21.15.103:3300/0 conn(0x7f4924146670 0x7f49241609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.793+0000 7f4923400640 1 --2- 172.21.15.28:0/1560827369 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4924146060 0x7f4924071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.795 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.793+0000 7f4923e00640 1 -- 172.21.15.28:0/1560827369 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4924146060 msgr2=0x7f4924071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:48.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.794+0000 7f4923e00640 1 --2- 172.21.15.28:0/1560827369 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4924146060 0x7f4924071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.794+0000 7f4923e00640 1 -- 172.21.15.28:0/1560827369 >> v2:172.21.15.103:3300/0 conn(0x7f4924146670 msgr2=0x7f49241609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:48.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.794+0000 7f4923e00640 1 --2- 172.21.15.28:0/1560827369 >> v2:172.21.15.103:3300/0 conn(0x7f4924146670 0x7f49241609b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.796 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.794+0000 7f4923e00640 1 -- 172.21.15.28:0/1560827369 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49241617a0 con 0x7f4924160ef0 2024-04-26T21:06:48.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.795+0000 7f4923e00640 1 --2- 172.21.15.28:0/1560827369 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4924160ef0 0x7f492406de80 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f492406d720 tx=0x7f491c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1002cce40fa837fb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:48.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.795+0000 7f4922000640 1 -- 172.21.15.28:0/1560827369 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c037070 con 0x7f4924160ef0 2024-04-26T21:06:48.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.795+0000 7f4922000640 1 -- 172.21.15.28:0/1560827369 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f491c032440 con 0x7f4924160ef0 2024-04-26T21:06:48.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.795+0000 7f4922000640 1 -- 172.21.15.28:0/1560827369 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c032810 con 0x7f4924160ef0 2024-04-26T21:06:48.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.795+0000 7f4929000640 1 -- 172.21.15.28:0/1560827369 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4924160ef0 msgr2=0x7f492406de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:48.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.795+0000 7f4929000640 1 --2- 172.21.15.28:0/1560827369 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4924160ef0 0x7f492406de80 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f492406d720 tx=0x7f491c02a140 comp rx=0 tx=0).stop 2024-04-26T21:06:48.797 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.796+0000 7f4929000640 1 -- 172.21.15.28:0/1560827369 shutdown_connections 2024-04-26T21:06:48.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.796+0000 7f4929000640 1 --2- 172.21.15.28:0/1560827369 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4924160ef0 0x7f492406de80 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.796+0000 7f4929000640 1 --2- 172.21.15.28:0/1560827369 >> v2:172.21.15.103:3300/0 conn(0x7f4924146670 0x7f49241609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.796+0000 7f4929000640 1 --2- 172.21.15.28:0/1560827369 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4924146060 0x7f4924071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.796+0000 7f4929000640 1 -- 172.21.15.28:0/1560827369 >> 172.21.15.28:0/1560827369 conn(0x7f49240d0a50 msgr2=0x7f49240d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:48.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.796+0000 7f4929000640 1 -- 172.21.15.28:0/1560827369 shutdown_connections 2024-04-26T21:06:48.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.796+0000 7f4929000640 1 -- 172.21.15.28:0/1560827369 wait complete. 2024-04-26T21:06:48.798 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.797+0000 7f4929000640 1 Processor -- start 2024-04-26T21:06:48.799 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.798+0000 7f4929000640 1 -- start start 2024-04-26T21:06:48.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.799+0000 7f4929000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4924146060 0x7f49240ce600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.799+0000 7f4929000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4924146670 0x7f49240b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.799+0000 7f4929000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4924160ef0 0x7f49240cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.799+0000 7f4929000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4924161360 con 0x7f4924146670 2024-04-26T21:06:48.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.799+0000 7f4929000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f492406ff60 con 0x7f4924146060 2024-04-26T21:06:48.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.799+0000 7f4929000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f49241614e0 con 0x7f4924160ef0 2024-04-26T21:06:48.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.799+0000 7f4922a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4924146670 0x7f49240b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.799+0000 7f4922a00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4924146670 0x7f49240b5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34214/0 (socket says 172.21.15.28:34214) 2024-04-26T21:06:48.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.799+0000 7f4922a00640 1 -- 172.21.15.28:0/1816775205 learned_addr learned my addr 172.21.15.28:0/1816775205 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:48.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.799+0000 7f4923e00640 1 --2- 172.21.15.28:0/1816775205 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4924160ef0 0x7f49240cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.799+0000 7f4922a00640 1 -- 172.21.15.28:0/1816775205 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4924160ef0 msgr2=0x7f49240cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:48.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.799+0000 7f4923400640 1 --2- 172.21.15.28:0/1816775205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4924146060 0x7f49240ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.801 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.799+0000 7f4922a00640 1 --2- 172.21.15.28:0/1816775205 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4924160ef0 0x7f49240cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.799+0000 7f4922a00640 1 -- 172.21.15.28:0/1816775205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4924146060 msgr2=0x7f49240ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:48.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.799+0000 7f4922a00640 1 --2- 172.21.15.28:0/1816775205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4924146060 0x7f49240ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:48.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.799+0000 7f4922a00640 1 -- 172.21.15.28:0/1816775205 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49240cee30 con 0x7f4924146670 2024-04-26T21:06:48.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.799+0000 7f4923400640 1 --2- 172.21.15.28:0/1816775205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4924146060 0x7f49240ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:48.802 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.800+0000 7f4922a00640 1 --2- 172.21.15.28:0/1816775205 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4924146670 0x7f49240b5a10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f490c007e70 tx=0x7f490c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:48.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.800+0000 7f491b400640 1 -- 172.21.15.28:0/1816775205 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c014620 con 0x7f4924146670 2024-04-26T21:06:48.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.800+0000 7f491b400640 1 -- 172.21.15.28:0/1816775205 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f490c019070 con 0x7f4924146670 2024-04-26T21:06:48.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.800+0000 7f491b400640 1 -- 172.21.15.28:0/1816775205 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f490c014a00 con 0x7f4924146670 2024-04-26T21:06:48.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.800+0000 7f4929000640 1 -- 172.21.15.28:0/1816775205 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49240cf110 con 0x7f4924146670 2024-04-26T21:06:48.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.800+0000 7f4929000640 1 -- 172.21.15.28:0/1816775205 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49240cf6d0 con 0x7f4924146670 2024-04-26T21:06:48.803 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.802+0000 7f4929000640 1 -- 172.21.15.28:0/1816775205 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f492406a360 con 0x7f4924146670 2024-04-26T21:06:48.804 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.802+0000 7f491b400640 1 -- 172.21.15.28:0/1816775205 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f490c014ba0 con 0x7f4924146670 2024-04-26T21:06:48.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.804+0000 7f491b400640 1 --2- 172.21.15.28:0/1816775205 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f490006c8f0 0x7f490006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:48.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.804+0000 7f491b400640 1 -- 172.21.15.28:0/1816775205 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f490c08c0e0 con 0x7f4924146670 2024-04-26T21:06:48.805 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.804+0000 7f4923400640 1 --2- 172.21.15.28:0/1816775205 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f490006c8f0 0x7f490006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:48.806 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.805+0000 7f4923400640 1 --2- 172.21.15.28:0/1816775205 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f490006c8f0 0x7f490006edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4914006fd0 tx=0x7f4914008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:48.809 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:48.809+0000 7f491b400640 1 -- 172.21.15.28:0/1816775205 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f490c011260 con 0x7f4924146670 2024-04-26T21:06:49.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.012+0000 7f4929000640 1 -- 172.21.15.28:0/1816775205 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f4924072400 con 0x7f490006c8f0 2024-04-26T21:06:49.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.013+0000 7f491b400640 1 -- 172.21.15.28:0/1816775205 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4924072400 con 0x7f490006c8f0 2024-04-26T21:06:49.014 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 3 to deep-scrub 2024-04-26T21:06:49.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.016+0000 7f4929000640 1 -- 172.21.15.28:0/1816775205 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f490006c8f0 msgr2=0x7f490006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:49.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.016+0000 7f4929000640 1 --2- 172.21.15.28:0/1816775205 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f490006c8f0 0x7f490006edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4914006fd0 tx=0x7f4914008040 comp rx=0 tx=0).stop 2024-04-26T21:06:49.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.016+0000 7f4929000640 1 -- 172.21.15.28:0/1816775205 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4924146670 msgr2=0x7f49240b5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:49.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.016+0000 7f4929000640 1 --2- 172.21.15.28:0/1816775205 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4924146670 0x7f49240b5a10 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f490c007e70 tx=0x7f490c00e4b0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.016+0000 7f4929000640 1 -- 172.21.15.28:0/1816775205 shutdown_connections 2024-04-26T21:06:49.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.016+0000 7f4929000640 1 --2- 172.21.15.28:0/1816775205 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f490006c8f0 0x7f490006edb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.016+0000 7f4929000640 1 --2- 172.21.15.28:0/1816775205 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4924160ef0 0x7f49240cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.016+0000 7f4929000640 1 --2- 172.21.15.28:0/1816775205 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4924146670 0x7f49240b5a10 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.016+0000 7f4929000640 1 --2- 172.21.15.28:0/1816775205 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4924146060 0x7f49240ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.016+0000 7f4929000640 1 -- 172.21.15.28:0/1816775205 >> 172.21.15.28:0/1816775205 conn(0x7f49240d0a50 msgr2=0x7f492415f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:49.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.016+0000 7f4929000640 1 -- 172.21.15.28:0/1816775205 shutdown_connections 2024-04-26T21:06:49.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.016+0000 7f4929000640 1 -- 172.21.15.28:0/1816775205 wait complete. 2024-04-26T21:06:49.038 INFO:tasks.ceph:Scrubbing osd.4 2024-04-26T21:06:49.039 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:06:49.261 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.260+0000 7f0400c00640 1 Processor -- start 2024-04-26T21:06:49.262 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.260+0000 7f0400c00640 1 -- start start 2024-04-26T21:06:49.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.262+0000 7f0400c00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f03fc0b4140 0x7f03fc0ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:49.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.262+0000 7f0400c00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03fc0d2fe0 con 0x7f03fc15f0f0 2024-04-26T21:06:49.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.262+0000 7f0400c00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03fc0d32c0 con 0x7f03fc0bac10 2024-04-26T21:06:49.263 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.262+0000 7f0400c00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f03fc0d3bf0 con 0x7f03fc0b4140 2024-04-26T21:06:49.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.262+0000 7f03fb400640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f03fc15f0f0 0x7f03fc0b7ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40284/0 (socket says 172.21.15.28:40284) 2024-04-26T21:06:49.264 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.262+0000 7f03fb400640 1 -- 172.21.15.28:0/967892130 learned_addr learned my addr 172.21.15.28:0/967892130 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:06:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.262+0000 7f03fa000640 1 -- 172.21.15.28:0/967892130 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3455551940 0 0) 0x7f03fc0d2fe0 con 0x7f03fc15f0f0 2024-04-26T21:06:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.263+0000 7f03fa000640 1 -- 172.21.15.28:0/967892130 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03e0003520 con 0x7f03fc15f0f0 2024-04-26T21:06:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.263+0000 7f03faa00640 1 --2- 172.21.15.28:0/967892130 >> v2:172.21.15.103:3300/0 conn(0x7f03fc0b4140 0x7f03fc0ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.263+0000 7f03fa000640 1 -- 172.21.15.28:0/967892130 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2677334145 0 0) 0x7f03e0003520 con 0x7f03fc15f0f0 2024-04-26T21:06:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.263+0000 7f03fa000640 1 -- 172.21.15.28:0/967892130 >> v2:172.21.15.103:3300/0 conn(0x7f03fc0b4140 msgr2=0x7f03fc0ba6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.263+0000 7f03fa000640 1 --2- 172.21.15.28:0/967892130 >> v2:172.21.15.103:3300/0 conn(0x7f03fc0b4140 0x7f03fc0ba6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.263+0000 7f03fa000640 1 -- 172.21.15.28:0/967892130 >> v1:172.21.15.103:6789/0 conn(0x7f03fc0bac10 legacy=0x7f03fc0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.263+0000 7f03fa000640 1 -- 172.21.15.28:0/967892130 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc0d35a0 con 0x7f03fc15f0f0 2024-04-26T21:06:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.263+0000 7f03fa000640 1 -- 172.21.15.28:0/967892130 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f03e8001670 con 0x7f03fc15f0f0 2024-04-26T21:06:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.263+0000 7f03fa000640 1 -- 172.21.15.28:0/967892130 >> v1:172.21.15.28:6789/0 conn(0x7f03fc15f0f0 legacy=0x7f03fc0b7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.263+0000 7f03fa000640 1 --2- 172.21.15.28:0/967892130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03e00039b0 0x7f03e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.263+0000 7f03fa000640 1 --2- 172.21.15.28:0/967892130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03e0004800 0x7f03e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:49.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.263+0000 7f03fa000640 1 --2- 172.21.15.28:0/967892130 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f03e00051c0 0x7f03e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:49.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.263+0000 7f03fa000640 1 -- 172.21.15.28:0/967892130 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc0d3bf0 con 0x7f03e00039b0 2024-04-26T21:06:49.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.263+0000 7f03fa000640 1 -- 172.21.15.28:0/967892130 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03e000c0f0 con 0x7f03e0004800 2024-04-26T21:06:49.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.263+0000 7f03fa000640 1 -- 172.21.15.28:0/967892130 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f03e000c2f0 con 0x7f03e00051c0 2024-04-26T21:06:49.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.263+0000 7f03fb400640 1 --2- 172.21.15.28:0/967892130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03e00039b0 0x7f03e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:49.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.264+0000 7f03faa00640 1 --2- 172.21.15.28:0/967892130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03e0004800 0x7f03e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:49.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.264+0000 7f03fb400640 1 -- 172.21.15.28:0/967892130 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f03e00051c0 msgr2=0x7f03e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:49.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.264+0000 7f03fb400640 1 --2- 172.21.15.28:0/967892130 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f03e00051c0 0x7f03e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.264+0000 7f03fb400640 1 -- 172.21.15.28:0/967892130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03e0004800 msgr2=0x7f03e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:49.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.264+0000 7f03fb400640 1 --2- 172.21.15.28:0/967892130 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03e0004800 0x7f03e0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.264+0000 7f03fb400640 1 -- 172.21.15.28:0/967892130 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e0003520 con 0x7f03e00039b0 2024-04-26T21:06:49.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.264+0000 7f03fbe00640 1 -- 172.21.15.28:0/967892130 reap_dead start 2024-04-26T21:06:49.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.265+0000 7f03fb400640 1 --2- 172.21.15.28:0/967892130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03e00039b0 0x7f03e0003dd0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f03e800c240 tx=0x7f03e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34619ae916a6c9c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:49.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.265+0000 7f03fa000640 1 -- 172.21.15.28:0/967892130 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8007d10 con 0x7f03e00039b0 2024-04-26T21:06:49.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.265+0000 7f03fa000640 1 -- 172.21.15.28:0/967892130 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03e8005ad0 con 0x7f03e00039b0 2024-04-26T21:06:49.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.265+0000 7f03fa000640 1 -- 172.21.15.28:0/967892130 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8007070 con 0x7f03e00039b0 2024-04-26T21:06:49.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.266+0000 7f0400c00640 1 -- 172.21.15.28:0/967892130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03e00039b0 msgr2=0x7f03e0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:49.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.266+0000 7f0400c00640 1 --2- 172.21.15.28:0/967892130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03e00039b0 0x7f03e0003dd0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f03e800c240 tx=0x7f03e0005830 comp rx=0 tx=0).stop 2024-04-26T21:06:49.276 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.266+0000 7f0400c00640 1 -- 172.21.15.28:0/967892130 shutdown_connections 2024-04-26T21:06:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.266+0000 7f0400c00640 1 --2- 172.21.15.28:0/967892130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03e00039b0 0x7f03e0003dd0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.266+0000 7f0400c00640 1 -- 172.21.15.28:0/967892130 >> 172.21.15.28:0/967892130 conn(0x7f03fc06c980 msgr2=0x7f03fc072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.266+0000 7f0400c00640 1 -- 172.21.15.28:0/967892130 shutdown_connections 2024-04-26T21:06:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.266+0000 7f0400c00640 1 -- 172.21.15.28:0/967892130 wait complete. 2024-04-26T21:06:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.268+0000 7f0400c00640 1 Processor -- start 2024-04-26T21:06:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.268+0000 7f0400c00640 1 -- start start 2024-04-26T21:06:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.269+0000 7f0400c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03e00039b0 0x7f03fc155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.269+0000 7f0400c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f03fc159400 0x7f03fc1619c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.269+0000 7f0400c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc161f00 0x7f03fc0d43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.269+0000 7f0400c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc0ba850 con 0x7f03e00039b0 2024-04-26T21:06:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.269+0000 7f0400c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc0ba6d0 con 0x7f03fc161f00 2024-04-26T21:06:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.269+0000 7f0400c00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f03fc0ba9d0 con 0x7f03fc159400 2024-04-26T21:06:49.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.269+0000 7f03fb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03e00039b0 0x7f03fc155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:49.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.269+0000 7f03fb400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03e00039b0 0x7f03fc155150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34240/0 (socket says 172.21.15.28:34240) 2024-04-26T21:06:49.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.269+0000 7f03fb400640 1 -- 172.21.15.28:0/488501726 learned_addr learned my addr 172.21.15.28:0/488501726 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:49.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.269+0000 7f03faa00640 1 --2- 172.21.15.28:0/488501726 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f03fc159400 0x7f03fc1619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:49.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.270+0000 7f03fb400640 1 -- 172.21.15.28:0/488501726 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f03fc159400 msgr2=0x7f03fc1619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:49.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.270+0000 7f03fb400640 1 --2- 172.21.15.28:0/488501726 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f03fc159400 0x7f03fc1619c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.270+0000 7f03fb400640 1 -- 172.21.15.28:0/488501726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc161f00 msgr2=0x7f03fc0d43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:49.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.270+0000 7f03fb400640 1 --2- 172.21.15.28:0/488501726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f03fc161f00 0x7f03fc0d43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.270+0000 7f03fb400640 1 -- 172.21.15.28:0/488501726 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc0cbfe0 con 0x7f03e00039b0 2024-04-26T21:06:49.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.270+0000 7f03fb400640 1 --2- 172.21.15.28:0/488501726 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03e00039b0 0x7f03fc155150 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f03e8034040 tx=0x7f03fc0d3490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:49.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.270+0000 7f03f3400640 1 -- 172.21.15.28:0/488501726 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8042020 con 0x7f03e00039b0 2024-04-26T21:06:49.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.270+0000 7f03f3400640 1 -- 172.21.15.28:0/488501726 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03e8004bf0 con 0x7f03e00039b0 2024-04-26T21:06:49.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.270+0000 7f03f3400640 1 -- 172.21.15.28:0/488501726 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8007070 con 0x7f03e00039b0 2024-04-26T21:06:49.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.270+0000 7f0400c00640 1 -- 172.21.15.28:0/488501726 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03fc0cc280 con 0x7f03e00039b0 2024-04-26T21:06:49.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.270+0000 7f0400c00640 1 -- 172.21.15.28:0/488501726 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03fc0cc840 con 0x7f03e00039b0 2024-04-26T21:06:49.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.272+0000 7f0400c00640 1 -- 172.21.15.28:0/488501726 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f03fc0ccc20 con 0x7f03e00039b0 2024-04-26T21:06:49.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.273+0000 7f03f3400640 1 -- 172.21.15.28:0/488501726 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03e8045020 con 0x7f03e00039b0 2024-04-26T21:06:49.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.274+0000 7f03f3400640 1 --2- 172.21.15.28:0/488501726 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f03dc06c8f0 0x7f03dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:49.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.274+0000 7f03f3400640 1 -- 172.21.15.28:0/488501726 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f03e80ac150 con 0x7f03e00039b0 2024-04-26T21:06:49.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.274+0000 7f03f3400640 1 --2- 172.21.15.28:0/488501726 >> [v2:172.21.15.103:6817/472173438,v1:172.21.15.103:6819/472173438] conn(0x7f03dc0727b0 0x7f03dc074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:49.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.274+0000 7f03faa00640 1 --2- 172.21.15.28:0/488501726 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f03dc06c8f0 0x7f03dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:49.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.274+0000 7f03f3400640 1 -- 172.21.15.28:0/488501726 --> [v2:172.21.15.103:6817/472173438,v1:172.21.15.103:6819/472173438] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03e8009eb0 con 0x7f03dc0727b0 2024-04-26T21:06:49.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.274+0000 7f03f3400640 1 -- 172.21.15.28:0/488501726 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f03e80b3050 con 0x7f03e00039b0 2024-04-26T21:06:49.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.275+0000 7f03faa00640 1 --2- 172.21.15.28:0/488501726 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f03dc06c8f0 0x7f03dc06edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0d55e0 tx=0x7f03ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:49.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.275+0000 7f03fbe00640 1 --2- 172.21.15.28:0/488501726 >> [v2:172.21.15.103:6817/472173438,v1:172.21.15.103:6819/472173438] conn(0x7f03dc0727b0 0x7f03dc074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:49.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.276+0000 7f03fbe00640 1 --2- 172.21.15.28:0/488501726 >> [v2:172.21.15.103:6817/472173438,v1:172.21.15.103:6819/472173438] conn(0x7f03dc0727b0 0x7f03dc074bd0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:49.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.277+0000 7f03f3400640 1 -- 172.21.15.28:0/488501726 <== osd.4 v2:172.21.15.103:6817/472173438 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03e8009eb0 con 0x7f03dc0727b0 2024-04-26T21:06:49.293 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.292+0000 7f0400c00640 1 -- 172.21.15.28:0/488501726 --> [v2:172.21.15.103:6817/472173438,v1:172.21.15.103:6819/472173438] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f03fc06a3d0 con 0x7f03dc0727b0 2024-04-26T21:06:49.294 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.294+0000 7f03f3400640 1 -- 172.21.15.28:0/488501726 <== osd.4 v2:172.21.15.103:6817/472173438 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f03fc06a3d0 con 0x7f03dc0727b0 2024-04-26T21:06:49.295 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T21:06:49.295 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:06:49.295 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T21:06:49.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.294+0000 7f0400c00640 1 -- 172.21.15.28:0/488501726 >> [v2:172.21.15.103:6817/472173438,v1:172.21.15.103:6819/472173438] conn(0x7f03dc0727b0 msgr2=0x7f03dc074bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:49.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.294+0000 7f0400c00640 1 --2- 172.21.15.28:0/488501726 >> [v2:172.21.15.103:6817/472173438,v1:172.21.15.103:6819/472173438] conn(0x7f03dc0727b0 0x7f03dc074bd0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.294+0000 7f0400c00640 1 -- 172.21.15.28:0/488501726 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f03dc06c8f0 msgr2=0x7f03dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:49.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.294+0000 7f0400c00640 1 --2- 172.21.15.28:0/488501726 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f03dc06c8f0 0x7f03dc06edb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f03fc0d55e0 tx=0x7f03ec008040 comp rx=0 tx=0).stop 2024-04-26T21:06:49.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.294+0000 7f0400c00640 1 -- 172.21.15.28:0/488501726 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03e00039b0 msgr2=0x7f03fc155150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:49.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.294+0000 7f0400c00640 1 --2- 172.21.15.28:0/488501726 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03e00039b0 0x7f03fc155150 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f03e8034040 tx=0x7f03fc0d3490 comp rx=0 tx=0).stop 2024-04-26T21:06:49.296 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.294+0000 7f03fbe00640 1 -- 172.21.15.28:0/488501726 reap_dead start 2024-04-26T21:06:49.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.294+0000 7f0400c00640 1 -- 172.21.15.28:0/488501726 shutdown_connections 2024-04-26T21:06:49.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.294+0000 7f0400c00640 1 -- 172.21.15.28:0/488501726 >> 172.21.15.28:0/488501726 conn(0x7f03fc06c980 msgr2=0x7f03fc0b9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:49.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.294+0000 7f0400c00640 1 -- 172.21.15.28:0/488501726 shutdown_connections 2024-04-26T21:06:49.297 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.295+0000 7f0400c00640 1 -- 172.21.15.28:0/488501726 wait complete. 2024-04-26T21:06:49.314 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-26T21:06:49.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.522+0000 7f6712600640 1 Processor -- start 2024-04-26T21:06:49.523 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.522+0000 7f6712600640 1 -- start start 2024-04-26T21:06:49.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.523+0000 7f6712600640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f670c146060 0x7f670c071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:49.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.524+0000 7f6712600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f670c146670 0x7f670c1609b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:49.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.524+0000 7f6712600640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f670c146440 con 0x7f670c160ef0 2024-04-26T21:06:49.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.524+0000 7f6712600640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f670c06f510 con 0x7f670c146060 2024-04-26T21:06:49.525 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.524+0000 7f6712600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f670c06f010 con 0x7f670c146670 2024-04-26T21:06:49.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.524+0000 7f6703e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f670c146670 0x7f670c1609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:49.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.524+0000 7f6703e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f670c146670 0x7f670c1609b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3301/0 says I am v2:172.21.15.28:52064/0 (socket says 172.21.15.28:52064) 2024-04-26T21:06:49.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.525+0000 7f6703e00640 1 -- 172.21.15.28:0/2262685424 learned_addr learned my addr 172.21.15.28:0/2262685424 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:49.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.525+0000 7f6711200640 1 --2- 172.21.15.28:0/2262685424 >> v2:172.21.15.103:3300/0 conn(0x7f670c146060 0x7f670c071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:49.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.525+0000 7f6703e00640 1 -- 172.21.15.28:0/2262685424 >> v2:172.21.15.103:3300/0 conn(0x7f670c146060 msgr2=0x7f670c071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:49.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.525+0000 7f6703e00640 1 --2- 172.21.15.28:0/2262685424 >> v2:172.21.15.103:3300/0 conn(0x7f670c146060 0x7f670c071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.526 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.525+0000 7f6703e00640 1 -- 172.21.15.28:0/2262685424 >> v1:172.21.15.103:6789/0 conn(0x7f670c160ef0 legacy=0x7f670c06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:49.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.525+0000 7f6703e00640 1 -- 172.21.15.28:0/2262685424 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670c1617a0 con 0x7f670c146670 2024-04-26T21:06:49.527 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.527+0000 7f6703e00640 1 --2- 172.21.15.28:0/2262685424 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f670c146670 0x7f670c1609b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009870 tx=0x7f66f402a1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94271e2757bf516e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:49.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.527+0000 7f670be00640 1 -- 172.21.15.28:0/2262685424 <== mon.2 v2:172.21.15.103:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4037070 con 0x7f670c146670 2024-04-26T21:06:49.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.527+0000 7f670be00640 1 -- 172.21.15.28:0/2262685424 <== mon.2 v2:172.21.15.103:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66f4032440 con 0x7f670c146670 2024-04-26T21:06:49.528 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.527+0000 7f670be00640 1 -- 172.21.15.28:0/2262685424 <== mon.2 v2:172.21.15.103:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4032810 con 0x7f670c146670 2024-04-26T21:06:49.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.528+0000 7f6712600640 1 -- 172.21.15.28:0/2262685424 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f670c146670 msgr2=0x7f670c1609b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:49.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.528+0000 7f6712600640 1 --2- 172.21.15.28:0/2262685424 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f670c146670 0x7f670c1609b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009870 tx=0x7f66f402a1d0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.528+0000 7f6712600640 1 -- 172.21.15.28:0/2262685424 shutdown_connections 2024-04-26T21:06:49.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.528+0000 7f6712600640 1 --2- 172.21.15.28:0/2262685424 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f670c146670 0x7f670c1609b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.528+0000 7f6712600640 1 --2- 172.21.15.28:0/2262685424 >> v2:172.21.15.103:3300/0 conn(0x7f670c146060 0x7f670c071e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.529 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.528+0000 7f6712600640 1 -- 172.21.15.28:0/2262685424 >> 172.21.15.28:0/2262685424 conn(0x7f670c0d0a50 msgr2=0x7f670c0d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:49.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.528+0000 7f6712600640 1 -- 172.21.15.28:0/2262685424 shutdown_connections 2024-04-26T21:06:49.530 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.528+0000 7f6712600640 1 -- 172.21.15.28:0/2262685424 wait complete. 2024-04-26T21:06:49.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.530+0000 7f6712600640 1 Processor -- start 2024-04-26T21:06:49.531 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.530+0000 7f6712600640 1 -- start start 2024-04-26T21:06:49.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.531+0000 7f6712600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f670c146060 0x7f670c0b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:49.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.531+0000 7f6712600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f670c146670 0x7f670c139e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:49.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.531+0000 7f6712600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f670c160ef0 0x7f670c15add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:49.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.531+0000 7f6712600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f670c0cc0b0 con 0x7f670c160ef0 2024-04-26T21:06:49.532 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.531+0000 7f6712600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f670c06ff60 con 0x7f670c146060 2024-04-26T21:06:49.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.531+0000 7f6712600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f670c0cc230 con 0x7f670c146670 2024-04-26T21:06:49.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.531+0000 7f6711c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f670c160ef0 0x7f670c15add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:49.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.531+0000 7f6711c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f670c160ef0 0x7f670c15add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:34254/0 (socket says 172.21.15.28:34254) 2024-04-26T21:06:49.533 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.531+0000 7f6711c00640 1 -- 172.21.15.28:0/2778615000 learned_addr learned my addr 172.21.15.28:0/2778615000 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:49.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.532+0000 7f6711200640 1 --2- 172.21.15.28:0/2778615000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f670c146060 0x7f670c0b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:49.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.532+0000 7f6703e00640 1 --2- 172.21.15.28:0/2778615000 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f670c146670 0x7f670c139e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:49.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.532+0000 7f6711c00640 1 -- 172.21.15.28:0/2778615000 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f670c146670 msgr2=0x7f670c139e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:49.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.532+0000 7f6711c00640 1 --2- 172.21.15.28:0/2778615000 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f670c146670 0x7f670c139e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.532+0000 7f6711c00640 1 -- 172.21.15.28:0/2778615000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f670c146060 msgr2=0x7f670c0b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:49.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.532+0000 7f6711c00640 1 --2- 172.21.15.28:0/2778615000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f670c146060 0x7f670c0b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.534 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.532+0000 7f6711c00640 1 -- 172.21.15.28:0/2778615000 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670c0b69a0 con 0x7f670c160ef0 2024-04-26T21:06:49.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.532+0000 7f6703e00640 1 --2- 172.21.15.28:0/2778615000 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f670c146670 0x7f670c139e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:49.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.532+0000 7f6711200640 1 --2- 172.21.15.28:0/2778615000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f670c146060 0x7f670c0b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:49.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.532+0000 7f6711c00640 1 --2- 172.21.15.28:0/2778615000 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f670c160ef0 0x7f670c15add0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f670400c6b0 tx=0x7f670400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:49.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.532+0000 7f6709600640 1 -- 172.21.15.28:0/2778615000 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704003b70 con 0x7f670c160ef0 2024-04-26T21:06:49.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.532+0000 7f6709600640 1 -- 172.21.15.28:0/2778615000 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6704003d30 con 0x7f670c160ef0 2024-04-26T21:06:49.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.532+0000 7f6709600640 1 -- 172.21.15.28:0/2778615000 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704005310 con 0x7f670c160ef0 2024-04-26T21:06:49.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.532+0000 7f6712600640 1 -- 172.21.15.28:0/2778615000 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670c13c3b0 con 0x7f670c160ef0 2024-04-26T21:06:49.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.532+0000 7f6712600640 1 -- 172.21.15.28:0/2778615000 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f670c13c970 con 0x7f670c160ef0 2024-04-26T21:06:49.535 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.533+0000 7f6712600640 1 -- 172.21.15.28:0/2778615000 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f670c06a360 con 0x7f670c160ef0 2024-04-26T21:06:49.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.534+0000 7f6709600640 1 -- 172.21.15.28:0/2778615000 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f67040054b0 con 0x7f670c160ef0 2024-04-26T21:06:49.536 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.535+0000 7f6709600640 1 --2- 172.21.15.28:0/2778615000 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f66e806c920 0x7f66e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:49.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.536+0000 7f6709600640 1 -- 172.21.15.28:0/2778615000 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f670408b660 con 0x7f670c160ef0 2024-04-26T21:06:49.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.536+0000 7f6711200640 1 --2- 172.21.15.28:0/2778615000 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f66e806c920 0x7f66e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:49.537 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.536+0000 7f6711200640 1 --2- 172.21.15.28:0/2778615000 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f66e806c920 0x7f66e806ede0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f670c0cfc30 tx=0x7f66fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:49.541 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.540+0000 7f6709600640 1 -- 172.21.15.28:0/2778615000 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6704059d90 con 0x7f670c160ef0 2024-04-26T21:06:49.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.751+0000 7f6712600640 1 -- 172.21.15.28:0/2778615000 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f670c0d1620 con 0x7f66e806c920 2024-04-26T21:06:49.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.752+0000 7f6709600640 1 -- 172.21.15.28:0/2778615000 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f670c0d1620 con 0x7f66e806c920 2024-04-26T21:06:49.753 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 4 to deep-scrub 2024-04-26T21:06:49.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.755+0000 7f6712600640 1 -- 172.21.15.28:0/2778615000 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f66e806c920 msgr2=0x7f66e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:49.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.755+0000 7f6712600640 1 --2- 172.21.15.28:0/2778615000 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f66e806c920 0x7f66e806ede0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f670c0cfc30 tx=0x7f66fc008040 comp rx=0 tx=0).stop 2024-04-26T21:06:49.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.755+0000 7f6712600640 1 -- 172.21.15.28:0/2778615000 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f670c160ef0 msgr2=0x7f670c15add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:49.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.755+0000 7f6712600640 1 --2- 172.21.15.28:0/2778615000 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f670c160ef0 0x7f670c15add0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f670400c6b0 tx=0x7f670400cb70 comp rx=0 tx=0).stop 2024-04-26T21:06:49.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.755+0000 7f6712600640 1 -- 172.21.15.28:0/2778615000 shutdown_connections 2024-04-26T21:06:49.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.755+0000 7f6712600640 1 --2- 172.21.15.28:0/2778615000 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f66e806c920 0x7f66e806ede0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.756 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.755+0000 7f6712600640 1 --2- 172.21.15.28:0/2778615000 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f670c160ef0 0x7f670c15add0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.755+0000 7f6712600640 1 --2- 172.21.15.28:0/2778615000 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f670c146670 0x7f670c139e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.755+0000 7f6712600640 1 --2- 172.21.15.28:0/2778615000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f670c146060 0x7f670c0b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:49.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.755+0000 7f6712600640 1 -- 172.21.15.28:0/2778615000 >> 172.21.15.28:0/2778615000 conn(0x7f670c0d0a50 msgr2=0x7f670c15f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:49.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.755+0000 7f6712600640 1 -- 172.21.15.28:0/2778615000 shutdown_connections 2024-04-26T21:06:49.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.755+0000 7f6712600640 1 -- 172.21.15.28:0/2778615000 wait complete. 2024-04-26T21:06:49.774 INFO:tasks.ceph:Scrubbing osd.5 2024-04-26T21:06:49.775 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:06:49.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.994+0000 7f66b7c00640 1 Processor -- start 2024-04-26T21:06:49.996 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.995+0000 7f66b7c00640 1 -- start start 2024-04-26T21:06:49.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.996+0000 7f66b7c00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f66b015cc90 0x7f66b00b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:49.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.997+0000 7f66b7c00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f66b015f7d0 0x7f66b015fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:49.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.997+0000 7f66b7c00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66b00b64e0 con 0x7f66b0160150 2024-04-26T21:06:49.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.997+0000 7f66b7c00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f66b0072480 con 0x7f66b015cc90 2024-04-26T21:06:49.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.997+0000 7f66b7c00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f66b00b67c0 con 0x7f66b015f7d0 2024-04-26T21:06:49.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.997+0000 7f66b5200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f66b015cc90 0x7f66b00b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:49.998 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.997+0000 7f66b5200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f66b015cc90 0x7f66b00b5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54124/0 (socket says 172.21.15.28:54124) 2024-04-26T21:06:49.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.997+0000 7f66b5200640 1 -- 172.21.15.28:0/4010254309 learned_addr learned my addr 172.21.15.28:0/4010254309 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:49.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.997+0000 7f66afe00640 1 --2- 172.21.15.28:0/4010254309 >> v2:172.21.15.103:3300/0 conn(0x7f66b015f7d0 0x7f66b015fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:49.999 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.998+0000 7f66b5200640 1 -- 172.21.15.28:0/4010254309 >> v2:172.21.15.103:3300/0 conn(0x7f66b015f7d0 msgr2=0x7f66b015fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:50.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.998+0000 7f66b5200640 1 --2- 172.21.15.28:0/4010254309 >> v2:172.21.15.103:3300/0 conn(0x7f66b015f7d0 0x7f66b015fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.998+0000 7f66b5200640 1 -- 172.21.15.28:0/4010254309 >> v1:172.21.15.103:6789/0 conn(0x7f66b0160150 legacy=0x7f66b0162dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:50.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.998+0000 7f66b5200640 1 -- 172.21.15.28:0/4010254309 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b00b6aa0 con 0x7f66b015cc90 2024-04-26T21:06:50.000 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.999+0000 7f66b5200640 1 --2- 172.21.15.28:0/4010254309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66b015cc90 0x7f66b00b5bb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f66b0071c20 tx=0x7f66a002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc5a695df619b9c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:50.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.999+0000 7f66af400640 1 -- 172.21.15.28:0/4010254309 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0036070 con 0x7f66b015cc90 2024-04-26T21:06:50.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.999+0000 7f66af400640 1 -- 172.21.15.28:0/4010254309 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66a0031600 con 0x7f66b015cc90 2024-04-26T21:06:50.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:49.999+0000 7f66af400640 1 -- 172.21.15.28:0/4010254309 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a00319d0 con 0x7f66b015cc90 2024-04-26T21:06:50.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.000+0000 7f66b7c00640 1 -- 172.21.15.28:0/4010254309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66b015cc90 msgr2=0x7f66b00b5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:50.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.000+0000 7f66b7c00640 1 --2- 172.21.15.28:0/4010254309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66b015cc90 0x7f66b00b5bb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f66b0071c20 tx=0x7f66a002a6f0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.000+0000 7f66b7c00640 1 -- 172.21.15.28:0/4010254309 shutdown_connections 2024-04-26T21:06:50.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.000+0000 7f66b7c00640 1 --2- 172.21.15.28:0/4010254309 >> v2:172.21.15.103:3300/0 conn(0x7f66b015f7d0 0x7f66b015fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.000+0000 7f66b7c00640 1 --2- 172.21.15.28:0/4010254309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66b015cc90 0x7f66b00b5bb0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.001 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.000+0000 7f66b7c00640 1 -- 172.21.15.28:0/4010254309 >> 172.21.15.28:0/4010254309 conn(0x7f66b006e220 msgr2=0x7f66b006e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:50.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.000+0000 7f66b7c00640 1 -- 172.21.15.28:0/4010254309 shutdown_connections 2024-04-26T21:06:50.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.000+0000 7f66b7c00640 1 -- 172.21.15.28:0/4010254309 wait complete. 2024-04-26T21:06:50.002 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.002+0000 7f66b7c00640 1 Processor -- start 2024-04-26T21:06:50.003 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.002+0000 7f66b7c00640 1 -- start start 2024-04-26T21:06:50.004 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.003+0000 7f66b7c00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66b015cc90 0x7f66b015bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:50.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.003+0000 7f66b7c00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66b015f7d0 0x7f66b00cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:50.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.003+0000 7f66b7c00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66b0160150 0x7f66b00c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:50.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.003+0000 7f66b7c00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f66b015d070 con 0x7f66b015f7d0 2024-04-26T21:06:50.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.003+0000 7f66b7c00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f66b0160600 con 0x7f66b0160150 2024-04-26T21:06:50.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.003+0000 7f66b7c00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f66b00b60f0 con 0x7f66b015cc90 2024-04-26T21:06:50.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.003+0000 7f66afe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66b015f7d0 0x7f66b00cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:50.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.003+0000 7f66afe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66b015f7d0 0x7f66b00cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54140/0 (socket says 172.21.15.28:54140) 2024-04-26T21:06:50.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.003+0000 7f66afe00640 1 -- 172.21.15.28:0/2033083086 learned_addr learned my addr 172.21.15.28:0/2033083086 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:50.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.003+0000 7f66afe00640 1 -- 172.21.15.28:0/2033083086 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66b015cc90 msgr2=0x7f66b015bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:50.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.003+0000 7f66afe00640 1 --2- 172.21.15.28:0/2033083086 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f66b015cc90 0x7f66b015bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.003+0000 7f66afe00640 1 -- 172.21.15.28:0/2033083086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66b0160150 msgr2=0x7f66b00c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:50.005 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.003+0000 7f66afe00640 1 --2- 172.21.15.28:0/2033083086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f66b0160150 0x7f66b00c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.004+0000 7f66afe00640 1 -- 172.21.15.28:0/2033083086 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b015c1f0 con 0x7f66b015f7d0 2024-04-26T21:06:50.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.004+0000 7f66afe00640 1 --2- 172.21.15.28:0/2033083086 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66b015f7d0 0x7f66b00cb250 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f669800d7d0 tx=0x7f669800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:50.006 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.004+0000 7f66acc00640 1 -- 172.21.15.28:0/2033083086 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698004020 con 0x7f66b015f7d0 2024-04-26T21:06:50.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.004+0000 7f66acc00640 1 -- 172.21.15.28:0/2033083086 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66980041e0 con 0x7f66b015f7d0 2024-04-26T21:06:50.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.004+0000 7f66b7c00640 1 -- 172.21.15.28:0/2033083086 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b00ca1b0 con 0x7f66b015f7d0 2024-04-26T21:06:50.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.004+0000 7f66acc00640 1 -- 172.21.15.28:0/2033083086 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66980043d0 con 0x7f66b015f7d0 2024-04-26T21:06:50.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.004+0000 7f66b7c00640 1 -- 172.21.15.28:0/2033083086 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b00ca770 con 0x7f66b015f7d0 2024-04-26T21:06:50.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.005+0000 7f66b7c00640 1 -- 172.21.15.28:0/2033083086 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f66b0076520 con 0x7f66b015f7d0 2024-04-26T21:06:50.007 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.006+0000 7f66acc00640 1 -- 172.21.15.28:0/2033083086 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f669801b020 con 0x7f66b015f7d0 2024-04-26T21:06:50.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.007+0000 7f66acc00640 1 --2- 172.21.15.28:0/2033083086 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f668c06c8f0 0x7f668c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:50.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.008+0000 7f66acc00640 1 -- 172.21.15.28:0/2033083086 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f669808b760 con 0x7f66b015f7d0 2024-04-26T21:06:50.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.008+0000 7f66acc00640 1 --2- 172.21.15.28:0/2033083086 >> [v2:172.21.15.103:6800/4239183489,v1:172.21.15.103:6801/4239183489] conn(0x7f668c0727b0 0x7f668c074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:50.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.008+0000 7f66acc00640 1 -- 172.21.15.28:0/2033083086 --> [v2:172.21.15.103:6800/4239183489,v1:172.21.15.103:6801/4239183489] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6698006810 con 0x7f668c0727b0 2024-04-26T21:06:50.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.008+0000 7f66acc00640 1 -- 172.21.15.28:0/2033083086 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f6698092050 con 0x7f66b015f7d0 2024-04-26T21:06:50.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.008+0000 7f66b5200640 1 --2- 172.21.15.28:0/2033083086 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f668c06c8f0 0x7f668c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:50.009 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.008+0000 7f66b5c00640 1 --2- 172.21.15.28:0/2033083086 >> [v2:172.21.15.103:6800/4239183489,v1:172.21.15.103:6801/4239183489] conn(0x7f668c0727b0 0x7f668c074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:50.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.009+0000 7f66b5200640 1 --2- 172.21.15.28:0/2033083086 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f668c06c8f0 0x7f668c06edb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f66a0033000 tx=0x7f66a002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:50.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.009+0000 7f66b5c00640 1 --2- 172.21.15.28:0/2033083086 >> [v2:172.21.15.103:6800/4239183489,v1:172.21.15.103:6801/4239183489] conn(0x7f668c0727b0 0x7f668c074bd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:50.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.012+0000 7f66acc00640 1 -- 172.21.15.28:0/2033083086 <== osd.5 v2:172.21.15.103:6800/4239183489 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6698006810 con 0x7f668c0727b0 2024-04-26T21:06:50.028 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.027+0000 7f66b7c00640 1 -- 172.21.15.28:0/2033083086 --> [v2:172.21.15.103:6800/4239183489,v1:172.21.15.103:6801/4239183489] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f66b006ff00 con 0x7f668c0727b0 2024-04-26T21:06:50.029 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.028+0000 7f66acc00640 1 -- 172.21.15.28:0/2033083086 <== osd.5 v2:172.21.15.103:6800/4239183489 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f66b006ff00 con 0x7f668c0727b0 2024-04-26T21:06:50.029 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T21:06:50.029 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:06:50.030 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T21:06:50.030 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.029+0000 7f66b7c00640 1 -- 172.21.15.28:0/2033083086 >> [v2:172.21.15.103:6800/4239183489,v1:172.21.15.103:6801/4239183489] conn(0x7f668c0727b0 msgr2=0x7f668c074bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:50.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.029+0000 7f66b7c00640 1 --2- 172.21.15.28:0/2033083086 >> [v2:172.21.15.103:6800/4239183489,v1:172.21.15.103:6801/4239183489] conn(0x7f668c0727b0 0x7f668c074bd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.029+0000 7f66b7c00640 1 -- 172.21.15.28:0/2033083086 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f668c06c8f0 msgr2=0x7f668c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:50.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.029+0000 7f66b7c00640 1 --2- 172.21.15.28:0/2033083086 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f668c06c8f0 0x7f668c06edb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f66a0033000 tx=0x7f66a002e000 comp rx=0 tx=0).stop 2024-04-26T21:06:50.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.029+0000 7f66b7c00640 1 -- 172.21.15.28:0/2033083086 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66b015f7d0 msgr2=0x7f66b00cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:50.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.029+0000 7f66b7c00640 1 --2- 172.21.15.28:0/2033083086 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66b015f7d0 0x7f66b00cb250 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f669800d7d0 tx=0x7f669800dc90 comp rx=0 tx=0).stop 2024-04-26T21:06:50.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.029+0000 7f66b5c00640 1 -- 172.21.15.28:0/2033083086 reap_dead start 2024-04-26T21:06:50.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.029+0000 7f66b7c00640 1 -- 172.21.15.28:0/2033083086 shutdown_connections 2024-04-26T21:06:50.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.029+0000 7f66b7c00640 1 -- 172.21.15.28:0/2033083086 >> 172.21.15.28:0/2033083086 conn(0x7f66b006e220 msgr2=0x7f66b01615d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:50.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.029+0000 7f66b7c00640 1 -- 172.21.15.28:0/2033083086 shutdown_connections 2024-04-26T21:06:50.031 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.029+0000 7f66b7c00640 1 -- 172.21.15.28:0/2033083086 wait complete. 2024-04-26T21:06:50.052 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-26T21:06:50.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.266+0000 7f4c88000640 1 Processor -- start 2024-04-26T21:06:50.267 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.266+0000 7f4c88000640 1 -- start start 2024-04-26T21:06:50.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.267+0000 7f4c88000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c80163c00 0x7f4c800b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:50.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.268+0000 7f4c88000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4c800b5ff0 0x7f4c80165330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:50.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.268+0000 7f4c88000640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c800b4740 con 0x7f4c801612d0 2024-04-26T21:06:50.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.268+0000 7f4c88000640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f4c800b6450 con 0x7f4c800b5ff0 2024-04-26T21:06:50.269 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.268+0000 7f4c88000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f4c800b4500 con 0x7f4c80163c00 2024-04-26T21:06:50.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.268+0000 7f4c86000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4c800b5ff0 0x7f4c80165330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:50.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.268+0000 7f4c86000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4c800b5ff0 0x7f4c80165330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54152/0 (socket says 172.21.15.28:54152) 2024-04-26T21:06:50.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.268+0000 7f4c86000640 1 -- 172.21.15.28:0/3827752669 learned_addr learned my addr 172.21.15.28:0/3827752669 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:50.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.268+0000 7f4c84c00640 1 --2- 172.21.15.28:0/3827752669 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c80163c00 0x7f4c800b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:50.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.269+0000 7f4c86000640 1 -- 172.21.15.28:0/3827752669 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c80163c00 msgr2=0x7f4c800b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:50.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.269+0000 7f4c86000640 1 --2- 172.21.15.28:0/3827752669 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c80163c00 0x7f4c800b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.269+0000 7f4c86000640 1 -- 172.21.15.28:0/3827752669 >> v1:172.21.15.103:6789/0 conn(0x7f4c801612d0 legacy=0x7f4c800b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:50.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.269+0000 7f4c86000640 1 -- 172.21.15.28:0/3827752669 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c800b4a20 con 0x7f4c800b5ff0 2024-04-26T21:06:50.270 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.269+0000 7f4c84c00640 1 --2- 172.21.15.28:0/3827752669 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c80163c00 0x7f4c800b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T21:06:50.271 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.270+0000 7f4c86000640 1 --2- 172.21.15.28:0/3827752669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c800b5ff0 0x7f4c80165330 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009e50 tx=0x7f4c7802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e3991a0b255747 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:50.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.271+0000 7f4c7fe00640 1 -- 172.21.15.28:0/3827752669 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c78036070 con 0x7f4c800b5ff0 2024-04-26T21:06:50.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.271+0000 7f4c7fe00640 1 -- 172.21.15.28:0/3827752669 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c78031600 con 0x7f4c800b5ff0 2024-04-26T21:06:50.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.271+0000 7f4c7fe00640 1 -- 172.21.15.28:0/3827752669 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c780319d0 con 0x7f4c800b5ff0 2024-04-26T21:06:50.272 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.271+0000 7f4c88000640 1 -- 172.21.15.28:0/3827752669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c800b5ff0 msgr2=0x7f4c80165330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:50.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.271+0000 7f4c88000640 1 --2- 172.21.15.28:0/3827752669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c800b5ff0 0x7f4c80165330 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009e50 tx=0x7f4c7802a710 comp rx=0 tx=0).stop 2024-04-26T21:06:50.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.271+0000 7f4c88000640 1 -- 172.21.15.28:0/3827752669 shutdown_connections 2024-04-26T21:06:50.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.271+0000 7f4c88000640 1 --2- 172.21.15.28:0/3827752669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c800b5ff0 0x7f4c80165330 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.271+0000 7f4c88000640 1 --2- 172.21.15.28:0/3827752669 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c80163c00 0x7f4c800b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.271+0000 7f4c88000640 1 -- 172.21.15.28:0/3827752669 >> 172.21.15.28:0/3827752669 conn(0x7f4c8006e220 msgr2=0x7f4c8006e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:50.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.272+0000 7f4c88000640 1 -- 172.21.15.28:0/3827752669 shutdown_connections 2024-04-26T21:06:50.273 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.272+0000 7f4c88000640 1 -- 172.21.15.28:0/3827752669 wait complete. 2024-04-26T21:06:50.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.274+0000 7f4c88000640 1 Processor -- start 2024-04-26T21:06:50.275 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.274+0000 7f4c88000640 1 -- start start 2024-04-26T21:06:50.277 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.275+0000 7f4c88000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c800b5ff0 0x7f4c800ce380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.275+0000 7f4c88000640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c801612d0 0x7f4c80158fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.275+0000 7f4c88000640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c80163c00 0x7f4c8014ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.275+0000 7f4c88000640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f4c80165870 con 0x7f4c800b5ff0 2024-04-26T21:06:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.275+0000 7f4c88000640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c80164e10 con 0x7f4c80163c00 2024-04-26T21:06:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.275+0000 7f4c88000640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f4c801659f0 con 0x7f4c801612d0 2024-04-26T21:06:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.275+0000 7f4c85600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c800b5ff0 0x7f4c800ce380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.275+0000 7f4c85600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c800b5ff0 0x7f4c800ce380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54164/0 (socket says 172.21.15.28:54164) 2024-04-26T21:06:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.275+0000 7f4c85600640 1 -- 172.21.15.28:0/1450366462 learned_addr learned my addr 172.21.15.28:0/1450366462 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.276+0000 7f4c84c00640 1 --2- 172.21.15.28:0/1450366462 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c801612d0 0x7f4c80158fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.276+0000 7f4c86000640 1 --2- 172.21.15.28:0/1450366462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c80163c00 0x7f4c8014ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.276+0000 7f4c85600640 1 -- 172.21.15.28:0/1450366462 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c801612d0 msgr2=0x7f4c80158fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.276+0000 7f4c85600640 1 --2- 172.21.15.28:0/1450366462 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c801612d0 0x7f4c80158fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.276+0000 7f4c85600640 1 -- 172.21.15.28:0/1450366462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c80163c00 msgr2=0x7f4c8014ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.276+0000 7f4c85600640 1 --2- 172.21.15.28:0/1450366462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c80163c00 0x7f4c8014ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.276+0000 7f4c85600640 1 -- 172.21.15.28:0/1450366462 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c800ce960 con 0x7f4c800b5ff0 2024-04-26T21:06:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.276+0000 7f4c85600640 1 --2- 172.21.15.28:0/1450366462 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c800b5ff0 0x7f4c800ce380 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400c400 tx=0x7f4c7400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.277+0000 7f4c7d600640 1 -- 172.21.15.28:0/1450366462 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c740052d0 con 0x7f4c800b5ff0 2024-04-26T21:06:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.277+0000 7f4c7d600640 1 -- 172.21.15.28:0/1450366462 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c74005490 con 0x7f4c800b5ff0 2024-04-26T21:06:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.277+0000 7f4c7d600640 1 -- 172.21.15.28:0/1450366462 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74007260 con 0x7f4c800b5ff0 2024-04-26T21:06:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.277+0000 7f4c88000640 1 -- 172.21.15.28:0/1450366462 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c800cec40 con 0x7f4c800b5ff0 2024-04-26T21:06:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.277+0000 7f4c88000640 1 -- 172.21.15.28:0/1450366462 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c800cf4e0 con 0x7f4c800b5ff0 2024-04-26T21:06:50.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.278+0000 7f4c88000640 1 -- 172.21.15.28:0/1450366462 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c800cf030 con 0x7f4c800b5ff0 2024-04-26T21:06:50.280 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.279+0000 7f4c7d600640 1 -- 172.21.15.28:0/1450366462 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4c74007400 con 0x7f4c800b5ff0 2024-04-26T21:06:50.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.280+0000 7f4c7d600640 1 --2- 172.21.15.28:0/1450366462 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4c6006c8f0 0x7f4c6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:50.281 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.280+0000 7f4c7d600640 1 -- 172.21.15.28:0/1450366462 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f4c7408ae80 con 0x7f4c800b5ff0 2024-04-26T21:06:50.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.280+0000 7f4c84c00640 1 --2- 172.21.15.28:0/1450366462 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4c6006c8f0 0x7f4c6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:50.282 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.281+0000 7f4c84c00640 1 --2- 172.21.15.28:0/1450366462 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4c6006c8f0 0x7f4c6006edb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4c801fc960 tx=0x7f4c6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:50.286 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.285+0000 7f4c7d600640 1 -- 172.21.15.28:0/1450366462 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4c74059510 con 0x7f4c800b5ff0 2024-04-26T21:06:50.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.491+0000 7f4c88000640 1 -- 172.21.15.28:0/1450366462 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f4c800cee10 con 0x7f4c6006c8f0 2024-04-26T21:06:50.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.492+0000 7f4c7d600640 1 -- 172.21.15.28:0/1450366462 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4c800cee10 con 0x7f4c6006c8f0 2024-04-26T21:06:50.493 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 5 to deep-scrub 2024-04-26T21:06:50.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.494+0000 7f4c88000640 1 -- 172.21.15.28:0/1450366462 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4c6006c8f0 msgr2=0x7f4c6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:50.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.494+0000 7f4c88000640 1 --2- 172.21.15.28:0/1450366462 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4c6006c8f0 0x7f4c6006edb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4c801fc960 tx=0x7f4c6c008040 comp rx=0 tx=0).stop 2024-04-26T21:06:50.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.494+0000 7f4c88000640 1 -- 172.21.15.28:0/1450366462 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c800b5ff0 msgr2=0x7f4c800ce380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:50.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.494+0000 7f4c88000640 1 --2- 172.21.15.28:0/1450366462 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c800b5ff0 0x7f4c800ce380 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4c7400c400 tx=0x7f4c7400c8c0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.494+0000 7f4c88000640 1 -- 172.21.15.28:0/1450366462 shutdown_connections 2024-04-26T21:06:50.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.494+0000 7f4c88000640 1 --2- 172.21.15.28:0/1450366462 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f4c6006c8f0 0x7f4c6006edb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.494+0000 7f4c88000640 1 --2- 172.21.15.28:0/1450366462 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c80163c00 0x7f4c8014ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.494+0000 7f4c88000640 1 --2- 172.21.15.28:0/1450366462 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f4c801612d0 0x7f4c80158fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.494+0000 7f4c88000640 1 --2- 172.21.15.28:0/1450366462 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c800b5ff0 0x7f4c800ce380 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.494+0000 7f4c88000640 1 -- 172.21.15.28:0/1450366462 >> 172.21.15.28:0/1450366462 conn(0x7f4c8006e220 msgr2=0x7f4c800b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:50.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.494+0000 7f4c88000640 1 -- 172.21.15.28:0/1450366462 shutdown_connections 2024-04-26T21:06:50.496 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.494+0000 7f4c88000640 1 -- 172.21.15.28:0/1450366462 wait complete. 2024-04-26T21:06:50.515 INFO:tasks.ceph:Scrubbing osd.6 2024-04-26T21:06:50.515 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:06:50.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.736+0000 7fa847e00640 1 Processor -- start 2024-04-26T21:06:50.737 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.736+0000 7fa847e00640 1 -- start start 2024-04-26T21:06:50.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.738+0000 7fa847e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa84806f0b0 0x7fa848167e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:50.739 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.738+0000 7fa847e00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa84806f6d0 con 0x7fa848161240 2024-04-26T21:06:50.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.738+0000 7fa847e00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8480ceb10 con 0x7fa8481434c0 2024-04-26T21:06:50.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.738+0000 7fa847e00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fa848143ce0 con 0x7fa84806f0b0 2024-04-26T21:06:50.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.738+0000 7fa847400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa84806f0b0 0x7fa848167e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:50.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.738+0000 7fa847400640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa84806f0b0 0x7fa848167e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54178/0 (socket says 172.21.15.28:54178) 2024-04-26T21:06:50.740 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.738+0000 7fa847400640 1 -- 172.21.15.28:0/4085140245 learned_addr learned my addr 172.21.15.28:0/4085140245 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:50.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.739+0000 7fa847400640 1 -- 172.21.15.28:0/4085140245 >> v1:172.21.15.103:6789/0 conn(0x7fa8481434c0 legacy=0x7fa848160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:50.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.739+0000 7fa847400640 1 -- 172.21.15.28:0/4085140245 >> v1:172.21.15.28:6789/0 conn(0x7fa848161240 legacy=0x7fa84806e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:50.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.739+0000 7fa847400640 1 -- 172.21.15.28:0/4085140245 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8480cedf0 con 0x7fa84806f0b0 2024-04-26T21:06:50.741 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.740+0000 7fa847400640 1 --2- 172.21.15.28:0/4085140245 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa84806f0b0 0x7fa848167e60 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa840009e50 tx=0x7fa84002a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2cfbabf5cb1667 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:50.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.740+0000 7fa845600640 1 -- 172.21.15.28:0/4085140245 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840036070 con 0x7fa84806f0b0 2024-04-26T21:06:50.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.740+0000 7fa845600640 1 -- 172.21.15.28:0/4085140245 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa840031600 con 0x7fa84806f0b0 2024-04-26T21:06:50.742 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.740+0000 7fa845600640 1 -- 172.21.15.28:0/4085140245 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8400319d0 con 0x7fa84806f0b0 2024-04-26T21:06:50.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.741+0000 7fa847e00640 1 -- 172.21.15.28:0/4085140245 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa84806f0b0 msgr2=0x7fa848167e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:50.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.741+0000 7fa847e00640 1 --2- 172.21.15.28:0/4085140245 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa84806f0b0 0x7fa848167e60 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa840009e50 tx=0x7fa84002a7f0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.741+0000 7fa847e00640 1 -- 172.21.15.28:0/4085140245 shutdown_connections 2024-04-26T21:06:50.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.741+0000 7fa847e00640 1 --2- 172.21.15.28:0/4085140245 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa84806f0b0 0x7fa848167e60 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.741+0000 7fa847e00640 1 -- 172.21.15.28:0/4085140245 >> 172.21.15.28:0/4085140245 conn(0x7fa8480b2820 msgr2=0x7fa8480b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:50.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.741+0000 7fa847e00640 1 -- 172.21.15.28:0/4085140245 shutdown_connections 2024-04-26T21:06:50.743 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.741+0000 7fa847e00640 1 -- 172.21.15.28:0/4085140245 wait complete. 2024-04-26T21:06:50.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.743+0000 7fa847e00640 1 Processor -- start 2024-04-26T21:06:50.744 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.743+0000 7fa847e00640 1 -- start start 2024-04-26T21:06:50.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.744+0000 7fa847e00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa84806f0b0 0x7fa84813e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:50.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.744+0000 7fa847e00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8481434c0 0x7fa848073440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:50.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.744+0000 7fa847e00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa848161240 0x7fa848140b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:50.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.744+0000 7fa847e00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fa848143a20 con 0x7fa8481434c0 2024-04-26T21:06:50.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.744+0000 7fa847e00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa8481438a0 con 0x7fa848161240 2024-04-26T21:06:50.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.744+0000 7fa847e00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fa8480d0600 con 0x7fa84806f0b0 2024-04-26T21:06:50.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.744+0000 7fa846000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8481434c0 0x7fa848073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:50.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.744+0000 7fa846000640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8481434c0 0x7fa848073440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54186/0 (socket says 172.21.15.28:54186) 2024-04-26T21:06:50.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.744+0000 7fa846000640 1 -- 172.21.15.28:0/2675005146 learned_addr learned my addr 172.21.15.28:0/2675005146 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:50.747 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.744+0000 7fa846a00640 1 --2- 172.21.15.28:0/2675005146 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa84806f0b0 0x7fa84813e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.744+0000 7fa846000640 1 -- 172.21.15.28:0/2675005146 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa84806f0b0 msgr2=0x7fa84813e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.744+0000 7fa846000640 1 --2- 172.21.15.28:0/2675005146 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa84806f0b0 0x7fa84813e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.745+0000 7fa846000640 1 -- 172.21.15.28:0/2675005146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa848161240 msgr2=0x7fa848140b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.745+0000 7fa847400640 1 --2- 172.21.15.28:0/2675005146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa848161240 0x7fa848140b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.745+0000 7fa846000640 1 --2- 172.21.15.28:0/2675005146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa848161240 0x7fa848140b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.745+0000 7fa846000640 1 -- 172.21.15.28:0/2675005146 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8480723a0 con 0x7fa8481434c0 2024-04-26T21:06:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.745+0000 7fa846a00640 1 --2- 172.21.15.28:0/2675005146 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fa84806f0b0 0x7fa84813e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.745+0000 7fa846000640 1 --2- 172.21.15.28:0/2675005146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8481434c0 0x7fa848073440 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa83400c400 tx=0x7fa83400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.745+0000 7fa83ea00640 1 -- 172.21.15.28:0/2675005146 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8340052d0 con 0x7fa8481434c0 2024-04-26T21:06:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.745+0000 7fa83ea00640 1 -- 172.21.15.28:0/2675005146 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa834005490 con 0x7fa8481434c0 2024-04-26T21:06:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.745+0000 7fa83ea00640 1 -- 172.21.15.28:0/2675005146 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834007260 con 0x7fa8481434c0 2024-04-26T21:06:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.745+0000 7fa847e00640 1 -- 172.21.15.28:0/2675005146 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa848072680 con 0x7fa8481434c0 2024-04-26T21:06:50.748 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.745+0000 7fa847e00640 1 -- 172.21.15.28:0/2675005146 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa848072c40 con 0x7fa8481434c0 2024-04-26T21:06:50.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.746+0000 7fa847e00640 1 -- 172.21.15.28:0/2675005146 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa84806fd70 con 0x7fa8481434c0 2024-04-26T21:06:50.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.748+0000 7fa83ea00640 1 -- 172.21.15.28:0/2675005146 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa834007400 con 0x7fa8481434c0 2024-04-26T21:06:50.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.748+0000 7fa83ea00640 1 --2- 172.21.15.28:0/2675005146 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa82806c8f0 0x7fa82806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:50.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.749+0000 7fa83ea00640 1 -- 172.21.15.28:0/2675005146 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fa83408ad80 con 0x7fa8481434c0 2024-04-26T21:06:50.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.749+0000 7fa846a00640 1 --2- 172.21.15.28:0/2675005146 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa82806c8f0 0x7fa82806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:50.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.749+0000 7fa83ea00640 1 --2- 172.21.15.28:0/2675005146 >> [v2:172.21.15.103:6816/1745016180,v1:172.21.15.103:6818/1745016180] conn(0x7fa8280727b0 0x7fa828074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:50.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.749+0000 7fa83ea00640 1 -- 172.21.15.28:0/2675005146 --> [v2:172.21.15.103:6816/1745016180,v1:172.21.15.103:6818/1745016180] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa834009e50 con 0x7fa8280727b0 2024-04-26T21:06:50.750 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.749+0000 7fa83ea00640 1 -- 172.21.15.28:0/2675005146 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fa83408d480 con 0x7fa8481434c0 2024-04-26T21:06:50.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.749+0000 7fa846a00640 1 --2- 172.21.15.28:0/2675005146 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa82806c8f0 0x7fa82806edb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa838003eb0 tx=0x7fa838009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:50.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.750+0000 7fa847400640 1 --2- 172.21.15.28:0/2675005146 >> [v2:172.21.15.103:6816/1745016180,v1:172.21.15.103:6818/1745016180] conn(0x7fa8280727b0 0x7fa828074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:50.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.751+0000 7fa847400640 1 --2- 172.21.15.28:0/2675005146 >> [v2:172.21.15.103:6816/1745016180,v1:172.21.15.103:6818/1745016180] conn(0x7fa8280727b0 0x7fa828074bd0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:50.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.753+0000 7fa83ea00640 1 -- 172.21.15.28:0/2675005146 <== osd.6 v2:172.21.15.103:6816/1745016180 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa834009e50 con 0x7fa8280727b0 2024-04-26T21:06:50.768 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.768+0000 7fa847e00640 1 -- 172.21.15.28:0/2675005146 --> [v2:172.21.15.103:6816/1745016180,v1:172.21.15.103:6818/1745016180] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa84806a3d0 con 0x7fa8280727b0 2024-04-26T21:06:50.770 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.769+0000 7fa83ea00640 1 -- 172.21.15.28:0/2675005146 <== osd.6 v2:172.21.15.103:6816/1745016180 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa84806a3d0 con 0x7fa8280727b0 2024-04-26T21:06:50.770 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T21:06:50.770 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:06:50.771 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T21:06:50.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.769+0000 7fa847e00640 1 -- 172.21.15.28:0/2675005146 >> [v2:172.21.15.103:6816/1745016180,v1:172.21.15.103:6818/1745016180] conn(0x7fa8280727b0 msgr2=0x7fa828074bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:50.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.769+0000 7fa847e00640 1 --2- 172.21.15.28:0/2675005146 >> [v2:172.21.15.103:6816/1745016180,v1:172.21.15.103:6818/1745016180] conn(0x7fa8280727b0 0x7fa828074bd0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.770+0000 7fa847e00640 1 -- 172.21.15.28:0/2675005146 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa82806c8f0 msgr2=0x7fa82806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:50.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.770+0000 7fa847e00640 1 --2- 172.21.15.28:0/2675005146 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fa82806c8f0 0x7fa82806edb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa838003eb0 tx=0x7fa838009040 comp rx=0 tx=0).stop 2024-04-26T21:06:50.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.770+0000 7fa847e00640 1 -- 172.21.15.28:0/2675005146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8481434c0 msgr2=0x7fa848073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:50.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.770+0000 7fa847e00640 1 --2- 172.21.15.28:0/2675005146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa8481434c0 0x7fa848073440 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fa83400c400 tx=0x7fa83400c8c0 comp rx=0 tx=0).stop 2024-04-26T21:06:50.772 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.770+0000 7fa847400640 1 -- 172.21.15.28:0/2675005146 reap_dead start 2024-04-26T21:06:50.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.770+0000 7fa847e00640 1 -- 172.21.15.28:0/2675005146 shutdown_connections 2024-04-26T21:06:50.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.770+0000 7fa847e00640 1 -- 172.21.15.28:0/2675005146 >> 172.21.15.28:0/2675005146 conn(0x7fa8480b2820 msgr2=0x7fa8480d1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:50.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.770+0000 7fa847e00640 1 -- 172.21.15.28:0/2675005146 shutdown_connections 2024-04-26T21:06:50.773 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:50.770+0000 7fa847e00640 1 -- 172.21.15.28:0/2675005146 wait complete. 2024-04-26T21:06:50.792 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-26T21:06:51.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.007+0000 7fd73ec00640 1 Processor -- start 2024-04-26T21:06:51.008 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.007+0000 7fd73ec00640 1 -- start start 2024-04-26T21:06:51.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.008+0000 7fd73ec00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd738146980 0x7fd7380b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:51.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.008+0000 7fd73ec00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd7380740e0 0x7fd7380b6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:51.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.008+0000 7fd73ec00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7fd73815d5b0 0x7fd73815da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:51.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.008+0000 7fd73ec00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7fd73815e510 con 0x7fd738146980 2024-04-26T21:06:51.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.008+0000 7fd73ec00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7fd7380d0a30 con 0x7fd73815d5b0 2024-04-26T21:06:51.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.008+0000 7fd73ec00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fd73815df80 con 0x7fd7380740e0 2024-04-26T21:06:51.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.009+0000 7fd737e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd738146980 0x7fd7380b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:51.010 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.009+0000 7fd737e00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd738146980 0x7fd7380b6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54202/0 (socket says 172.21.15.28:54202) 2024-04-26T21:06:51.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.009+0000 7fd737e00640 1 -- 172.21.15.28:0/383701012 learned_addr learned my addr 172.21.15.28:0/383701012 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:51.011 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.009+0000 7fd737e00640 1 -- 172.21.15.28:0/383701012 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd7380740e0 msgr2=0x7fd7380b6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:51.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.009+0000 7fd72fe00640 1 --2- 172.21.15.28:0/383701012 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd7380740e0 0x7fd7380b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:51.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.009+0000 7fd737e00640 1 --2- 172.21.15.28:0/383701012 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd7380740e0 0x7fd7380b6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.009+0000 7fd73cc00640 1 --2- 172.21.15.28:0/383701012 >> v2:172.21.15.103:3300/0 conn(0x7fd73815d5b0 0x7fd73815da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:51.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.009+0000 7fd737e00640 1 -- 172.21.15.28:0/383701012 >> v2:172.21.15.103:3300/0 conn(0x7fd73815d5b0 msgr2=0x7fd73815da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:51.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.009+0000 7fd737e00640 1 --2- 172.21.15.28:0/383701012 >> v2:172.21.15.103:3300/0 conn(0x7fd73815d5b0 0x7fd73815da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.009+0000 7fd737e00640 1 -- 172.21.15.28:0/383701012 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73815e230 con 0x7fd738146980 2024-04-26T21:06:51.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.010+0000 7fd72fe00640 1 --2- 172.21.15.28:0/383701012 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd7380740e0 0x7fd7380b6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:51.012 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.011+0000 7fd737e00640 1 --2- 172.21.15.28:0/383701012 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd738146980 0x7fd7380b6100 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd738075f80 tx=0x7fd72802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf7575fc735d080 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:51.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.011+0000 7fd737400640 1 -- 172.21.15.28:0/383701012 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728037070 con 0x7fd738146980 2024-04-26T21:06:51.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.011+0000 7fd737400640 1 -- 172.21.15.28:0/383701012 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd728032440 con 0x7fd738146980 2024-04-26T21:06:51.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.011+0000 7fd737400640 1 -- 172.21.15.28:0/383701012 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd728032810 con 0x7fd738146980 2024-04-26T21:06:51.013 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.011+0000 7fd73ec00640 1 -- 172.21.15.28:0/383701012 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd738146980 msgr2=0x7fd7380b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:51.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.011+0000 7fd73ec00640 1 --2- 172.21.15.28:0/383701012 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd738146980 0x7fd7380b6100 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fd738075f80 tx=0x7fd72802a140 comp rx=0 tx=0).stop 2024-04-26T21:06:51.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.012+0000 7fd73ec00640 1 -- 172.21.15.28:0/383701012 shutdown_connections 2024-04-26T21:06:51.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.012+0000 7fd73ec00640 1 --2- 172.21.15.28:0/383701012 >> v2:172.21.15.103:3300/0 conn(0x7fd73815d5b0 0x7fd73815da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.012+0000 7fd73ec00640 1 --2- 172.21.15.28:0/383701012 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd7380740e0 0x7fd7380b6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.012+0000 7fd73ec00640 1 --2- 172.21.15.28:0/383701012 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd738146980 0x7fd7380b6100 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.012+0000 7fd73ec00640 1 -- 172.21.15.28:0/383701012 >> 172.21.15.28:0/383701012 conn(0x7fd73806e220 msgr2=0x7fd73806e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:51.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.012+0000 7fd73ec00640 1 -- 172.21.15.28:0/383701012 shutdown_connections 2024-04-26T21:06:51.014 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.012+0000 7fd73ec00640 1 -- 172.21.15.28:0/383701012 wait complete. 2024-04-26T21:06:51.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.013+0000 7fd73ec00640 1 Processor -- start 2024-04-26T21:06:51.015 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.014+0000 7fd73ec00640 1 -- start start 2024-04-26T21:06:51.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.014+0000 7fd73ec00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd7380740e0 0x7fd7381587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:51.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.015+0000 7fd73ec00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd738146980 0x7fd7381409b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:51.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.015+0000 7fd73ec00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd73815d5b0 0x7fd73813e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:51.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.015+0000 7fd73ec00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7fd738073bc0 con 0x7fd738146980 2024-04-26T21:06:51.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.015+0000 7fd73ec00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd7380738e0 con 0x7fd7380740e0 2024-04-26T21:06:51.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.015+0000 7fd73ec00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7fd738160bd0 con 0x7fd73815d5b0 2024-04-26T21:06:51.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.015+0000 7fd72fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd738146980 0x7fd7381409b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:51.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.015+0000 7fd72fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd738146980 0x7fd7381409b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54210/0 (socket says 172.21.15.28:54210) 2024-04-26T21:06:51.016 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.015+0000 7fd72fe00640 1 -- 172.21.15.28:0/44654135 learned_addr learned my addr 172.21.15.28:0/44654135 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:51.017 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.015+0000 7fd737e00640 1 --2- 172.21.15.28:0/44654135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd7380740e0 0x7fd7381587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:51.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.015+0000 7fd73cc00640 1 --2- 172.21.15.28:0/44654135 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd73815d5b0 0x7fd73813e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:51.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.015+0000 7fd72fe00640 1 -- 172.21.15.28:0/44654135 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd73815d5b0 msgr2=0x7fd73813e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:51.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.015+0000 7fd72fe00640 1 --2- 172.21.15.28:0/44654135 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd73815d5b0 0x7fd73813e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.015+0000 7fd72fe00640 1 -- 172.21.15.28:0/44654135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd7380740e0 msgr2=0x7fd7381587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:51.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.015+0000 7fd72fe00640 1 --2- 172.21.15.28:0/44654135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd7380740e0 0x7fd7381587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.015+0000 7fd72fe00640 1 -- 172.21.15.28:0/44654135 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7381413b0 con 0x7fd738146980 2024-04-26T21:06:51.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.015+0000 7fd737e00640 1 --2- 172.21.15.28:0/44654135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd7380740e0 0x7fd7381587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:51.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.016+0000 7fd72fe00640 1 --2- 172.21.15.28:0/44654135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd738146980 0x7fd7381409b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd72000c7f0 tx=0x7fd72000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:51.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.016+0000 7fd734c00640 1 -- 172.21.15.28:0/44654135 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd720015070 con 0x7fd738146980 2024-04-26T21:06:51.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.016+0000 7fd734c00640 1 -- 172.21.15.28:0/44654135 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd720007a30 con 0x7fd738146980 2024-04-26T21:06:51.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.016+0000 7fd734c00640 1 -- 172.21.15.28:0/44654135 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd720007e00 con 0x7fd738146980 2024-04-26T21:06:51.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.016+0000 7fd73ec00640 1 -- 172.21.15.28:0/44654135 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd738141690 con 0x7fd738146980 2024-04-26T21:06:51.018 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.016+0000 7fd73ec00640 1 -- 172.21.15.28:0/44654135 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd738141c50 con 0x7fd738146980 2024-04-26T21:06:51.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.018+0000 7fd73ec00640 1 -- 172.21.15.28:0/44654135 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd73806a400 con 0x7fd738146980 2024-04-26T21:06:51.019 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.018+0000 7fd734c00640 1 -- 172.21.15.28:0/44654135 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd720018070 con 0x7fd738146980 2024-04-26T21:06:51.020 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.019+0000 7fd734c00640 1 --2- 172.21.15.28:0/44654135 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd71406c8f0 0x7fd71406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:51.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.020+0000 7fd734c00640 1 -- 172.21.15.28:0/44654135 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fd72008b780 con 0x7fd738146980 2024-04-26T21:06:51.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.020+0000 7fd737e00640 1 --2- 172.21.15.28:0/44654135 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd71406c8f0 0x7fd71406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:51.021 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.021+0000 7fd737e00640 1 --2- 172.21.15.28:0/44654135 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd71406c8f0 0x7fd71406edb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd728034000 tx=0x7fd72802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:51.025 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.024+0000 7fd734c00640 1 -- 172.21.15.28:0/44654135 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd720059ea0 con 0x7fd738146980 2024-04-26T21:06:51.236 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.235+0000 7fd73ec00640 1 -- 172.21.15.28:0/44654135 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fd73806ff00 con 0x7fd71406c8f0 2024-04-26T21:06:51.237 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.236+0000 7fd734c00640 1 -- 172.21.15.28:0/44654135 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd73806ff00 con 0x7fd71406c8f0 2024-04-26T21:06:51.237 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 6 to deep-scrub 2024-04-26T21:06:51.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.238+0000 7fd73ec00640 1 -- 172.21.15.28:0/44654135 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd71406c8f0 msgr2=0x7fd71406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:51.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.238+0000 7fd73ec00640 1 --2- 172.21.15.28:0/44654135 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd71406c8f0 0x7fd71406edb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd728034000 tx=0x7fd72802e000 comp rx=0 tx=0).stop 2024-04-26T21:06:51.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.238+0000 7fd73ec00640 1 -- 172.21.15.28:0/44654135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd738146980 msgr2=0x7fd7381409b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:51.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.238+0000 7fd73ec00640 1 --2- 172.21.15.28:0/44654135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd738146980 0x7fd7381409b0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd72000c7f0 tx=0x7fd72000ccb0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.239+0000 7fd73ec00640 1 -- 172.21.15.28:0/44654135 shutdown_connections 2024-04-26T21:06:51.240 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.239+0000 7fd73ec00640 1 --2- 172.21.15.28:0/44654135 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7fd71406c8f0 0x7fd71406edb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.239+0000 7fd73ec00640 1 --2- 172.21.15.28:0/44654135 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7fd73815d5b0 0x7fd73813e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.239+0000 7fd73ec00640 1 --2- 172.21.15.28:0/44654135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd738146980 0x7fd7381409b0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.239+0000 7fd73ec00640 1 --2- 172.21.15.28:0/44654135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd7380740e0 0x7fd7381587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.239+0000 7fd73ec00640 1 -- 172.21.15.28:0/44654135 >> 172.21.15.28:0/44654135 conn(0x7fd73806e220 msgr2=0x7fd73815b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:51.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.239+0000 7fd73ec00640 1 -- 172.21.15.28:0/44654135 shutdown_connections 2024-04-26T21:06:51.241 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.239+0000 7fd73ec00640 1 -- 172.21.15.28:0/44654135 wait complete. 2024-04-26T21:06:51.260 INFO:tasks.ceph:Scrubbing osd.7 2024-04-26T21:06:51.261 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-26T21:06:51.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.477+0000 7facf6600640 1 Processor -- start 2024-04-26T21:06:51.479 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.478+0000 7facf6600640 1 -- start start 2024-04-26T21:06:51.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.479+0000 7facf6600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7facf00b4140 0x7facf00ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:51.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.479+0000 7facf6600640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7facf00bac10 0x7facf00cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:51.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.479+0000 7facf6600640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facf00d2fe0 con 0x7facf015f0f0 2024-04-26T21:06:51.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.479+0000 7facf6600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7facf00d3bf0 con 0x7facf00b4140 2024-04-26T21:06:51.481 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.479+0000 7facf6600640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7facf00d32c0 con 0x7facf00bac10 2024-04-26T21:06:51.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.480+0000 7facefe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7facf00b4140 0x7facf00ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:51.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.480+0000 7facefe00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7facf00b4140 0x7facf00ba6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54224/0 (socket says 172.21.15.28:54224) 2024-04-26T21:06:51.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.480+0000 7facefe00640 1 -- 172.21.15.28:0/2562222222 learned_addr learned my addr 172.21.15.28:0/2562222222 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:51.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.480+0000 7facf5c00640 1 --2- 172.21.15.28:0/2562222222 >> v2:172.21.15.103:3300/0 conn(0x7facf00bac10 0x7facf00cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:51.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.480+0000 7facefe00640 1 -- 172.21.15.28:0/2562222222 >> v2:172.21.15.103:3300/0 conn(0x7facf00bac10 msgr2=0x7facf00cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:51.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.480+0000 7facefe00640 1 --2- 172.21.15.28:0/2562222222 >> v2:172.21.15.103:3300/0 conn(0x7facf00bac10 0x7facf00cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.480+0000 7facefe00640 1 -- 172.21.15.28:0/2562222222 >> v1:172.21.15.103:6789/0 conn(0x7facf015f0f0 legacy=0x7facf00b7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:51.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.480+0000 7facefe00640 1 -- 172.21.15.28:0/2562222222 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf00d35a0 con 0x7facf00b4140 2024-04-26T21:06:51.482 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.480+0000 7facf5c00640 1 --2- 172.21.15.28:0/2562222222 >> v2:172.21.15.103:3300/0 conn(0x7facf00bac10 0x7facf00cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:51.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.481+0000 7facefe00640 1 --2- 172.21.15.28:0/2562222222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7facf00b4140 0x7facf00ba6d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7facf00d2420 tx=0x7facdc02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7268942298a786 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:51.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.482+0000 7facef400640 1 -- 172.21.15.28:0/2562222222 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc037070 con 0x7facf00b4140 2024-04-26T21:06:51.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.482+0000 7facef400640 1 -- 172.21.15.28:0/2562222222 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facdc032440 con 0x7facf00b4140 2024-04-26T21:06:51.483 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.482+0000 7facef400640 1 -- 172.21.15.28:0/2562222222 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc032810 con 0x7facf00b4140 2024-04-26T21:06:51.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.482+0000 7facf6600640 1 -- 172.21.15.28:0/2562222222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7facf00b4140 msgr2=0x7facf00ba6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:51.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.482+0000 7facf6600640 1 --2- 172.21.15.28:0/2562222222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7facf00b4140 0x7facf00ba6d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7facf00d2420 tx=0x7facdc02a2c0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.483+0000 7facf6600640 1 -- 172.21.15.28:0/2562222222 shutdown_connections 2024-04-26T21:06:51.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.483+0000 7facf6600640 1 --2- 172.21.15.28:0/2562222222 >> v2:172.21.15.103:3300/0 conn(0x7facf00bac10 0x7facf00cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.483+0000 7facf6600640 1 --2- 172.21.15.28:0/2562222222 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7facf00b4140 0x7facf00ba6d0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.483+0000 7facf6600640 1 -- 172.21.15.28:0/2562222222 >> 172.21.15.28:0/2562222222 conn(0x7facf006c980 msgr2=0x7facf0072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:51.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.483+0000 7facf6600640 1 -- 172.21.15.28:0/2562222222 shutdown_connections 2024-04-26T21:06:51.484 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.483+0000 7facf6600640 1 -- 172.21.15.28:0/2562222222 wait complete. 2024-04-26T21:06:51.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.484+0000 7facf6600640 1 Processor -- start 2024-04-26T21:06:51.485 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.485+0000 7facf6600640 1 -- start start 2024-04-26T21:06:51.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.486+0000 7facf6600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7facf00b4140 0x7facf0153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:51.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.486+0000 7facf6600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf00bac10 0x7facf01f8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:51.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.486+0000 7facf6600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7facf015f0f0 0x7facf00cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:51.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.486+0000 7facf6600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7facf00d01c0 con 0x7facf00b4140 2024-04-26T21:06:51.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.486+0000 7facf6600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7facf00d0040 con 0x7facf00bac10 2024-04-26T21:06:51.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.486+0000 7facf6600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7facf00d0340 con 0x7facf015f0f0 2024-04-26T21:06:51.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.486+0000 7facf5200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7facf00b4140 0x7facf0153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:51.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.486+0000 7facf5200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7facf00b4140 0x7facf0153c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54230/0 (socket says 172.21.15.28:54230) 2024-04-26T21:06:51.487 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.486+0000 7facf5200640 1 -- 172.21.15.28:0/2840622630 learned_addr learned my addr 172.21.15.28:0/2840622630 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:51.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.486+0000 7facf5c00640 1 --2- 172.21.15.28:0/2840622630 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7facf015f0f0 0x7facf00cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:51.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.486+0000 7facf5200640 1 -- 172.21.15.28:0/2840622630 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7facf015f0f0 msgr2=0x7facf00cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:51.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.486+0000 7facf5200640 1 --2- 172.21.15.28:0/2840622630 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7facf015f0f0 0x7facf00cbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.486+0000 7facf5200640 1 -- 172.21.15.28:0/2840622630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf00bac10 msgr2=0x7facf01f8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:51.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.486+0000 7facf5200640 1 --2- 172.21.15.28:0/2840622630 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf00bac10 0x7facf01f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.486+0000 7facf5200640 1 -- 172.21.15.28:0/2840622630 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf00ced00 con 0x7facf00b4140 2024-04-26T21:06:51.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.487+0000 7facf5200640 1 --2- 172.21.15.28:0/2840622630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7facf00b4140 0x7facf0153c80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7face400c430 tx=0x7face400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:51.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.487+0000 7facecc00640 1 -- 172.21.15.28:0/2840622630 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4011790 con 0x7facf00b4140 2024-04-26T21:06:51.489 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.487+0000 7facecc00640 1 -- 172.21.15.28:0/2840622630 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face4017070 con 0x7facf00b4140 2024-04-26T21:06:51.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.487+0000 7facecc00640 1 -- 172.21.15.28:0/2840622630 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face4011bb0 con 0x7facf00b4140 2024-04-26T21:06:51.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.487+0000 7facf6600640 1 -- 172.21.15.28:0/2840622630 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf01541c0 con 0x7facf00b4140 2024-04-26T21:06:51.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.487+0000 7facf6600640 1 -- 172.21.15.28:0/2840622630 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf01f9190 con 0x7facf00b4140 2024-04-26T21:06:51.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.488+0000 7facf6600640 1 -- 172.21.15.28:0/2840622630 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7facf006a3d0 con 0x7facf00b4140 2024-04-26T21:06:51.490 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.489+0000 7facecc00640 1 -- 172.21.15.28:0/2840622630 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7face401c460 con 0x7facf00b4140 2024-04-26T21:06:51.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.490+0000 7facecc00640 1 --2- 172.21.15.28:0/2840622630 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7facd006c8f0 0x7facd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:51.491 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.490+0000 7facecc00640 1 -- 172.21.15.28:0/2840622630 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7face408b780 con 0x7facf00b4140 2024-04-26T21:06:51.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.490+0000 7facecc00640 1 --2- 172.21.15.28:0/2840622630 >> [v2:172.21.15.103:6808/583391321,v1:172.21.15.103:6809/583391321] conn(0x7facd00727b0 0x7facd0074bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:51.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.490+0000 7facecc00640 1 -- 172.21.15.28:0/2840622630 --> [v2:172.21.15.103:6808/583391321,v1:172.21.15.103:6809/583391321] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7facd0075280 con 0x7facd00727b0 2024-04-26T21:06:51.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.490+0000 7facecc00640 1 -- 172.21.15.28:0/2840622630 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7face4092050 con 0x7facf00b4140 2024-04-26T21:06:51.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.490+0000 7facefe00640 1 --2- 172.21.15.28:0/2840622630 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7facd006c8f0 0x7facd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:51.492 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.491+0000 7facefe00640 1 --2- 172.21.15.28:0/2840622630 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7facd006c8f0 0x7facd006edb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7facdc034000 tx=0x7facdc02c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:51.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.491+0000 7facf5c00640 1 --2- 172.21.15.28:0/2840622630 >> [v2:172.21.15.103:6808/583391321,v1:172.21.15.103:6809/583391321] conn(0x7facd00727b0 0x7facd0074bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:51.493 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.492+0000 7facf5c00640 1 --2- 172.21.15.28:0/2840622630 >> [v2:172.21.15.103:6808/583391321,v1:172.21.15.103:6809/583391321] conn(0x7facd00727b0 0x7facd0074bd0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:51.495 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.494+0000 7facecc00640 1 -- 172.21.15.28:0/2840622630 <== osd.7 v2:172.21.15.103:6808/583391321 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7facd0075280 con 0x7facd00727b0 2024-04-26T21:06:51.510 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.509+0000 7facf6600640 1 -- 172.21.15.28:0/2840622630 --> [v2:172.21.15.103:6808/583391321,v1:172.21.15.103:6809/583391321] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7facf00d5c80 con 0x7facd00727b0 2024-04-26T21:06:51.511 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.510+0000 7facecc00640 1 -- 172.21.15.28:0/2840622630 <== osd.7 v2:172.21.15.103:6808/583391321 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7facf00d5c80 con 0x7facd00727b0 2024-04-26T21:06:51.512 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2024-04-26T21:06:51.512 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T21:06:51.512 INFO:teuthology.orchestra.run.smithi028.stdout:} 2024-04-26T21:06:51.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.511+0000 7facf6600640 1 -- 172.21.15.28:0/2840622630 >> [v2:172.21.15.103:6808/583391321,v1:172.21.15.103:6809/583391321] conn(0x7facd00727b0 msgr2=0x7facd0074bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:51.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.511+0000 7facf6600640 1 --2- 172.21.15.28:0/2840622630 >> [v2:172.21.15.103:6808/583391321,v1:172.21.15.103:6809/583391321] conn(0x7facd00727b0 0x7facd0074bd0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.511+0000 7facf6600640 1 -- 172.21.15.28:0/2840622630 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7facd006c8f0 msgr2=0x7facd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:51.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.511+0000 7facf6600640 1 --2- 172.21.15.28:0/2840622630 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7facd006c8f0 0x7facd006edb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7facdc034000 tx=0x7facdc02c000 comp rx=0 tx=0).stop 2024-04-26T21:06:51.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.511+0000 7facf6600640 1 -- 172.21.15.28:0/2840622630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7facf00b4140 msgr2=0x7facf0153c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:51.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.511+0000 7facf6600640 1 --2- 172.21.15.28:0/2840622630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7facf00b4140 0x7facf0153c80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7face400c430 tx=0x7face400c8f0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.511+0000 7facf5c00640 1 -- 172.21.15.28:0/2840622630 reap_dead start 2024-04-26T21:06:51.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.511+0000 7facf6600640 1 -- 172.21.15.28:0/2840622630 shutdown_connections 2024-04-26T21:06:51.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.511+0000 7facf6600640 1 -- 172.21.15.28:0/2840622630 >> 172.21.15.28:0/2840622630 conn(0x7facf006c980 msgr2=0x7facf00b7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:51.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.511+0000 7facf6600640 1 -- 172.21.15.28:0/2840622630 shutdown_connections 2024-04-26T21:06:51.513 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.511+0000 7facf6600640 1 -- 172.21.15.28:0/2840622630 wait complete. 2024-04-26T21:06:51.532 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-26T21:06:51.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.747+0000 7f37bdc00640 1 Processor -- start 2024-04-26T21:06:51.749 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.748+0000 7f37bdc00640 1 -- start start 2024-04-26T21:06:51.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.749+0000 7f37bdc00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f37b81612d0 0x7f37b80b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:51.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.749+0000 7f37bdc00640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37b80b4740 con 0x7f37b80b61c0 2024-04-26T21:06:51.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.749+0000 7f37bdc00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37b80b4a20 con 0x7f37b8163c00 2024-04-26T21:06:51.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.749+0000 7f37bdc00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f37b80b4d00 con 0x7f37b81612d0 2024-04-26T21:06:51.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.750+0000 7f37b6a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f37b81612d0 0x7f37b80b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:51.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.750+0000 7f37b6a00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f37b81612d0 0x7f37b80b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54232/0 (socket says 172.21.15.28:54232) 2024-04-26T21:06:51.751 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.750+0000 7f37b6a00640 1 -- 172.21.15.28:0/1452957522 learned_addr learned my addr 172.21.15.28:0/1452957522 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:51.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.750+0000 7f37b6a00640 1 -- 172.21.15.28:0/1452957522 >> v1:172.21.15.103:6789/0 conn(0x7f37b8163c00 legacy=0x7f37b80b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:51.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.750+0000 7f37b6a00640 1 -- 172.21.15.28:0/1452957522 >> v1:172.21.15.28:6789/0 conn(0x7f37b80b61c0 legacy=0x7f37b8165330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:51.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.750+0000 7f37b6a00640 1 -- 172.21.15.28:0/1452957522 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b8164e10 con 0x7f37b81612d0 2024-04-26T21:06:51.752 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.751+0000 7f37b6a00640 1 --2- 172.21.15.28:0/1452957522 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37b81612d0 0x7f37b80b3df0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009870 tx=0x7f37a802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509292ac4b6e049c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:51.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.752+0000 7f37b6000640 1 -- 172.21.15.28:0/1452957522 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8037070 con 0x7f37b81612d0 2024-04-26T21:06:51.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.752+0000 7f37b6000640 1 -- 172.21.15.28:0/1452957522 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37a8032440 con 0x7f37b81612d0 2024-04-26T21:06:51.753 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.752+0000 7f37b6000640 1 -- 172.21.15.28:0/1452957522 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8032810 con 0x7f37b81612d0 2024-04-26T21:06:51.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.752+0000 7f37bdc00640 1 -- 172.21.15.28:0/1452957522 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37b81612d0 msgr2=0x7f37b80b3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:51.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.752+0000 7f37bdc00640 1 --2- 172.21.15.28:0/1452957522 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37b81612d0 0x7f37b80b3df0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009870 tx=0x7f37a802a2a0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.753+0000 7f37bdc00640 1 -- 172.21.15.28:0/1452957522 shutdown_connections 2024-04-26T21:06:51.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.753+0000 7f37bdc00640 1 --2- 172.21.15.28:0/1452957522 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37b81612d0 0x7f37b80b3df0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.753+0000 7f37bdc00640 1 -- 172.21.15.28:0/1452957522 >> 172.21.15.28:0/1452957522 conn(0x7f37b806e220 msgr2=0x7f37b806e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:51.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.753+0000 7f37bdc00640 1 -- 172.21.15.28:0/1452957522 shutdown_connections 2024-04-26T21:06:51.754 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.753+0000 7f37bdc00640 1 -- 172.21.15.28:0/1452957522 wait complete. 2024-04-26T21:06:51.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.754+0000 7f37bdc00640 1 Processor -- start 2024-04-26T21:06:51.755 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.754+0000 7f37bdc00640 1 -- start start 2024-04-26T21:06:51.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.755+0000 7f37bdc00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37b80b61c0 0x7f37b81ede40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:51.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.755+0000 7f37bdc00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37b81612d0 0x7f37b8149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:51.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.755+0000 7f37bdc00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37b8163c00 0x7f37b81eeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:51.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.755+0000 7f37bdc00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8165bc0 con 0x7f37b8163c00 2024-04-26T21:06:51.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.755+0000 7f37bdc00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f37b8165a40 con 0x7f37b80b61c0 2024-04-26T21:06:51.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.755+0000 7f37bdc00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f37b8165d40 con 0x7f37b81612d0 2024-04-26T21:06:51.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.756+0000 7f37b7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37b8163c00 0x7f37b81eeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:51.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.756+0000 7f37b7400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37b8163c00 0x7f37b81eeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54234/0 (socket says 172.21.15.28:54234) 2024-04-26T21:06:51.757 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.756+0000 7f37b7400640 1 -- 172.21.15.28:0/552482749 learned_addr learned my addr 172.21.15.28:0/552482749 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:51.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.756+0000 7f37b6a00640 1 --2- 172.21.15.28:0/552482749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37b80b61c0 0x7f37b81ede40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:51.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.756+0000 7f37afe00640 1 --2- 172.21.15.28:0/552482749 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37b81612d0 0x7f37b8149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:51.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.756+0000 7f37b7400640 1 -- 172.21.15.28:0/552482749 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37b81612d0 msgr2=0x7f37b8149e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:51.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.756+0000 7f37b7400640 1 --2- 172.21.15.28:0/552482749 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37b81612d0 0x7f37b8149e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.756+0000 7f37b7400640 1 -- 172.21.15.28:0/552482749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37b80b61c0 msgr2=0x7f37b81ede40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:51.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.756+0000 7f37b7400640 1 --2- 172.21.15.28:0/552482749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37b80b61c0 0x7f37b81ede40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.756+0000 7f37b7400640 1 -- 172.21.15.28:0/552482749 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b814a380 con 0x7f37b8163c00 2024-04-26T21:06:51.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.756+0000 7f37afe00640 1 --2- 172.21.15.28:0/552482749 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37b81612d0 0x7f37b8149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:51.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.756+0000 7f37b6a00640 1 --2- 172.21.15.28:0/552482749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37b80b61c0 0x7f37b81ede40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:06:51.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.757+0000 7f37b7400640 1 --2- 172.21.15.28:0/552482749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37b8163c00 0x7f37b81eeae0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f37b000c730 tx=0x7f37b000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:51.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.757+0000 7f37aea00640 1 -- 172.21.15.28:0/552482749 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0016020 con 0x7f37b8163c00 2024-04-26T21:06:51.759 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.757+0000 7f37aea00640 1 -- 172.21.15.28:0/552482749 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b0006d50 con 0x7f37b8163c00 2024-04-26T21:06:51.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.757+0000 7f37aea00640 1 -- 172.21.15.28:0/552482749 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0003bb0 con 0x7f37b8163c00 2024-04-26T21:06:51.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.757+0000 7f37bdc00640 1 -- 172.21.15.28:0/552482749 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b81576b0 con 0x7f37b8163c00 2024-04-26T21:06:51.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.757+0000 7f37bdc00640 1 -- 172.21.15.28:0/552482749 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b81ed080 con 0x7f37b8163c00 2024-04-26T21:06:51.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.758+0000 7f37bdc00640 1 -- 172.21.15.28:0/552482749 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37b806a400 con 0x7f37b8163c00 2024-04-26T21:06:51.760 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.760+0000 7f37aea00640 1 -- 172.21.15.28:0/552482749 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37b000a7f0 con 0x7f37b8163c00 2024-04-26T21:06:51.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.761+0000 7f37aea00640 1 --2- 172.21.15.28:0/552482749 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f379406c8f0 0x7f379406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:51.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.761+0000 7f37aea00640 1 -- 172.21.15.28:0/552482749 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f37b008b510 con 0x7f37b8163c00 2024-04-26T21:06:51.762 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.761+0000 7f37b6a00640 1 --2- 172.21.15.28:0/552482749 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f379406c8f0 0x7f379406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:51.763 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.762+0000 7f37b6a00640 1 --2- 172.21.15.28:0/552482749 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f379406c8f0 0x7f379406edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f37a8034000 tx=0x7f37a802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:51.767 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.766+0000 7f37aea00640 1 -- 172.21.15.28:0/552482749 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f37b0020020 con 0x7f37b8163c00 2024-04-26T21:06:51.971 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.970+0000 7f37bdc00640 1 -- 172.21.15.28:0/552482749 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f37b80b0740 con 0x7f379406c8f0 2024-04-26T21:06:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.971+0000 7f37aea00640 1 -- 172.21.15.28:0/552482749 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f37b80b0740 con 0x7f379406c8f0 2024-04-26T21:06:51.972 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 7 to deep-scrub 2024-04-26T21:06:51.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.973+0000 7f37bdc00640 1 -- 172.21.15.28:0/552482749 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f379406c8f0 msgr2=0x7f379406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:51.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.973+0000 7f37bdc00640 1 --2- 172.21.15.28:0/552482749 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f379406c8f0 0x7f379406edb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f37a8034000 tx=0x7f37a802e000 comp rx=0 tx=0).stop 2024-04-26T21:06:51.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.973+0000 7f37bdc00640 1 -- 172.21.15.28:0/552482749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37b8163c00 msgr2=0x7f37b81eeae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:51.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.973+0000 7f37bdc00640 1 --2- 172.21.15.28:0/552482749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37b8163c00 0x7f37b81eeae0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f37b000c730 tx=0x7f37b000cbf0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.974+0000 7f37bdc00640 1 -- 172.21.15.28:0/552482749 shutdown_connections 2024-04-26T21:06:51.975 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.974+0000 7f37bdc00640 1 --2- 172.21.15.28:0/552482749 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f379406c8f0 0x7f379406edb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.974+0000 7f37bdc00640 1 --2- 172.21.15.28:0/552482749 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f37b8163c00 0x7f37b81eeae0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.974+0000 7f37bdc00640 1 --2- 172.21.15.28:0/552482749 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f37b81612d0 0x7f37b8149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.974+0000 7f37bdc00640 1 --2- 172.21.15.28:0/552482749 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f37b80b61c0 0x7f37b81ede40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.974+0000 7f37bdc00640 1 -- 172.21.15.28:0/552482749 >> 172.21.15.28:0/552482749 conn(0x7f37b806e220 msgr2=0x7f37b80b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.974+0000 7f37bdc00640 1 -- 172.21.15.28:0/552482749 shutdown_connections 2024-04-26T21:06:51.976 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:51.974+0000 7f37bdc00640 1 -- 172.21.15.28:0/552482749 wait complete. 2024-04-26T21:06:51.994 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:06:52.208 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.207+0000 7f9b16600640 1 Processor -- start 2024-04-26T21:06:52.209 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.207+0000 7f9b16600640 1 -- start start 2024-04-26T21:06:52.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.209+0000 7f9b16600640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9b10146060 0x7f9b10071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:52.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.209+0000 7f9b16600640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f9b101610c0 0x7f9b100cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:52.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.209+0000 7f9b16600640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b10146440 con 0x7f9b10146670 2024-04-26T21:06:52.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.209+0000 7f9b16600640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f9b1006f510 con 0x7f9b10146060 2024-04-26T21:06:52.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.209+0000 7f9b16600640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f9b1006f010 con 0x7f9b101610c0 2024-04-26T21:06:52.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.210+0000 7f9b15200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9b10146060 0x7f9b10071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:52.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.210+0000 7f9b15200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9b10146060 0x7f9b10071e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54244/0 (socket says 172.21.15.28:54244) 2024-04-26T21:06:52.211 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.210+0000 7f9b15200640 1 -- 172.21.15.28:0/350449834 learned_addr learned my addr 172.21.15.28:0/350449834 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:52.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.210+0000 7f9b15200640 1 -- 172.21.15.28:0/350449834 >> v2:172.21.15.103:3300/0 conn(0x7f9b101610c0 msgr2=0x7f9b100cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:52.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.210+0000 7f9b15200640 1 --2- 172.21.15.28:0/350449834 >> v2:172.21.15.103:3300/0 conn(0x7f9b101610c0 0x7f9b100cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:52.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.210+0000 7f9b15200640 1 -- 172.21.15.28:0/350449834 >> v1:172.21.15.103:6789/0 conn(0x7f9b10146670 legacy=0x7f9b101609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:06:52.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.210+0000 7f9b15200640 1 -- 172.21.15.28:0/350449834 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b101614d0 con 0x7f9b10146060 2024-04-26T21:06:52.212 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.211+0000 7f9b15200640 1 --2- 172.21.15.28:0/350449834 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b10146060 0x7f9b10071e20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9b10072360 tx=0x7f9b0402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42a6b7b49460a32 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:52.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.212+0000 7f9b0f400640 1 -- 172.21.15.28:0/350449834 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b04037070 con 0x7f9b10146060 2024-04-26T21:06:52.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.212+0000 7f9b0f400640 1 -- 172.21.15.28:0/350449834 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b04032440 con 0x7f9b10146060 2024-04-26T21:06:52.213 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.212+0000 7f9b0f400640 1 -- 172.21.15.28:0/350449834 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b04032810 con 0x7f9b10146060 2024-04-26T21:06:52.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.212+0000 7f9b16600640 1 -- 172.21.15.28:0/350449834 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b10146060 msgr2=0x7f9b10071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:52.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.212+0000 7f9b16600640 1 --2- 172.21.15.28:0/350449834 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b10146060 0x7f9b10071e20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9b10072360 tx=0x7f9b0402a1a0 comp rx=0 tx=0).stop 2024-04-26T21:06:52.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.212+0000 7f9b16600640 1 -- 172.21.15.28:0/350449834 shutdown_connections 2024-04-26T21:06:52.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.212+0000 7f9b16600640 1 --2- 172.21.15.28:0/350449834 >> v2:172.21.15.103:3300/0 conn(0x7f9b101610c0 0x7f9b100cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:52.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.212+0000 7f9b16600640 1 --2- 172.21.15.28:0/350449834 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b10146060 0x7f9b10071e20 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:52.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.212+0000 7f9b16600640 1 -- 172.21.15.28:0/350449834 >> 172.21.15.28:0/350449834 conn(0x7f9b100d0a50 msgr2=0x7f9b100d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:52.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.213+0000 7f9b16600640 1 -- 172.21.15.28:0/350449834 shutdown_connections 2024-04-26T21:06:52.214 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.213+0000 7f9b16600640 1 -- 172.21.15.28:0/350449834 wait complete. 2024-04-26T21:06:52.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.215+0000 7f9b16600640 1 Processor -- start 2024-04-26T21:06:52.216 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.215+0000 7f9b16600640 1 -- start start 2024-04-26T21:06:52.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.216+0000 7f9b16600640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b10146060 0x7f9b1015b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:52.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.216+0000 7f9b16600640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b10146670 0x7f9b100c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:52.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.216+0000 7f9b16600640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b101610c0 0x7f9b100c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:52.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.216+0000 7f9b16600640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1006ff60 con 0x7f9b10146060 2024-04-26T21:06:52.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.216+0000 7f9b16600640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1006fc80 con 0x7f9b101610c0 2024-04-26T21:06:52.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.216+0000 7f9b16600640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f9b100cb5e0 con 0x7f9b10146670 2024-04-26T21:06:52.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.217+0000 7f9b15200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b10146060 0x7f9b1015b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:52.219 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.217+0000 7f9b15200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b10146060 0x7f9b1015b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:54250/0 (socket says 172.21.15.28:54250) 2024-04-26T21:06:52.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.217+0000 7f9b15200640 1 -- 172.21.15.28:0/250886560 learned_addr learned my addr 172.21.15.28:0/250886560 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:06:52.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.217+0000 7f9b0fe00640 1 --2- 172.21.15.28:0/250886560 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b10146670 0x7f9b100c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:52.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.217+0000 7f9b15c00640 1 --2- 172.21.15.28:0/250886560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b101610c0 0x7f9b100c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:52.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.217+0000 7f9b15200640 1 -- 172.21.15.28:0/250886560 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b10146670 msgr2=0x7f9b100c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:52.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.217+0000 7f9b15200640 1 --2- 172.21.15.28:0/250886560 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b10146670 0x7f9b100c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:52.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.217+0000 7f9b15200640 1 -- 172.21.15.28:0/250886560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b101610c0 msgr2=0x7f9b100c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:52.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.217+0000 7f9b15200640 1 --2- 172.21.15.28:0/250886560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b101610c0 0x7f9b100c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:52.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.217+0000 7f9b15200640 1 -- 172.21.15.28:0/250886560 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1015bc20 con 0x7f9b10146060 2024-04-26T21:06:52.220 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.218+0000 7f9b15200640 1 --2- 172.21.15.28:0/250886560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b10146060 0x7f9b1015b6e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9b0402a1e0 tx=0x7f9b040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:52.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.218+0000 7f9b0cc00640 1 -- 172.21.15.28:0/250886560 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b04041070 con 0x7f9b10146060 2024-04-26T21:06:52.221 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.218+0000 7f9b0cc00640 1 -- 172.21.15.28:0/250886560 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b0403c870 con 0x7f9b10146060 2024-04-26T21:06:52.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.218+0000 7f9b0cc00640 1 -- 172.21.15.28:0/250886560 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b04037040 con 0x7f9b10146060 2024-04-26T21:06:52.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.218+0000 7f9b16600640 1 -- 172.21.15.28:0/250886560 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b100c9b00 con 0x7f9b10146060 2024-04-26T21:06:52.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.218+0000 7f9b16600640 1 -- 172.21.15.28:0/250886560 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b100ca0c0 con 0x7f9b10146060 2024-04-26T21:06:52.222 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.220+0000 7f9b16600640 1 -- 172.21.15.28:0/250886560 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b100ca780 con 0x7f9b10146060 2024-04-26T21:06:52.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.221+0000 7f9b0cc00640 1 -- 172.21.15.28:0/250886560 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b04044070 con 0x7f9b10146060 2024-04-26T21:06:52.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.222+0000 7f9b0cc00640 1 --2- 172.21.15.28:0/250886560 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9af006c8f0 0x7f9af006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:06:52.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.222+0000 7f9b0cc00640 1 -- 172.21.15.28:0/250886560 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f9b040ac3d0 con 0x7f9b10146060 2024-04-26T21:06:52.223 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.222+0000 7f9b0fe00640 1 --2- 172.21.15.28:0/250886560 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9af006c8f0 0x7f9af006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:06:52.224 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.223+0000 7f9b0fe00640 1 --2- 172.21.15.28:0/250886560 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9af006c8f0 0x7f9af006edb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9b101456b0 tx=0x7f9afc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:06:52.228 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.227+0000 7f9b0cc00640 1 -- 172.21.15.28:0/250886560 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9b100ca780 con 0x7f9b10146060 2024-04-26T21:06:52.429 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.428+0000 7f9b16600640 1 -- 172.21.15.28:0/250886560 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b1015c310 con 0x7f9af006c8f0 2024-04-26T21:06:52.431 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.430+0000 7f9b0cc00640 1 -- 172.21.15.28:0/250886560 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13387 (secure 0 0 0) 0x7f9b1015c310 con 0x7f9af006c8f0 2024-04-26T21:06:52.432 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T21:06:52.432 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T21:06:52.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.433+0000 7f9b16600640 1 -- 172.21.15.28:0/250886560 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9af006c8f0 msgr2=0x7f9af006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:52.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.433+0000 7f9b16600640 1 --2- 172.21.15.28:0/250886560 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9af006c8f0 0x7f9af006edb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9b101456b0 tx=0x7f9afc009040 comp rx=0 tx=0).stop 2024-04-26T21:06:52.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.433+0000 7f9b16600640 1 -- 172.21.15.28:0/250886560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b10146060 msgr2=0x7f9b1015b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:06:52.434 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.433+0000 7f9b16600640 1 --2- 172.21.15.28:0/250886560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b10146060 0x7f9b1015b6e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9b0402a1e0 tx=0x7f9b040046c0 comp rx=0 tx=0).stop 2024-04-26T21:06:52.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.433+0000 7f9b16600640 1 -- 172.21.15.28:0/250886560 shutdown_connections 2024-04-26T21:06:52.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.433+0000 7f9b16600640 1 --2- 172.21.15.28:0/250886560 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f9af006c8f0 0x7f9af006edb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:52.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.433+0000 7f9b16600640 1 --2- 172.21.15.28:0/250886560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b101610c0 0x7f9b100c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:52.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.433+0000 7f9b16600640 1 --2- 172.21.15.28:0/250886560 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f9b10146670 0x7f9b100c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:52.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.433+0000 7f9b16600640 1 --2- 172.21.15.28:0/250886560 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b10146060 0x7f9b1015b6e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:06:52.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.433+0000 7f9b16600640 1 -- 172.21.15.28:0/250886560 >> 172.21.15.28:0/250886560 conn(0x7f9b100d0a50 msgr2=0x7f9b100b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:06:52.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.433+0000 7f9b16600640 1 -- 172.21.15.28:0/250886560 shutdown_connections 2024-04-26T21:06:52.435 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:06:52.433+0000 7f9b16600640 1 -- 172.21.15.28:0/250886560 wait complete. 2024-04-26T21:06:52.454 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-04-26T21:06:50.913222+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":255288,"kb_used_data":5832,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754719432,"statfs":{"total":773094113280,"available":772832698368,"internally_reserved":0,"allocated":5971968,"data_stored":3135024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10739774,"num_objects":-969,"num_object_clones":0,"num_object_copies":-1938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-990,"num_write_kb":-11231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-36,"num_bytes_recovered":-103097,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002264"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":230,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:06:41.707148+0000","last_change":"2024-04-26T20:56:46.848603+0000","last_active":"2024-04-26T21:06:41.707148+0000","last_peered":"2024-04-26T21:06:41.707148+0000","last_clean":"2024-04-26T21:06:41.707148+0000","last_became_active":"2024-04-26T20:56:46.846091+0000","last_became_peered":"2024-04-26T20:56:46.846091+0000","last_unstale":"2024-04-26T21:06:41.707148+0000","last_undegraded":"2024-04-26T21:06:41.707148+0000","last_fullsized":"2024-04-26T21:06:41.707148+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T20:56:45.594041+0000","last_clean_scrub_stamp":"2024-04-26T20:56:45.594041+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:00.232387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574972,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1191936,"data_stored":834654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31764,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":598016,"data_stored":244286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31764,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":598016,"data_stored":244286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574971,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31764,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":598016,"data_stored":244286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574972,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31768,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":598016,"data_stored":244286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574972,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31764,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":598016,"data_stored":244286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574972,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31768,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":598016,"data_stored":244286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574971,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1191936,"data_stored":834654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:06:52.458 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-26T20:56:45.594041+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=20, tm_min=56, tm_sec=45, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=21, tm_min=6, tm_sec=45, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T21:06:52.458 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-26T21:07:12.458 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T21:07:12.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.676+0000 7f0735800640 1 Processor -- start 2024-04-26T21:07:12.678 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.677+0000 7f0735800640 1 -- start start 2024-04-26T21:07:12.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.679+0000 7f0735800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0730146670 0x7f0730071e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:12.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.679+0000 7f0735800640 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0730161530 con 0x7f0730146060 2024-04-26T21:07:12.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.679+0000 7f0735800640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0730146440 con 0x7f07301610c0 2024-04-26T21:07:12.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.679+0000 7f0735800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f073006f510 con 0x7f0730146670 2024-04-26T21:07:12.683 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.679+0000 7f072fe00640 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f0730146060 0x7f07301609b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:44534/0 (socket says 172.21.15.28:44534) 2024-04-26T21:07:12.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.679+0000 7f072fe00640 1 -- 172.21.15.28:0/3283665051 learned_addr learned my addr 172.21.15.28:0/3283665051 (peer_addr_for_me v1:172.21.15.28:0/0) 2024-04-26T21:07:12.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.680+0000 7f072f400640 1 --2- 172.21.15.28:0/3283665051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0730146670 0x7f0730071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:12.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.680+0000 7f072ea00640 1 -- 172.21.15.28:0/3283665051 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2895406850 0 0) 0x7f0730161530 con 0x7f0730146060 2024-04-26T21:07:12.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.680+0000 7f072ea00640 1 -- 172.21.15.28:0/3283665051 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0714003520 con 0x7f0730146060 2024-04-26T21:07:12.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.680+0000 7f072ea00640 1 -- 172.21.15.28:0/3283665051 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 305901801 0 0) 0x7f0714003520 con 0x7f0730146060 2024-04-26T21:07:12.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.680+0000 7f072ea00640 1 -- 172.21.15.28:0/3283665051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0730146670 msgr2=0x7f0730071e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:07:12.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.680+0000 7f072ea00640 1 --2- 172.21.15.28:0/3283665051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0730146670 0x7f0730071e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:12.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.680+0000 7f072ea00640 1 -- 172.21.15.28:0/3283665051 >> v1:172.21.15.103:6789/0 conn(0x7f07301610c0 legacy=0x7f07300cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:12.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.680+0000 7f072ea00640 1 -- 172.21.15.28:0/3283665051 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073006f010 con 0x7f0730146060 2024-04-26T21:07:12.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.680+0000 7f072ea00640 1 -- 172.21.15.28:0/3283665051 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 504668055 0 0) 0x7f07200019f0 con 0x7f0730146060 2024-04-26T21:07:12.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.680+0000 7f072ea00640 1 -- 172.21.15.28:0/3283665051 >> v1:172.21.15.28:6789/0 conn(0x7f0730146060 legacy=0x7f07301609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:12.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.680+0000 7f072ea00640 1 --2- 172.21.15.28:0/3283665051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07140039e0 0x7f0714003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:12.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.680+0000 7f072ea00640 1 --2- 172.21.15.28:0/3283665051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0714004800 0x7f0714004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:12.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.680+0000 7f072ea00640 1 --2- 172.21.15.28:0/3283665051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0730146670 0x7f0714005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:12.684 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.680+0000 7f072ea00640 1 -- 172.21.15.28:0/3283665051 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f073006f510 con 0x7f0714004800 2024-04-26T21:07:12.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.680+0000 7f072ea00640 1 -- 172.21.15.28:0/3283665051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f071400bb20 con 0x7f07140039e0 2024-04-26T21:07:12.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.680+0000 7f072ea00640 1 -- 172.21.15.28:0/3283665051 --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f071400bd20 con 0x7f0730146670 2024-04-26T21:07:12.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.681+0000 7f072f400640 1 --2- 172.21.15.28:0/3283665051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0714004800 0x7f0714004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:12.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.681+0000 7f072fe00640 1 --2- 172.21.15.28:0/3283665051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07140039e0 0x7f0714003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:12.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.681+0000 7f0734e00640 1 --2- 172.21.15.28:0/3283665051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0730146670 0x7f0714005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:12.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.681+0000 7f072f400640 1 -- 172.21.15.28:0/3283665051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0730146670 msgr2=0x7f0714005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:07:12.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.681+0000 7f072f400640 1 --2- 172.21.15.28:0/3283665051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0730146670 0x7f0714005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:12.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.681+0000 7f072f400640 1 -- 172.21.15.28:0/3283665051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07140039e0 msgr2=0x7f0714003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:07:12.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.681+0000 7f072f400640 1 --2- 172.21.15.28:0/3283665051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07140039e0 0x7f0714003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:12.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.681+0000 7f072f400640 1 -- 172.21.15.28:0/3283665051 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0714003520 con 0x7f0714004800 2024-04-26T21:07:12.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.681+0000 7f0734e00640 1 --2- 172.21.15.28:0/3283665051 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0730146670 0x7f0714005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:07:12.685 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.682+0000 7f072f400640 1 --2- 172.21.15.28:0/3283665051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0714004800 0x7f0714004c80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f071c009ac0 tx=0x7f0714005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94479ab500dfbaf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:07:12.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.683+0000 7f072ea00640 1 -- 172.21.15.28:0/3283665051 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c036070 con 0x7f0714004800 2024-04-26T21:07:12.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.683+0000 7f072ea00640 1 -- 172.21.15.28:0/3283665051 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f071c031400 con 0x7f0714004800 2024-04-26T21:07:12.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.683+0000 7f072ea00640 1 -- 172.21.15.28:0/3283665051 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071c009690 con 0x7f0714004800 2024-04-26T21:07:12.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.683+0000 7f0735800640 1 -- 172.21.15.28:0/3283665051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0714004800 msgr2=0x7f0714004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:12.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.683+0000 7f0735800640 1 --2- 172.21.15.28:0/3283665051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0714004800 0x7f0714004c80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f071c009ac0 tx=0x7f0714005830 comp rx=0 tx=0).stop 2024-04-26T21:07:12.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.683+0000 7f0734e00640 1 -- 172.21.15.28:0/3283665051 reap_dead start 2024-04-26T21:07:12.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.683+0000 7f0735800640 1 -- 172.21.15.28:0/3283665051 shutdown_connections 2024-04-26T21:07:12.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.683+0000 7f0735800640 1 -- 172.21.15.28:0/3283665051 >> 172.21.15.28:0/3283665051 conn(0x7f07300d0a50 msgr2=0x7f07300d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:07:12.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.683+0000 7f0735800640 1 -- 172.21.15.28:0/3283665051 shutdown_connections 2024-04-26T21:07:12.687 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.683+0000 7f0735800640 1 -- 172.21.15.28:0/3283665051 wait complete. 2024-04-26T21:07:12.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.685+0000 7f0735800640 1 Processor -- start 2024-04-26T21:07:12.688 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.685+0000 7f0735800640 1 -- start start 2024-04-26T21:07:12.689 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.686+0000 7f0735800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0714004800 0x7f0730135b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:12.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.686+0000 7f0735800640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0730141720 0x7f07300b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:12.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.686+0000 7f0735800640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07300b4120 0x7f073015e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:12.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.686+0000 7f0735800640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f073006ff60 con 0x7f0714004800 2024-04-26T21:07:12.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.686+0000 7f0735800640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f073006fc80 con 0x7f07300b4120 2024-04-26T21:07:12.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.686+0000 7f0735800640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f0730071e20 con 0x7f0730141720 2024-04-26T21:07:12.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.687+0000 7f072fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0714004800 0x7f0730135b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:12.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.687+0000 7f072fe00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0714004800 0x7f0730135b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46840/0 (socket says 172.21.15.28:46840) 2024-04-26T21:07:12.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.687+0000 7f072fe00640 1 -- 172.21.15.28:0/1089515922 learned_addr learned my addr 172.21.15.28:0/1089515922 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:07:12.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.687+0000 7f072f400640 1 --2- 172.21.15.28:0/1089515922 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0730141720 0x7f07300b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:12.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.687+0000 7f0734e00640 1 --2- 172.21.15.28:0/1089515922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07300b4120 0x7f073015e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:12.690 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.687+0000 7f072fe00640 1 -- 172.21.15.28:0/1089515922 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0730141720 msgr2=0x7f07300b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:12.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.687+0000 7f072fe00640 1 --2- 172.21.15.28:0/1089515922 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0730141720 0x7f07300b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:12.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.687+0000 7f072fe00640 1 -- 172.21.15.28:0/1089515922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07300b4120 msgr2=0x7f073015e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:12.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.687+0000 7f072fe00640 1 --2- 172.21.15.28:0/1089515922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07300b4120 0x7f073015e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:12.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.687+0000 7f072fe00640 1 -- 172.21.15.28:0/1089515922 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07300b6a60 con 0x7f0714004800 2024-04-26T21:07:12.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.687+0000 7f0734e00640 1 --2- 172.21.15.28:0/1089515922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07300b4120 0x7f073015e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:07:12.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.688+0000 7f072fe00640 1 --2- 172.21.15.28:0/1089515922 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0714004800 0x7f0730135b90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0720007b00 tx=0x7f072000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:07:12.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.688+0000 7f0727e00640 1 -- 172.21.15.28:0/1089515922 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0720013420 con 0x7f0714004800 2024-04-26T21:07:12.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.688+0000 7f0727e00640 1 -- 172.21.15.28:0/1089515922 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0720018070 con 0x7f0714004800 2024-04-26T21:07:12.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.688+0000 7f0727e00640 1 -- 172.21.15.28:0/1089515922 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0720013820 con 0x7f0714004800 2024-04-26T21:07:12.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.688+0000 7f0735800640 1 -- 172.21.15.28:0/1089515922 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07300b5ca0 con 0x7f0714004800 2024-04-26T21:07:12.691 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.688+0000 7f0735800640 1 -- 172.21.15.28:0/1089515922 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07300b6260 con 0x7f0714004800 2024-04-26T21:07:12.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.689+0000 7f0735800640 1 -- 172.21.15.28:0/1089515922 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f073006a360 con 0x7f0714004800 2024-04-26T21:07:12.692 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.690+0000 7f0727e00640 1 -- 172.21.15.28:0/1089515922 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07200139c0 con 0x7f0714004800 2024-04-26T21:07:12.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.691+0000 7f0727e00640 1 --2- 172.21.15.28:0/1089515922 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f071006c8f0 0x7f071006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:12.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.691+0000 7f0727e00640 1 -- 172.21.15.28:0/1089515922 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f072008c0c0 con 0x7f0714004800 2024-04-26T21:07:12.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.691+0000 7f072f400640 1 --2- 172.21.15.28:0/1089515922 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f071006c8f0 0x7f071006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:12.693 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.692+0000 7f072f400640 1 --2- 172.21.15.28:0/1089515922 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f071006c8f0 0x7f071006edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f071c033000 tx=0x7f071c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:07:12.697 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.696+0000 7f0727e00640 1 -- 172.21.15.28:0/1089515922 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f072005a750 con 0x7f0714004800 2024-04-26T21:07:12.901 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.900+0000 7f0735800640 1 -- 172.21.15.28:0/1089515922 --> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f07300cc580 con 0x7f071006c8f0 2024-04-26T21:07:12.903 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.901+0000 7f0727e00640 1 -- 172.21.15.28:0/1089515922 <== mgr.4104 v2:172.21.15.28:6832/2367459182 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13360 (secure 0 0 0) 0x7f07300cc580 con 0x7f071006c8f0 2024-04-26T21:07:12.903 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T21:07:12.904 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2024-04-26T21:07:12.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.904+0000 7f0735800640 1 -- 172.21.15.28:0/1089515922 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f071006c8f0 msgr2=0x7f071006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:12.906 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.904+0000 7f0735800640 1 --2- 172.21.15.28:0/1089515922 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f071006c8f0 0x7f071006edb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f071c033000 tx=0x7f071c008030 comp rx=0 tx=0).stop 2024-04-26T21:07:12.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.904+0000 7f0735800640 1 -- 172.21.15.28:0/1089515922 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0714004800 msgr2=0x7f0730135b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:12.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.904+0000 7f0735800640 1 --2- 172.21.15.28:0/1089515922 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0714004800 0x7f0730135b90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0720007b00 tx=0x7f072000e530 comp rx=0 tx=0).stop 2024-04-26T21:07:12.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.905+0000 7f0735800640 1 -- 172.21.15.28:0/1089515922 shutdown_connections 2024-04-26T21:07:12.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.905+0000 7f0735800640 1 --2- 172.21.15.28:0/1089515922 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f071006c8f0 0x7f071006edb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:12.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.905+0000 7f0735800640 1 --2- 172.21.15.28:0/1089515922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07300b4120 0x7f073015e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:12.907 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.905+0000 7f0735800640 1 --2- 172.21.15.28:0/1089515922 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f0730141720 0x7f07300b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:12.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.905+0000 7f0735800640 1 --2- 172.21.15.28:0/1089515922 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0714004800 0x7f0730135b90 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:12.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.905+0000 7f0735800640 1 -- 172.21.15.28:0/1089515922 >> 172.21.15.28:0/1089515922 conn(0x7f07300d0a50 msgr2=0x7f07300b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:07:12.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.905+0000 7f0735800640 1 -- 172.21.15.28:0/1089515922 shutdown_connections 2024-04-26T21:07:12.908 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:12.905+0000 7f0735800640 1 -- 172.21.15.28:0/1089515922 wait complete. 2024-04-26T21:07:12.926 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-04-26T21:07:10.919008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":255272,"kb_used_data":5832,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754719448,"statfs":{"total":773094113280,"available":772832714752,"internally_reserved":0,"allocated":5971968,"data_stored":3135024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003439"},"pg_stats":[{"pgid":"1.0","version":"50'119","reported_seq":241,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T21:06:52.934739+0000","last_change":"2024-04-26T21:06:52.934816+0000","last_active":"2024-04-26T21:06:52.934739+0000","last_peered":"2024-04-26T21:06:52.934739+0000","last_clean":"2024-04-26T21:06:52.934739+0000","last_became_active":"2024-04-26T20:56:46.846091+0000","last_became_peered":"2024-04-26T20:56:46.846091+0000","last_unstale":"2024-04-26T21:06:52.934739+0000","last_undegraded":"2024-04-26T21:06:52.934739+0000","last_fullsized":"2024-04-26T21:06:52.934739+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"50'119","last_scrub_stamp":"2024-04-26T21:06:52.934537+0000","last_deep_scrub":"50'119","last_deep_scrub_stamp":"2024-04-26T21:06:52.934537+0000","last_clean_scrub_stamp":"2024-04-26T21:06:52.934537+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:41.451571+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574976,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1191936,"data_stored":834654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31764,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":598016,"data_stored":244286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31764,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":598016,"data_stored":244286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574975,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31764,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":598016,"data_stored":244286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31768,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":598016,"data_stored":244286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31764,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340076,"statfs":{"total":96636764160,"available":96604237824,"internally_reserved":0,"allocated":598016,"data_stored":244286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31768,"kb_used_data":584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340072,"statfs":{"total":96636764160,"available":96604233728,"internally_reserved":0,"allocated":598016,"data_stored":244286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1191936,"data_stored":834654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T21:07:12.927 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-26T21:07:13.133 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.132+0000 7f040be00640 1 Processor -- start 2024-04-26T21:07:13.134 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.132+0000 7f040be00640 1 -- start start 2024-04-26T21:07:13.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.134+0000 7f040be00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f040c0d1420 0x7f040c06e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:13.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.134+0000 7f040be00640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f040c0b4f40 0x7f040c06e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:13.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.134+0000 7f040be00640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f040c0b3ea0 con 0x7f040c15efd0 2024-04-26T21:07:13.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.134+0000 7f040be00640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f040c0b4180 con 0x7f040c0d1420 2024-04-26T21:07:13.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.134+0000 7f040be00640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f040c0b4460 con 0x7f040c0b4f40 2024-04-26T21:07:13.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.134+0000 7f040a000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f040c0d1420 0x7f040c06e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:13.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.135+0000 7f040a000640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f040c0d1420 0x7f040c06e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46842/0 (socket says 172.21.15.28:46842) 2024-04-26T21:07:13.136 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.135+0000 7f040a000640 1 -- 172.21.15.28:0/2949421281 learned_addr learned my addr 172.21.15.28:0/2949421281 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:07:13.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.135+0000 7f040a000640 1 -- 172.21.15.28:0/2949421281 >> v2:172.21.15.103:3300/0 conn(0x7f040c0b4f40 msgr2=0x7f040c06e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:07:13.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.135+0000 7f040a000640 1 --2- 172.21.15.28:0/2949421281 >> v2:172.21.15.103:3300/0 conn(0x7f040c0b4f40 0x7f040c06e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:13.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.135+0000 7f040a000640 1 -- 172.21.15.28:0/2949421281 >> v1:172.21.15.103:6789/0 conn(0x7f040c15efd0 legacy=0x7f040c0d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:07:13.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.135+0000 7f040a000640 1 -- 172.21.15.28:0/2949421281 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040c0b4740 con 0x7f040c0d1420 2024-04-26T21:07:13.137 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.136+0000 7f040a000640 1 --2- 172.21.15.28:0/2949421281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f040c0d1420 0x7f040c06e270 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f040c0b5be0 tx=0x7f03f802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31adf97fe2f75542 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:07:13.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.137+0000 7f0409600640 1 -- 172.21.15.28:0/2949421281 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8037070 con 0x7f040c0d1420 2024-04-26T21:07:13.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.137+0000 7f0409600640 1 -- 172.21.15.28:0/2949421281 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f8032440 con 0x7f040c0d1420 2024-04-26T21:07:13.138 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.137+0000 7f0409600640 1 -- 172.21.15.28:0/2949421281 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8032810 con 0x7f040c0d1420 2024-04-26T21:07:13.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.137+0000 7f040be00640 1 -- 172.21.15.28:0/2949421281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f040c0d1420 msgr2=0x7f040c06e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:13.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.137+0000 7f040be00640 1 --2- 172.21.15.28:0/2949421281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f040c0d1420 0x7f040c06e270 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f040c0b5be0 tx=0x7f03f802a230 comp rx=0 tx=0).stop 2024-04-26T21:07:13.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.137+0000 7f040be00640 1 -- 172.21.15.28:0/2949421281 shutdown_connections 2024-04-26T21:07:13.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.137+0000 7f040be00640 1 --2- 172.21.15.28:0/2949421281 >> v2:172.21.15.103:3300/0 conn(0x7f040c0b4f40 0x7f040c06e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:13.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.137+0000 7f040be00640 1 --2- 172.21.15.28:0/2949421281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f040c0d1420 0x7f040c06e270 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:13.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.137+0000 7f040be00640 1 -- 172.21.15.28:0/2949421281 >> 172.21.15.28:0/2949421281 conn(0x7f040c06c530 msgr2=0x7f040c06c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:07:13.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.137+0000 7f040be00640 1 -- 172.21.15.28:0/2949421281 shutdown_connections 2024-04-26T21:07:13.139 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.137+0000 7f040be00640 1 -- 172.21.15.28:0/2949421281 wait complete. 2024-04-26T21:07:13.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.139+0000 7f040be00640 1 Processor -- start 2024-04-26T21:07:13.140 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.139+0000 7f040be00640 1 -- start start 2024-04-26T21:07:13.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.140+0000 7f040be00640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f040c0b4f40 0x7f040c1475d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:13.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.140+0000 7f040be00640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f040c0d1420 0x7f040c06e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:13.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.140+0000 7f040be00640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f040c15efd0 0x7f040c06edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:13.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.140+0000 7f040be00640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f040c0b56c0 con 0x7f040c15efd0 2024-04-26T21:07:13.142 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.140+0000 7f040be00640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f040c15f3b0 con 0x7f040c0b4f40 2024-04-26T21:07:13.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.140+0000 7f040be00640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f040c0b5840 con 0x7f040c0d1420 2024-04-26T21:07:13.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.141+0000 7f040b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f040c15efd0 0x7f040c06edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:13.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.141+0000 7f040b400640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f040c15efd0 0x7f040c06edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46848/0 (socket says 172.21.15.28:46848) 2024-04-26T21:07:13.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.141+0000 7f040b400640 1 -- 172.21.15.28:0/2114869389 learned_addr learned my addr 172.21.15.28:0/2114869389 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:07:13.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.141+0000 7f040aa00640 1 --2- 172.21.15.28:0/2114869389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f040c0b4f40 0x7f040c1475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:13.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.141+0000 7f040a000640 1 --2- 172.21.15.28:0/2114869389 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f040c0d1420 0x7f040c06e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:13.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.141+0000 7f040b400640 1 -- 172.21.15.28:0/2114869389 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f040c0d1420 msgr2=0x7f040c06e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:13.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.141+0000 7f040b400640 1 --2- 172.21.15.28:0/2114869389 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f040c0d1420 0x7f040c06e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:13.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.141+0000 7f040b400640 1 -- 172.21.15.28:0/2114869389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f040c0b4f40 msgr2=0x7f040c1475d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:13.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.141+0000 7f040b400640 1 --2- 172.21.15.28:0/2114869389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f040c0b4f40 0x7f040c1475d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:13.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.141+0000 7f040b400640 1 -- 172.21.15.28:0/2114869389 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040c1f93a0 con 0x7f040c15efd0 2024-04-26T21:07:13.143 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.141+0000 7f040b400640 1 --2- 172.21.15.28:0/2114869389 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f040c15efd0 0x7f040c06edc0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f040400b9a0 tx=0x7f040400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:07:13.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.142+0000 7f0402a00640 1 -- 172.21.15.28:0/2114869389 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404019070 con 0x7f040c15efd0 2024-04-26T21:07:13.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.142+0000 7f0402a00640 1 -- 172.21.15.28:0/2114869389 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f040400cbc0 con 0x7f040c15efd0 2024-04-26T21:07:13.144 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.142+0000 7f0402a00640 1 -- 172.21.15.28:0/2114869389 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404014440 con 0x7f040c15efd0 2024-04-26T21:07:13.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.142+0000 7f040be00640 1 -- 172.21.15.28:0/2114869389 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040c15a0f0 con 0x7f040c15efd0 2024-04-26T21:07:13.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.142+0000 7f040be00640 1 -- 172.21.15.28:0/2114869389 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f040c15a6b0 con 0x7f040c15efd0 2024-04-26T21:07:13.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.143+0000 7f040be00640 1 -- 172.21.15.28:0/2114869389 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f040c06a250 con 0x7f040c15efd0 2024-04-26T21:07:13.145 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.144+0000 7f0402a00640 1 -- 172.21.15.28:0/2114869389 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f040401c070 con 0x7f040c15efd0 2024-04-26T21:07:13.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.145+0000 7f0402a00640 1 --2- 172.21.15.28:0/2114869389 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f03ec06c8f0 0x7f03ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:13.146 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.145+0000 7f0402a00640 1 -- 172.21.15.28:0/2114869389 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f040408b5f0 con 0x7f040c15efd0 2024-04-26T21:07:13.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.146+0000 7f040aa00640 1 --2- 172.21.15.28:0/2114869389 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f03ec06c8f0 0x7f03ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:13.147 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.146+0000 7f040aa00640 1 --2- 172.21.15.28:0/2114869389 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f03ec06c8f0 0x7f03ec06edb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f03fc006fd0 tx=0x7f03fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:07:13.151 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.150+0000 7f0402a00640 1 -- 172.21.15.28:0/2114869389 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0404059d10 con 0x7f040c15efd0 2024-04-26T21:07:13.355 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.354+0000 7f040be00640 1 -- 172.21.15.28:0/2114869389 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f040c0b2150 con 0x7f040c15efd0 2024-04-26T21:07:13.360 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.359+0000 7f0402a00640 1 -- 172.21.15.28:0/2114869389 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) ==== 123+0+0 (secure 0 0 0) 0x7f0404059610 con 0x7f040c15efd0 2024-04-26T21:07:13.361 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.360+0000 7f0402a00640 1 -- 172.21.15.28:0/2114869389 <== mon.0 v2:172.21.15.28:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f040405d9c0 con 0x7f040c15efd0 2024-04-26T21:07:13.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.362+0000 7f040be00640 1 -- 172.21.15.28:0/2114869389 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f03ec06c8f0 msgr2=0x7f03ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:13.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.363+0000 7f040be00640 1 --2- 172.21.15.28:0/2114869389 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f03ec06c8f0 0x7f03ec06edb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f03fc006fd0 tx=0x7f03fc008040 comp rx=0 tx=0).stop 2024-04-26T21:07:13.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.363+0000 7f040be00640 1 -- 172.21.15.28:0/2114869389 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f040c15efd0 msgr2=0x7f040c06edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:13.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.363+0000 7f040be00640 1 --2- 172.21.15.28:0/2114869389 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f040c15efd0 0x7f040c06edc0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f040400b9a0 tx=0x7f040400be60 comp rx=0 tx=0).stop 2024-04-26T21:07:13.364 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.363+0000 7f040be00640 1 -- 172.21.15.28:0/2114869389 shutdown_connections 2024-04-26T21:07:13.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.363+0000 7f040be00640 1 --2- 172.21.15.28:0/2114869389 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f03ec06c8f0 0x7f03ec06edb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:13.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.363+0000 7f040be00640 1 --2- 172.21.15.28:0/2114869389 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f040c15efd0 0x7f040c06edc0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:13.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.363+0000 7f040be00640 1 --2- 172.21.15.28:0/2114869389 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f040c0d1420 0x7f040c06e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:13.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.363+0000 7f040be00640 1 --2- 172.21.15.28:0/2114869389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f040c0b4f40 0x7f040c1475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:13.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.363+0000 7f040be00640 1 -- 172.21.15.28:0/2114869389 >> 172.21.15.28:0/2114869389 conn(0x7f040c06c530 msgr2=0x7f040c15e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:07:13.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.363+0000 7f040be00640 1 -- 172.21.15.28:0/2114869389 shutdown_connections 2024-04-26T21:07:13.365 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.363+0000 7f040be00640 1 -- 172.21.15.28:0/2114869389 wait complete. 2024-04-26T21:07:13.386 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T21:07:13.386 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T21:07:13.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.591+0000 7f3b7f200640 1 Processor -- start 2024-04-26T21:07:13.592 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.591+0000 7f3b7f200640 1 -- start start 2024-04-26T21:07:13.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.593+0000 7f3b7f200640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3b80144b90 0x7f3b800cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:13.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.593+0000 7f3b7f200640 1 --2- >> v2:172.21.15.103:3300/0 conn(0x7f3b80145080 0x7f3b800ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:13.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.593+0000 7f3b7f200640 1 -- --> v1:172.21.15.103:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b80072760 con 0x7f3b800ceb60 2024-04-26T21:07:13.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.593+0000 7f3b7f200640 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 -- 0x7f3b80072a40 con 0x7f3b80144b90 2024-04-26T21:07:13.594 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.593+0000 7f3b7f200640 1 -- --> v2:172.21.15.103:3300/0 -- mon_getmap magic: 0 -- 0x7f3b80072d20 con 0x7f3b80145080 2024-04-26T21:07:13.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.593+0000 7f3b7de00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3b80144b90 0x7f3b800cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:13.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.593+0000 7f3b7de00640 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3b80144b90 0x7f3b800cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46856/0 (socket says 172.21.15.28:46856) 2024-04-26T21:07:13.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.593+0000 7f3b7de00640 1 -- 172.21.15.28:0/503754788 learned_addr learned my addr 172.21.15.28:0/503754788 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:07:13.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.594+0000 7f3b77e00640 1 --2- 172.21.15.28:0/503754788 >> v2:172.21.15.103:3300/0 conn(0x7f3b80145080 0x7f3b800ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:13.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.594+0000 7f3b7de00640 1 -- 172.21.15.28:0/503754788 >> v2:172.21.15.103:3300/0 conn(0x7f3b80145080 msgr2=0x7f3b800ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:07:13.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.594+0000 7f3b7de00640 1 --2- 172.21.15.28:0/503754788 >> v2:172.21.15.103:3300/0 conn(0x7f3b80145080 0x7f3b800ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:13.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.594+0000 7f3b7de00640 1 -- 172.21.15.28:0/503754788 >> v1:172.21.15.103:6789/0 conn(0x7f3b800ceb60 legacy=0x7f3b80073800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T21:07:13.595 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.594+0000 7f3b7de00640 1 -- 172.21.15.28:0/503754788 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b80073000 con 0x7f3b80144b90 2024-04-26T21:07:13.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.594+0000 7f3b77e00640 1 --2- 172.21.15.28:0/503754788 >> v2:172.21.15.103:3300/0 conn(0x7f3b80145080 0x7f3b800ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:07:13.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.595+0000 7f3b7de00640 1 --2- 172.21.15.28:0/503754788 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b80144b90 0x7f3b800cb0a0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3b80071a90 tx=0x7f3b7002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f111740f9d8c65f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:07:13.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.595+0000 7f3b7d400640 1 -- 172.21.15.28:0/503754788 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b70037070 con 0x7f3b80144b90 2024-04-26T21:07:13.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.595+0000 7f3b7d400640 1 -- 172.21.15.28:0/503754788 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3b70032440 con 0x7f3b80144b90 2024-04-26T21:07:13.596 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.595+0000 7f3b7d400640 1 -- 172.21.15.28:0/503754788 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b70032830 con 0x7f3b80144b90 2024-04-26T21:07:13.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.596+0000 7f3b7f200640 1 -- 172.21.15.28:0/503754788 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b80144b90 msgr2=0x7f3b800cb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:13.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.596+0000 7f3b7f200640 1 --2- 172.21.15.28:0/503754788 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b80144b90 0x7f3b800cb0a0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3b80071a90 tx=0x7f3b7002a1a0 comp rx=0 tx=0).stop 2024-04-26T21:07:13.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.596+0000 7f3b7f200640 1 -- 172.21.15.28:0/503754788 shutdown_connections 2024-04-26T21:07:13.597 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.596+0000 7f3b7f200640 1 --2- 172.21.15.28:0/503754788 >> v2:172.21.15.103:3300/0 conn(0x7f3b80145080 0x7f3b800ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:13.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.596+0000 7f3b7f200640 1 --2- 172.21.15.28:0/503754788 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b80144b90 0x7f3b800cb0a0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:13.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.596+0000 7f3b7f200640 1 -- 172.21.15.28:0/503754788 >> 172.21.15.28:0/503754788 conn(0x7f3b800d0a50 msgr2=0x7f3b800d0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:07:13.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.596+0000 7f3b7f200640 1 -- 172.21.15.28:0/503754788 shutdown_connections 2024-04-26T21:07:13.598 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.596+0000 7f3b7f200640 1 -- 172.21.15.28:0/503754788 wait complete. 2024-04-26T21:07:13.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.598+0000 7f3b7f200640 1 Processor -- start 2024-04-26T21:07:13.599 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.598+0000 7f3b7f200640 1 -- start start 2024-04-26T21:07:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.599+0000 7f3b7f200640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b800ceb60 0x7f3b80157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.599+0000 7f3b7f200640 1 --2- >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3b80144b90 0x7f3b800b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.599+0000 7f3b7f200640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b80145080 0x7f3b800b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.599+0000 7f3b7f200640 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 -- 0x7f3b80074090 con 0x7f3b80145080 2024-04-26T21:07:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.599+0000 7f3b7f200640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b80073f10 con 0x7f3b800ceb60 2024-04-26T21:07:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.599+0000 7f3b7f200640 1 -- --> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] -- mon_getmap magic: 0 -- 0x7f3b80166a30 con 0x7f3b80144b90 2024-04-26T21:07:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.599+0000 7f3b7e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b80145080 0x7f3b800b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.599+0000 7f3b7e800640 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b80145080 0x7f3b800b61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46870/0 (socket says 172.21.15.28:46870) 2024-04-26T21:07:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.599+0000 7f3b7e800640 1 -- 172.21.15.28:0/200967445 learned_addr learned my addr 172.21.15.28:0/200967445 (peer_addr_for_me v2:172.21.15.28:0/0) 2024-04-26T21:07:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.599+0000 7f3b7de00640 1 --2- 172.21.15.28:0/200967445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b800ceb60 0x7f3b80157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:13.601 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.600+0000 7f3b7e800640 1 -- 172.21.15.28:0/200967445 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3b80144b90 msgr2=0x7f3b800b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:13.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.600+0000 7f3b7e800640 1 --2- 172.21.15.28:0/200967445 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3b80144b90 0x7f3b800b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:13.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.600+0000 7f3b7e800640 1 -- 172.21.15.28:0/200967445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b800ceb60 msgr2=0x7f3b80157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:13.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.600+0000 7f3b7e800640 1 --2- 172.21.15.28:0/200967445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b800ceb60 0x7f3b80157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:13.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.600+0000 7f3b7e800640 1 -- 172.21.15.28:0/200967445 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b800b46d0 con 0x7f3b80145080 2024-04-26T21:07:13.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.600+0000 7f3b7de00640 1 --2- 172.21.15.28:0/200967445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b800ceb60 0x7f3b80157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T21:07:13.602 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.600+0000 7f3b7e800640 1 --2- 172.21.15.28:0/200967445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b80145080 0x7f3b800b61e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800c700 tx=0x7f3b7800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:07:13.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.600+0000 7f3b76000640 1 -- 172.21.15.28:0/200967445 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b78016020 con 0x7f3b80145080 2024-04-26T21:07:13.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.600+0000 7f3b76000640 1 -- 172.21.15.28:0/200967445 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f3b7800aa50 con 0x7f3b80145080 2024-04-26T21:07:13.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.600+0000 7f3b76000640 1 -- 172.21.15.28:0/200967445 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7800ae60 con 0x7f3b80145080 2024-04-26T21:07:13.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.600+0000 7f3b7f200640 1 -- 172.21.15.28:0/200967445 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8012dd90 con 0x7f3b80145080 2024-04-26T21:07:13.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.600+0000 7f3b7f200640 1 -- 172.21.15.28:0/200967445 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b8012e350 con 0x7f3b80145080 2024-04-26T21:07:13.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.602+0000 7f3b7f200640 1 -- 172.21.15.28:0/200967445 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b8006a360 con 0x7f3b80145080 2024-04-26T21:07:13.603 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.603+0000 7f3b76000640 1 -- 172.21.15.28:0/200967445 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b78019020 con 0x7f3b80145080 2024-04-26T21:07:13.604 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.603+0000 7f3b76000640 1 --2- 172.21.15.28:0/200967445 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3b5c06c8f0 0x7f3b5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T21:07:13.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.604+0000 7f3b76000640 1 -- 172.21.15.28:0/200967445 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f3b7808bc80 con 0x7f3b80145080 2024-04-26T21:07:13.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.604+0000 7f3b7de00640 1 --2- 172.21.15.28:0/200967445 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3b5c06c8f0 0x7f3b5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T21:07:13.605 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.605+0000 7f3b7de00640 1 --2- 172.21.15.28:0/200967445 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3b5c06c8f0 0x7f3b5c06edb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3b70034000 tx=0x7f3b7002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T21:07:13.609 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.609+0000 7f3b76000640 1 -- 172.21.15.28:0/200967445 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3b78020020 con 0x7f3b80145080 2024-04-26T21:07:13.860 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.859+0000 7f3b7f200640 1 -- 172.21.15.28:0/200967445 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b800739a0 con 0x7f3b80145080 2024-04-26T21:07:13.861 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.860+0000 7f3b76000640 1 -- 172.21.15.28:0/200967445 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+3869 (secure 0 0 0) 0x7f3b78059c30 con 0x7f3b80145080 2024-04-26T21:07:13.861 INFO:teuthology.orchestra.run.smithi028.stdout: 2024-04-26T21:07:13.861 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4820,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6835/356870645","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":356870645},{"type":"v1","addr":"172.21.15.28:6835","nonce":356870645}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":4829,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2957511478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6833","nonce":2957511478},{"type":"v1","addr":"172.21.15.103:6835","nonce":2957511478}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5053,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6834/2060632176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":2060632176},{"type":"v1","addr":"172.21.15.103:6834","nonce":2060632176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5895,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/513461768","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6836","nonce":513461768},{"type":"v1","addr":"172.21.15.28:6837","nonce":513461768}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[]} 2024-04-26T21:07:13.861 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 28 2024-04-26T21:07:13.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.862+0000 7f3b7f200640 1 -- 172.21.15.28:0/200967445 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3b5c06c8f0 msgr2=0x7f3b5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:13.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.862+0000 7f3b7f200640 1 --2- 172.21.15.28:0/200967445 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3b5c06c8f0 0x7f3b5c06edb0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3b70034000 tx=0x7f3b7002e000 comp rx=0 tx=0).stop 2024-04-26T21:07:13.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.862+0000 7f3b7f200640 1 -- 172.21.15.28:0/200967445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b80145080 msgr2=0x7f3b800b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T21:07:13.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.862+0000 7f3b7f200640 1 --2- 172.21.15.28:0/200967445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b80145080 0x7f3b800b61e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3b7800c700 tx=0x7f3b7800cbc0 comp rx=0 tx=0).stop 2024-04-26T21:07:13.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.863+0000 7f3b7f200640 1 -- 172.21.15.28:0/200967445 shutdown_connections 2024-04-26T21:07:13.864 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.863+0000 7f3b7f200640 1 --2- 172.21.15.28:0/200967445 >> [v2:172.21.15.28:6832/2367459182,v1:172.21.15.28:6833/2367459182] conn(0x7f3b5c06c8f0 0x7f3b5c06edb0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:13.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.863+0000 7f3b7f200640 1 --2- 172.21.15.28:0/200967445 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b80145080 0x7f3b800b61e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:13.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.863+0000 7f3b7f200640 1 --2- 172.21.15.28:0/200967445 >> [v2:172.21.15.103:3301/0,v1:172.21.15.103:6790/0] conn(0x7f3b80144b90 0x7f3b800b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:13.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.863+0000 7f3b7f200640 1 --2- 172.21.15.28:0/200967445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b800ceb60 0x7f3b80157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T21:07:13.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.863+0000 7f3b7f200640 1 -- 172.21.15.28:0/200967445 >> 172.21.15.28:0/200967445 conn(0x7f3b800d0a50 msgr2=0x7f3b80159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T21:07:13.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.863+0000 7f3b7f200640 1 -- 172.21.15.28:0/200967445 shutdown_connections 2024-04-26T21:07:13.865 INFO:teuthology.orchestra.run.smithi028.stderr:2024-04-26T21:07:13.863+0000 7f3b7f200640 1 -- 172.21.15.28:0/200967445 wait complete. 2024-04-26T21:07:13.883 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-26T21:07:13.883 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-26T21:07:13.883 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T21:07:13.883 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:07:13.903 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T21:07:13.904 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T21:07:13.904 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:07:13.966 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T21:07:13.967 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T21:07:13.967 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:07:14.001 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T21:07:14.002 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T21:07:14.002 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:07:14.076 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T21:07:14.076 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-26T21:07:14.076 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-26T21:07:14.076 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:07:14.128 INFO:tasks.ceph.osd.0:Stopped 2024-04-26T21:07:14.128 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-26T21:07:14.128 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:07:14.206 INFO:tasks.ceph.osd.1:Stopped 2024-04-26T21:07:14.206 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-26T21:07:14.207 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:07:14.293 INFO:tasks.ceph.osd.2:Stopped 2024-04-26T21:07:14.293 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-26T21:07:14.293 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:07:14.361 INFO:tasks.ceph.osd.3:Stopped 2024-04-26T21:07:14.362 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-26T21:07:14.362 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:07:14.445 INFO:tasks.ceph.osd.4:Stopped 2024-04-26T21:07:14.445 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-26T21:07:14.445 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:07:14.522 INFO:tasks.ceph.osd.5:Stopped 2024-04-26T21:07:14.523 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-26T21:07:14.523 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:07:14.609 INFO:tasks.ceph.osd.6:Stopped 2024-04-26T21:07:14.609 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-26T21:07:14.609 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:07:14.687 INFO:tasks.ceph.osd.7:Stopped 2024-04-26T21:07:14.687 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-26T21:07:14.687 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-26T21:07:14.687 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:07:14.967 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2024-04-26T21:07:18.390 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:07:18.398 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:07:20.372 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~6s 2024-04-26T21:07:20.789 INFO:tasks.ceph.mgr.y:Stopped 2024-04-26T21:07:20.790 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-26T21:07:20.790 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:07:25.676 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2024-04-26T21:07:25.676 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2024-04-26T21:07:26.891 INFO:tasks.ceph.mgr.x:Stopped 2024-04-26T21:07:26.892 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-26T21:07:26.892 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-26T21:07:26.892 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:07:26.945 INFO:tasks.ceph.mon.a:Stopped 2024-04-26T21:07:26.945 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-26T21:07:26.945 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:07:26.985 INFO:tasks.ceph.mon.b:Stopped 2024-04-26T21:07:26.986 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-26T21:07:26.986 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T21:07:27.053 INFO:tasks.ceph.mon.c:Stopped 2024-04-26T21:07:27.053 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-26T21:07:27.054 DEBUG:teuthology.orchestra.run.smithi028:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-26T21:07:27.155 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T21:07:27.156 DEBUG:teuthology.orchestra.run.smithi028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-26T21:07:27.363 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T21:07:27.363 DEBUG:teuthology.orchestra.run.smithi028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-26T21:07:27.533 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T21:07:27.533 DEBUG:teuthology.orchestra.run.smithi028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-26T21:07:27.677 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T21:07:27.677 DEBUG:teuthology.orchestra.run.smithi028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-26T21:07:27.839 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi103.front.sepia.ceph.com 2024-04-26T21:07:27.839 DEBUG:teuthology.orchestra.run.smithi103:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-26T21:07:28.067 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi103.front.sepia.ceph.com 2024-04-26T21:07:28.067 DEBUG:teuthology.orchestra.run.smithi103:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-26T21:07:28.204 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi103.front.sepia.ceph.com 2024-04-26T21:07:28.204 DEBUG:teuthology.orchestra.run.smithi103:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-26T21:07:28.336 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi103.front.sepia.ceph.com 2024-04-26T21:07:28.336 DEBUG:teuthology.orchestra.run.smithi103:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-26T21:07:28.484 INFO:tasks.ceph:Archiving mon data... 2024-04-26T21:07:28.485 DEBUG:teuthology.misc:Transferring archived files from smithi028:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675182/data/mon.a.tgz 2024-04-26T21:07:28.485 DEBUG:teuthology.orchestra.run.smithi028:> mktemp 2024-04-26T21:07:28.546 INFO:teuthology.orchestra.run.smithi028.stdout:/tmp/tmp.H7bOfKZ4QV 2024-04-26T21:07:28.546 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.H7bOfKZ4QV 2024-04-26T21:07:28.701 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 0666 /tmp/tmp.H7bOfKZ4QV 2024-04-26T21:07:28.849 DEBUG:teuthology.orchestra.remote:smithi028:/tmp/tmp.H7bOfKZ4QV is 240KB 2024-04-26T21:07:28.961 DEBUG:teuthology.orchestra.run.smithi028:> rm -fr /tmp/tmp.H7bOfKZ4QV 2024-04-26T21:07:29.020 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675182/data/mon.b.tgz 2024-04-26T21:07:29.021 DEBUG:teuthology.orchestra.run.smithi103:> mktemp 2024-04-26T21:07:29.071 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/tmp.rWAiE6DaCw 2024-04-26T21:07:29.071 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.rWAiE6DaCw 2024-04-26T21:07:29.216 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0666 /tmp/tmp.rWAiE6DaCw 2024-04-26T21:07:29.342 DEBUG:teuthology.orchestra.remote:smithi103:/tmp/tmp.rWAiE6DaCw is 243KB 2024-04-26T21:07:29.437 DEBUG:teuthology.orchestra.run.smithi103:> rm -fr /tmp/tmp.rWAiE6DaCw 2024-04-26T21:07:29.487 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675182/data/mon.c.tgz 2024-04-26T21:07:29.487 DEBUG:teuthology.orchestra.run.smithi103:> mktemp 2024-04-26T21:07:29.577 INFO:teuthology.orchestra.run.smithi103.stdout:/tmp/tmp.5VhiIQLnS4 2024-04-26T21:07:29.578 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.5VhiIQLnS4 2024-04-26T21:07:29.702 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 0666 /tmp/tmp.5VhiIQLnS4 2024-04-26T21:07:29.815 DEBUG:teuthology.orchestra.remote:smithi103:/tmp/tmp.5VhiIQLnS4 is 243KB 2024-04-26T21:07:29.906 DEBUG:teuthology.orchestra.run.smithi103:> rm -fr /tmp/tmp.5VhiIQLnS4 2024-04-26T21:07:29.961 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-26T21:07:29.961 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T21:07:29.969 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T21:07:30.111 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T21:07:30.167 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T21:07:30.167 INFO:tasks.ceph:Archiving crash dumps... 2024-04-26T21:07:30.169 DEBUG:teuthology.misc:Transferring archived files from smithi028:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675182/remote/smithi028/crash 2024-04-26T21:07:30.170 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T21:07:30.265 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675182/remote/smithi103/crash 2024-04-26T21:07:30.265 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T21:07:30.351 INFO:tasks.ceph:Shutting down logrotate 2024-04-26T21:07:30.351 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:07:30.358 DEBUG:teuthology.orchestra.run.smithi103:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T21:07:30.478 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T21:07:30.570 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T21:07:30.657 INFO:tasks.ceph:Compressing logs... 2024-04-26T21:07:30.658 DEBUG:teuthology.orchestra.run.smithi028:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T21:07:30.665 DEBUG:teuthology.orchestra.run.smithi103:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T21:07:30.676 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2024-04-26T21:07:30.743 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39268.log 2024-04-26T21:07:30.744 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33289.log 2024-04-26T21:07:30.745 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.3.39268.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T21:07:30.745 INFO:teuthology.orchestra.run.smithi103.stderr: 91.6%/var/log/ceph/ceph-client.admin.33289.log: -- replaced with /var/log/ceph/ceph-client.3.39268.log.gz 2024-04-26T21:07:30.746 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33460.log 2024-04-26T21:07:30.746 INFO:teuthology.orchestra.run.smithi103.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33289.log.gz 2024-04-26T21:07:30.747 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T21:07:30.748 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44936.log 2024-04-26T21:07:30.749 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33460.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.33460.log.gz 2024-04-26T21:07:30.749 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-26T21:07:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph.audit.log: 93.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T21:07:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-26T21:07:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-26T21:07:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-26T21:07:30.760 INFO:teuthology.orchestra.run.smithi103.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T21:07:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33380.log 2024-04-26T21:07:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-osd.4.log: 93.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-26T21:07:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33329.log 2024-04-26T21:07:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-26T21:07:30.760 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33380.log: /var/log/ceph/ceph-client.admin.33329.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33380.log.gz 2024-04-26T21:07:30.761 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-26T21:07:30.761 INFO:teuthology.orchestra.run.smithi103.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33329.log.gz 2024-04-26T21:07:30.761 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-26T21:07:30.762 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33632.log 2024-04-26T21:07:30.762 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44936.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44936.log.gz 2024-04-26T21:07:30.762 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.83426.log 2024-04-26T21:07:30.762 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.33632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42854.log 2024-04-26T21:07:30.762 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33632.log.gz 2024-04-26T21:07:30.762 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41205.log 2024-04-26T21:07:30.762 INFO:teuthology.orchestra.run.smithi028.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83426.log.gz 2024-04-26T21:07:30.762 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42854.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42854.log.gz 2024-04-26T21:07:30.762 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41247.log 2024-04-26T21:07:30.762 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83708.log 2024-04-26T21:07:30.762 INFO:teuthology.orchestra.run.smithi028.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41205.log.gz 2024-04-26T21:07:30.763 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37962.log 2024-04-26T21:07:30.763 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41247.log.gz 2024-04-26T21:07:30.763 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83708.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.83708.log.gz 2024-04-26T21:07:30.763 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.25960.log 2024-04-26T21:07:30.763 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47052.log 2024-04-26T21:07:30.763 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37962.log.gz 2024-04-26T21:07:30.763 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph.tmp-client.admin.25960.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.25960.log.gz 2024-04-26T21:07:30.763 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48142.log 2024-04-26T21:07:30.763 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47052.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47052.log.gz 2024-04-26T21:07:30.763 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40995.log 2024-04-26T21:07:30.765 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48142.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48142.log.gz 2024-04-26T21:07:30.766 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41751.log 2024-04-26T21:07:30.766 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39642.log 2024-04-26T21:07:30.766 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40995.log.gz 2024-04-26T21:07:30.766 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42402.log 2024-04-26T21:07:30.767 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33420.log 2024-04-26T21:07:30.767 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33169.log 2024-04-26T21:07:30.767 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33420.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33420.log.gz 2024-04-26T21:07:30.767 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33169.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33169.log.gz 2024-04-26T21:07:30.767 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.38812.log 2024-04-26T21:07:30.768 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33209.log 2024-04-26T21:07:30.770 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41751.log.gz 2024-04-26T21:07:30.770 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83289.log 2024-04-26T21:07:30.770 INFO:teuthology.orchestra.run.smithi028.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.39642.log.gz 2024-04-26T21:07:30.770 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43344.log 2024-04-26T21:07:30.770 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42402.log.gz 2024-04-26T21:07:30.770 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44980.log 2024-04-26T21:07:30.770 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83289.log.gz 2024-04-26T21:07:30.770 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38256.log 2024-04-26T21:07:30.770 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43344.log.gz 2024-04-26T21:07:30.771 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47220.log 2024-04-26T21:07:30.771 INFO:teuthology.orchestra.run.smithi028.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44980.log.gz 2024-04-26T21:07:30.773 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.1.38812.log: /var/log/ceph/ceph-client.admin.33209.log: gzip -5 --verbose -- 90.6% -- replaced with /var/log/ceph/ceph-client.1.38812.log.gz 2024-04-26T21:07:30.773 INFO:teuthology.orchestra.run.smithi103.stderr: /var/log/ceph/ceph-client.admin.33249.log 2024-04-26T21:07:30.773 INFO:teuthology.orchestra.run.smithi103.stderr: 94.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-26T21:07:30.773 INFO:teuthology.orchestra.run.smithi103.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33209.log.gz 2024-04-26T21:07:30.773 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-26T21:07:30.773 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-client.admin.33249.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-26T21:07:30.773 INFO:teuthology.orchestra.run.smithi103.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33249.log.gz 2024-04-26T21:07:30.774 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38811.log 2024-04-26T21:07:30.775 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38256.log.gz 2024-04-26T21:07:30.775 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47220.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47220.log.gz 2024-04-26T21:07:30.775 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43470.log 2024-04-26T21:07:30.775 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47262.log 2024-04-26T21:07:30.775 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38811.log.gz 2024-04-26T21:07:30.776 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47645.log 2024-04-26T21:07:30.776 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43470.log.gz 2024-04-26T21:07:30.777 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47262.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47262.log.gz 2024-04-26T21:07:30.777 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41079.log 2024-04-26T21:07:30.778 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44471.log 2024-04-26T21:07:30.778 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47645.log.gz 2024-04-26T21:07:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43848.log 2024-04-26T21:07:30.780 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41079.log.gz 2024-04-26T21:07:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44471.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44471.log.gz 2024-04-26T21:07:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48478.log 2024-04-26T21:07:30.781 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44556.log 2024-04-26T21:07:30.782 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43848.log.gz 2024-04-26T21:07:30.782 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38813.log 2024-04-26T21:07:30.783 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44556.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.48478.log.gz 2024-04-26T21:07:30.784 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44556.log.gz 2024-04-26T21:07:30.784 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38807.log 2024-04-26T21:07:30.785 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84451.log 2024-04-26T21:07:30.785 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38807.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38813.log.gz 2024-04-26T21:07:30.786 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38807.log.gz 2024-04-26T21:07:30.786 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44390.log 2024-04-26T21:07:30.787 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42654.log 2024-04-26T21:07:30.788 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.84451.log.gz 2024-04-26T21:07:30.788 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44390.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44390.log.gz 2024-04-26T21:07:30.789 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41373.log 2024-04-26T21:07:30.790 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38475.log 2024-04-26T21:07:30.791 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42654.log.gz 2024-04-26T21:07:30.792 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83753.log 2024-04-26T21:07:30.792 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41373.log.gz 2024-04-26T21:07:30.794 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38475.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38475.log.gz 2024-04-26T21:07:30.794 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48436.log 2024-04-26T21:07:30.794 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83753.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T21:07:30.796 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.83753.log.gz 2024-04-26T21:07:30.796 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46328.log 2024-04-26T21:07:30.796 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48436.log.gz 2024-04-26T21:07:30.797 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84283.log 2024-04-26T21:07:30.798 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46328.log: 87.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T21:07:30.798 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83623.log 2024-04-26T21:07:30.798 INFO:teuthology.orchestra.run.smithi028.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46328.log.gz 2024-04-26T21:07:30.800 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48394.log 2024-04-26T21:07:30.800 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.84283.log.gz 2024-04-26T21:07:30.800 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83623.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.83623.log.gz 2024-04-26T21:07:30.801 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48058.log 2024-04-26T21:07:30.801 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47136.log 2024-04-26T21:07:30.802 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48394.log.gz 2024-04-26T21:07:30.803 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43512.log 2024-04-26T21:07:30.803 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3%/var/log/ceph/ceph-client.admin.47136.log: -- replaced with /var/log/ceph/ceph-client.admin.48058.log.gz 2024-04-26T21:07:30.804 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47136.log.gz 2024-04-26T21:07:30.804 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37878.log 2024-04-26T21:07:30.805 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41121.log 2024-04-26T21:07:30.806 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43512.log.gz 2024-04-26T21:07:30.807 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40783.log 2024-04-26T21:07:30.807 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37878.log.gz 2024-04-26T21:07:30.808 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43974.log 2024-04-26T21:07:30.808 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41121.log.gz 2024-04-26T21:07:30.809 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40783.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40783.log.gz 2024-04-26T21:07:30.809 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41541.log 2024-04-26T21:07:30.810 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33366.log 2024-04-26T21:07:30.811 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43974.log.gz 2024-04-26T21:07:30.812 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41541.log: gzip 87.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.41541.log.gz -- 2024-04-26T21:07:30.812 INFO:teuthology.orchestra.run.smithi028.stderr: /var/log/ceph/ceph-client.admin.47479.log 2024-04-26T21:07:30.813 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.33366.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-26T21:07:30.814 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47479.log: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.33366.log.gz 2024-04-26T21:07:30.814 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47479.log.gz 2024-04-26T21:07:30.814 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42738.log 2024-04-26T21:07:30.815 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47304.log 2024-04-26T21:07:30.816 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42738.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42738.log.gz 2024-04-26T21:07:30.816 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43428.log 2024-04-26T21:07:30.817 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83029.log 2024-04-26T21:07:30.818 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47304.log.gz 2024-04-26T21:07:30.819 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43428.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43428.log.gz 2024-04-26T21:07:30.819 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48100.log 2024-04-26T21:07:30.820 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41667.log 2024-04-26T21:07:30.821 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48100.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.83029.log.gz 2024-04-26T21:07:30.821 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48100.log.gz 2024-04-26T21:07:30.821 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44808.log 2024-04-26T21:07:30.823 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41163.log 2024-04-26T21:07:30.823 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41667.log.gz 2024-04-26T21:07:30.824 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47952.log 2024-04-26T21:07:30.824 INFO:teuthology.orchestra.run.smithi028.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44808.log.gz 2024-04-26T21:07:30.826 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38477.log 2024-04-26T21:07:30.826 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41163.log.gz 2024-04-26T21:07:30.827 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47952.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47952.log.gz 2024-04-26T21:07:30.827 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38466.log 2024-04-26T21:07:30.828 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84409.log 2024-04-26T21:07:30.829 INFO:teuthology.orchestra.run.smithi028.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38477.log.gz 2024-04-26T21:07:30.829 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82725.log 2024-04-26T21:07:30.830 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38466.log.gz 2024-04-26T21:07:30.831 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48352.log 2024-04-26T21:07:30.831 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.84409.log.gz 2024-04-26T21:07:30.832 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43554.log 2024-04-26T21:07:30.833 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.82725.log.gz 2024-04-26T21:07:30.833 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48352.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48352.log.gz 2024-04-26T21:07:30.833 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38479.log 2024-04-26T21:07:30.835 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43554.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T21:07:30.835 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43554.log.gz 2024-04-26T21:07:30.837 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47010.log 2024-04-26T21:07:30.837 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38479.log.gz 2024-04-26T21:07:30.838 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37385.log 2024-04-26T21:07:30.839 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47010.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47010.log.gz 2024-04-26T21:07:30.840 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83071.log 2024-04-26T21:07:30.840 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T21:07:30.842 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40953.log 2024-04-26T21:07:30.842 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37385.log.gz 2024-04-26T21:07:30.842 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43722.log 2024-04-26T21:07:30.842 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83071.log.gz 2024-04-26T21:07:30.843 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83803.log 2024-04-26T21:07:30.844 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40953.log.gz 2024-04-26T21:07:30.844 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46506.log 2024-04-26T21:07:30.846 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43722.log.gz 2024-04-26T21:07:30.846 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83803.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83803.log.gz 2024-04-26T21:07:30.850 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39138.log 2024-04-26T21:07:30.854 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37698.log 2024-04-26T21:07:30.854 INFO:teuthology.orchestra.run.smithi028.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46506.log.gz 2024-04-26T21:07:30.855 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43224.log 2024-04-26T21:07:30.855 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39138.log.gz 2024-04-26T21:07:30.855 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38816.log 2024-04-26T21:07:30.855 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37698.log.gz 2024-04-26T21:07:30.855 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37466.log 2024-04-26T21:07:30.855 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43224.log.gz 2024-04-26T21:07:30.856 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39096.log 2024-04-26T21:07:30.856 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38816.log.gz 2024-04-26T21:07:30.856 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37466.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37466.log.gz -5 2024-04-26T21:07:30.856 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38809.log 2024-04-26T21:07:30.857 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: 92.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-26T21:07:30.863 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47094.log 2024-04-26T21:07:30.863 INFO:teuthology.orchestra.run.smithi028.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39096.log.gz 2024-04-26T21:07:30.863 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47178.log 2024-04-26T21:07:30.863 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38809.log.gz 2024-04-26T21:07:30.864 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38480.log 2024-04-26T21:07:30.864 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47094.log.gz 2024-04-26T21:07:30.864 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47178.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47178.log.gz 2024-04-26T21:07:30.864 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44766.log 2024-04-26T21:07:30.864 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82945.log 2024-04-26T21:07:30.864 INFO:teuthology.orchestra.run.smithi028.stderr: 87.5%/var/log/ceph/ceph-client.admin.44766.log: -- replaced with /var/log/ceph/ceph-client.admin.38480.log.gz 2024-04-26T21:07:30.865 INFO:teuthology.orchestra.run.smithi028.stderr:gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.44766.log.gz 2024-04-26T21:07:30.865 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.0.48828.log 2024-04-26T21:07:30.865 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41835.log 2024-04-26T21:07:30.865 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.82945.log.gz 2024-04-26T21:07:30.865 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.48828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47560.log 2024-04-26T21:07:30.868 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41835.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-26T21:07:30.868 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41835.log.gz 2024-04-26T21:07:30.868 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40285.log 2024-04-26T21:07:30.869 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47560.log.gz 2024-04-26T21:07:30.869 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39264.log 2024-04-26T21:07:30.871 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42570.log 88.5% 2024-04-26T21:07:30.871 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40285.log.gz 2024-04-26T21:07:30.871 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45400.log 2024-04-26T21:07:30.871 INFO:teuthology.orchestra.run.smithi028.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39264.log.gz 2024-04-26T21:07:30.873 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42570.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42570.log.gz 2024-04-26T21:07:30.873 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82683.log 2024-04-26T21:07:30.873 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39012.log 2024-04-26T21:07:30.875 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45400.log.gz 2024-04-26T21:07:30.875 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38088.log 2024-04-26T21:07:30.875 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.82683.log.gz 2024-04-26T21:07:30.876 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82819.log 2024-04-26T21:07:30.876 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39012.log.gz 2024-04-26T21:07:30.877 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38088.log: gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38088.log.gz -5 2024-04-26T21:07:30.877 INFO:teuthology.orchestra.run.smithi028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41499.log 2024-04-26T21:07:30.878 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43596.log 2024-04-26T21:07:30.879 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.82819.log.gz 2024-04-26T21:07:30.880 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41499.log: gzip -5 --verbose -- 87.4% /var/log/ceph/ceph-client.admin.41625.log -- replaced with /var/log/ceph/ceph-client.admin.41499.log.gz 2024-04-26T21:07:30.880 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-04-26T21:07:30.881 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41930.log 2024-04-26T21:07:30.882 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43596.log.gz 2024-04-26T21:07:30.883 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41625.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41625.log.gz 2024-04-26T21:07:30.883 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42240.log 2024-04-26T21:07:30.884 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44226.log 2024-04-26T21:07:30.885 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41930.log.gz 2024-04-26T21:07:30.886 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84618.log 2024-04-26T21:07:30.886 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42240.log.gz 2024-04-26T21:07:30.887 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48226.log 2024-04-26T21:07:30.887 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44226.log.gz 2024-04-26T21:07:30.889 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42780.log 2024-04-26T21:07:30.889 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.84618.log.gz 2024-04-26T21:07:30.890 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48226.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-26T21:07:30.890 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48226.log.gz 2024-04-26T21:07:30.892 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44058.log 2024-04-26T21:07:30.892 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42780.log.gz 2024-04-26T21:07:30.893 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39222.log 2024-04-26T21:07:30.895 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83384.log 2024-04-26T21:07:30.895 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44058.log.gz 2024-04-26T21:07:30.896 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42980.log 2024-04-26T21:07:30.896 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39222.log.gz 2024-04-26T21:07:30.898 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42056.log 2024-04-26T21:07:30.898 INFO:teuthology.orchestra.run.smithi028.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.83384.log.gz 2024-04-26T21:07:30.899 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84536.log 2024-04-26T21:07:30.899 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42980.log.gz 2024-04-26T21:07:30.901 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43182.log 2024-04-26T21:07:30.901 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42056.log.gz 2024-04-26T21:07:30.902 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44513.log 2024-04-26T21:07:30.902 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84536.log.gz 2024-04-26T21:07:30.904 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39390.log 2024-04-26T21:07:30.904 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43182.log.gz 2024-04-26T21:07:30.905 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84073.log 2024-04-26T21:07:30.905 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44513.log.gz 2024-04-26T21:07:30.906 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39390.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-26T21:07:30.906 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39390.log.gz 2024-04-26T21:07:30.907 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84073.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-26T21:07:30.907 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84073.log.gz 2024-04-26T21:07:30.908 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38798.log 2024-04-26T21:07:30.910 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83155.log 2024-04-26T21:07:30.911 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38798.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38798.log.gz 2024-04-26T21:07:30.912 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-26T21:07:30.913 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83155.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.83155.log.gz 2024-04-26T21:07:30.915 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45460.log 2024-04-26T21:07:30.917 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38478.log 2024-04-26T21:07:30.920 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43932.log 2024-04-26T21:07:30.921 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38478.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.45460.log.gz 2024-04-26T21:07:30.922 INFO:teuthology.orchestra.run.smithi028.stderr:gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38478.log.gz 2024-04-26T21:07:30.922 INFO:teuthology.orchestra.run.smithi028.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.84576.log 2024-04-26T21:07:30.923 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43932.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43932.log.gz 2024-04-26T21:07:30.924 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41331.log 2024-04-26T21:07:30.928 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84576.log: /var/log/ceph/ceph-client.admin.41331.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41331.log.gz 2024-04-26T21:07:30.929 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47602.log 2024-04-26T21:07:30.931 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47602.log: -- replaced with /var/log/ceph/ceph-client.admin.84576.log.gz 2024-04-26T21:07:30.932 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38046.log 2024-04-26T21:07:30.934 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39054.log 2024-04-26T21:07:30.936 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3%/var/log/ceph/ceph-client.admin.38046.log: -- replaced with /var/log/ceph/ceph-client.admin.47602.log.gz 2024-04-26T21:07:30.938 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82380.log 2024-04-26T21:07:30.938 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38046.log.gz 2024-04-26T21:07:30.941 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82380.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.39054.log.gz -5 --verbose 2024-04-26T21:07:30.941 INFO:teuthology.orchestra.run.smithi028.stderr: -- /var/log/ceph/ceph-client.admin.37792.log 2024-04-26T21:07:30.943 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84013.log 2024-04-26T21:07:30.945 INFO:teuthology.orchestra.run.smithi028.stderr: 87.5%/var/log/ceph/ceph-client.admin.37792.log: -- replaced with /var/log/ceph/ceph-client.admin.82380.log.gz 2024-04-26T21:07:30.947 INFO:teuthology.orchestra.run.smithi028.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37792.log.gz 2024-04-26T21:07:30.948 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82861.log 2024-04-26T21:07:30.951 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82861.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.84013.log.gz 2024-04-26T21:07:30.952 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42098.log 2024-04-26T21:07:30.953 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41289.log 2024-04-26T21:07:30.956 INFO:teuthology.orchestra.run.smithi028.stderr: 89.4%/var/log/ceph/ceph-client.admin.42098.log: /var/log/ceph/ceph-client.admin.41289.log: -- replaced with /var/log/ceph/ceph-client.admin.82861.log.gz 2024-04-26T21:07:30.956 INFO:teuthology.orchestra.run.smithi028.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41289.log.gz 2024-04-26T21:07:30.957 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42098.log.gz 2024-04-26T21:07:30.958 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48016.log 2024-04-26T21:07:30.961 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46716.log 2024-04-26T21:07:30.962 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48016.log.gz 2024-04-26T21:07:30.964 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45358.log 2024-04-26T21:07:30.968 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46716.log: /var/log/ceph/ceph-client.admin.45358.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45190.log 2024-04-26T21:07:30.970 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46716.log.gz 2024-04-26T21:07:30.970 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45358.log.gz 2024-04-26T21:07:30.971 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41972.log 2024-04-26T21:07:30.973 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45190.log: /var/log/ceph/ceph-client.admin.41972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41709.log 2024-04-26T21:07:30.977 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45022.log 2024-04-26T21:07:30.977 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45190.log.gz 2024-04-26T21:07:30.978 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41972.log.gz 2024-04-26T21:07:30.979 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41709.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41709.log.gz 2024-04-26T21:07:30.980 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37920.log 2024-04-26T21:07:30.984 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37920.log: /var/log/ceph/ceph-client.admin.45022.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45022.log.gz 2024-04-26T21:07:30.985 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47399.log 2024-04-26T21:07:30.986 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37920.log.gz 2024-04-26T21:07:30.987 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84367.log 2024-04-26T21:07:30.991 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84367.log: /var/log/ceph/ceph-client.admin.47399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46244.log 2024-04-26T21:07:30.993 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47399.log.gz 2024-04-26T21:07:30.993 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.84367.log.gz 2024-04-26T21:07:30.995 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46244.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46244.log.gz 2024-04-26T21:07:30.996 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84157.log 2024-04-26T21:07:30.998 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41457.log 2024-04-26T21:07:31.000 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.84157.log.gz 2024-04-26T21:07:31.001 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38130.log 2024-04-26T21:07:31.006 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41457.log: /var/log/ceph/ceph-client.admin.38130.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44850.log 2024-04-26T21:07:31.006 INFO:teuthology.orchestra.run.smithi103.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-26T21:07:31.007 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41457.log.gz 2024-04-26T21:07:31.007 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38130.log.gz 2024-04-26T21:07:31.009 INFO:teuthology.orchestra.run.smithi103.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-26T21:07:31.009 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39432.log 2024-04-26T21:07:31.014 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44850.log: /var/log/ceph/ceph-client.admin.39432.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37572.log 2024-04-26T21:07:31.014 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44850.log.gz 2024-04-26T21:07:31.015 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39432.log.gz 2024-04-26T21:07:31.017 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42696.log 2024-04-26T21:07:31.022 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37572.log: /var/log/ceph/ceph-client.admin.42696.log: 89.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46800.log 2024-04-26T21:07:31.022 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37572.log.gz 2024-04-26T21:07:31.023 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42696.log.gz 2024-04-26T21:07:31.025 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39516.log 2024-04-26T21:07:31.029 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46800.log: /var/log/ceph/ceph-client.admin.39516.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44724.log 2024-04-26T21:07:31.030 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46800.log.gz 2024-04-26T21:07:31.031 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39516.log.gz 2024-04-26T21:07:31.033 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44724.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44724.log.gz 2024-04-26T21:07:31.034 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37746.log 2024-04-26T21:07:31.036 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83113.log 2024-04-26T21:07:31.038 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37746.log.gz 2024-04-26T21:07:31.039 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82442.log 2024-04-26T21:07:31.044 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83113.log: /var/log/ceph/ceph-client.admin.82442.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46926.log 2024-04-26T21:07:31.045 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83113.log.gz 2024-04-26T21:07:31.046 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82442.log.gz 2024-04-26T21:07:31.047 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-26T21:07:31.052 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46926.log: /var/log/ceph/ceph-mon.a.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46926.log.gz 2024-04-26T21:07:31.052 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43266.log 2024-04-26T21:07:31.054 INFO:teuthology.orchestra.run.smithi028.stderr: 91.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38172.log 2024-04-26T21:07:31.054 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-26T21:07:31.055 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43266.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43266.log.gz 2024-04-26T21:07:31.058 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38814.log 2024-04-26T21:07:31.061 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38814.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38172.log.gz 2024-04-26T21:07:31.061 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46842.log 2024-04-26T21:07:31.063 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38814.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47729.log 2024-04-26T21:07:31.067 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46842.log: /var/log/ceph/ceph-client.admin.47729.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47729.log.gz 2024-04-26T21:07:31.067 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-04-26T21:07:31.067 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37289.log 2024-04-26T21:07:31.068 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46842.log.gz 2024-04-26T21:07:31.071 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47773.log 2024-04-26T21:07:31.073 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47773.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37289.log.gz 2024-04-26T21:07:31.074 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46548.log 2024-04-26T21:07:31.076 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39348.log 2024-04-26T21:07:31.078 INFO:teuthology.orchestra.run.smithi028.stderr: 88.1%/var/log/ceph/ceph-client.admin.46548.log: -- replaced with /var/log/ceph/ceph-client.admin.47773.log.gz 2024-04-26T21:07:31.080 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39348.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46548.log.gz 2024-04-26T21:07:31.080 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38802.log 2024-04-26T21:07:31.084 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38802.log: 88.8%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.39348.log.gz --verbose -- /var/log/ceph/ceph-client.admin.84241.log 2024-04-26T21:07:31.084 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-04-26T21:07:31.085 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38802.log.gz 2024-04-26T21:07:31.087 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84199.log 2024-04-26T21:07:31.090 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84199.log: /var/log/ceph/ceph-client.admin.84241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42014.log 2024-04-26T21:07:31.092 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.84241.log.gz 2024-04-26T21:07:31.093 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.84199.log.gz 2024-04-26T21:07:31.095 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45106.log 2024-04-26T21:07:31.098 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45106.log: /var/log/ceph/ceph-client.admin.42014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42198.log 2024-04-26T21:07:31.100 INFO:teuthology.orchestra.run.smithi028.stderr: 87.6% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42014.log.gz 2024-04-26T21:07:31.101 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45106.log.gz 2024-04-26T21:07:31.105 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47868.log 2024-04-26T21:07:31.105 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47868.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47868.log.gz 2024-04-26T21:07:31.105 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42324.log 2024-04-26T21:07:31.107 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42198.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42198.log.gz 2024-04-26T21:07:31.111 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42324.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42324.log.gz 2024-04-26T21:07:31.111 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41415.log 2024-04-26T21:07:31.112 INFO:teuthology.orchestra.run.smithi103.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-26T21:07:31.114 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83468.log 2024-04-26T21:07:31.114 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41415.log.gz 2024-04-26T21:07:31.116 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42896.log 2024-04-26T21:07:31.121 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83468.log: /var/log/ceph/ceph-client.admin.42896.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82641.log 2024-04-26T21:07:31.121 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83468.log.gz 2024-04-26T21:07:31.122 INFO:teuthology.orchestra.run.smithi028.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42896.log.gz 2024-04-26T21:07:31.123 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46370.log 2024-04-26T21:07:31.129 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82641.log: /var/log/ceph/ceph-client.admin.46370.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.2.40679.log 2024-04-26T21:07:31.129 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.82641.log.gz 2024-04-26T21:07:31.129 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46370.log.gz 2024-04-26T21:07:31.130 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42156.log 2024-04-26T21:07:31.132 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.2.40679.log: /var/log/ceph/ceph-client.admin.42156.log: 91.9% -- replaced with /var/log/ceph/ceph-client.2.40679.log.gz 2024-04-26T21:07:31.133 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84115.log 2024-04-26T21:07:31.135 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38464.log 2024-04-26T21:07:31.135 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42156.log.gz 2024-04-26T21:07:31.137 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84115.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.84115.log.gz 2024-04-26T21:07:31.139 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46968.log 2024-04-26T21:07:31.144 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38464.log: /var/log/ceph/ceph-client.admin.46968.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41037.log 2024-04-26T21:07:31.144 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38464.log.gz 2024-04-26T21:07:31.145 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46968.log.gz 2024-04-26T21:07:31.146 INFO:teuthology.orchestra.run.smithi103.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-26T21:07:31.146 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83342.log 2024-04-26T21:07:31.151 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41037.log: /var/log/ceph/ceph-client.admin.83342.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46632.log 2024-04-26T21:07:31.151 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41037.log.gz 2024-04-26T21:07:31.153 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83342.log.gz 2024-04-26T21:07:31.153 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46884.log 2024-04-26T21:07:31.154 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46632.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46632.log.gz 2024-04-26T21:07:31.156 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46286.log 2024-04-26T21:07:31.157 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46884.log.gz 2024-04-26T21:07:31.159 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83971.log 2024-04-26T21:07:31.163 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46286.log: /var/log/ceph/ceph-client.admin.83971.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44268.log 2024-04-26T21:07:31.164 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46286.log.gz 2024-04-26T21:07:31.165 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.83971.log.gz 2024-04-26T21:07:31.167 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48544.log 2024-04-26T21:07:31.171 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44268.log: /var/log/ceph/ceph-client.admin.48544.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37640.log 2024-04-26T21:07:31.171 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44268.log.gz 2024-04-26T21:07:31.173 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48544.log.gz 2024-04-26T21:07:31.175 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37640.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37640.log.gz 2024-04-26T21:07:31.176 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38298.log 2024-04-26T21:07:31.178 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82987.log 2024-04-26T21:07:31.179 INFO:teuthology.orchestra.run.smithi028.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38298.log.gz 2024-04-26T21:07:31.181 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82557.log 2024-04-26T21:07:31.185 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82987.log: /var/log/ceph/ceph-client.admin.82557.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44142.log 2024-04-26T21:07:31.186 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.82987.log.gz 2024-04-26T21:07:31.187 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.82557.log.gz 2024-04-26T21:07:31.187 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42486.log 2024-04-26T21:07:31.190 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42486.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42486.log.gz 2024-04-26T21:07:31.192 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44142.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44142.log.gz 2024-04-26T21:07:31.193 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43080.log 2024-04-26T21:07:31.196 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83239.log 2024-04-26T21:07:31.198 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43080.log.gz 2024-04-26T21:07:31.198 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46149.log 2024-04-26T21:07:31.199 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83239.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83239.log.gz 2024-04-26T21:07:31.200 INFO:teuthology.orchestra.run.smithi103.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-26T21:07:31.201 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48310.log 2024-04-26T21:07:31.203 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40181.log 2024-04-26T21:07:31.203 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46149.log.gz 2024-04-26T21:07:31.204 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43680.log 2024-04-26T21:07:31.208 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48310.log: /var/log/ceph/ceph-client.0.40181.log: /var/log/ceph/ceph-client.admin.43680.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48310.log.gz 2024-04-26T21:07:31.208 INFO:teuthology.orchestra.run.smithi028.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43680.log.gz 2024-04-26T21:07:31.209 INFO:teuthology.orchestra.run.smithi028.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.40181.log.gz 2024-04-26T21:07:31.209 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43386.log 2024-04-26T21:07:31.211 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43764.log 2024-04-26T21:07:31.213 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38004.log 2024-04-26T21:07:31.215 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43764.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43386.log.gz 2024-04-26T21:07:31.215 INFO:teuthology.orchestra.run.smithi103.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-26T21:07:31.216 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45804.log 2024-04-26T21:07:31.219 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37529.log 2024-04-26T21:07:31.219 INFO:teuthology.orchestra.run.smithi028.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43764.log.gz 2024-04-26T21:07:31.219 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-04-26T21:07:31.220 INFO:teuthology.orchestra.run.smithi103.stderr:real 0m0.506s 2024-04-26T21:07:31.220 INFO:teuthology.orchestra.run.smithi103.stderr:user 0m2.077s 2024-04-26T21:07:31.220 INFO:teuthology.orchestra.run.smithi103.stderr:sys 0m0.285s 2024-04-26T21:07:31.221 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38004.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38004.log.gz 2024-04-26T21:07:31.222 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.0.45804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37836.log 2024-04-26T21:07:31.227 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.37529.log: 88.9%/var/log/ceph/ceph-client.admin.37836.log: -- replaced with /var/log/ceph/ceph-client.admin.37529.log.gz 2024-04-26T21:07:31.227 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46758.log 2024-04-26T21:07:31.230 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37836.log.gz 2024-04-26T21:07:31.232 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46758.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46758.log.gz 2024-04-26T21:07:31.232 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48268.log 2024-04-26T21:07:31.235 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83665.log 2024-04-26T21:07:31.237 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.48268.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48268.log.gz 2024-04-26T21:07:31.238 INFO:teuthology.orchestra.run.smithi028.stderr: 93.0% -- replaced with /var/log/ceph/ceph-client.0.45804.log.gz 2024-04-26T21:07:31.239 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47687.log 2024-04-26T21:07:31.242 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47687.log: gzip/var/log/ceph/ceph-client.admin.83665.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46590.log 2024-04-26T21:07:31.243 INFO:teuthology.orchestra.run.smithi028.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.83665.log.gz 2024-04-26T21:07:31.243 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47687.log.gz 2024-04-26T21:07:31.245 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44184.log 2024-04-26T21:07:31.249 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46590.log: /var/log/ceph/ceph-client.admin.44184.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33734.log 2024-04-26T21:07:31.250 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46590.log.gz 2024-04-26T21:07:31.251 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44100.log 2024-04-26T21:07:31.252 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.33734.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44184.log.gz 2024-04-26T21:07:31.255 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44640.log 2024-04-26T21:07:31.258 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33734.log.gz 2024-04-26T21:07:31.258 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44640.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44640.log.gz 2024-04-26T21:07:31.258 INFO:teuthology.orchestra.run.smithi028.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44100.log.gz 2024-04-26T21:07:31.259 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83845.log 2024-04-26T21:07:31.262 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45148.log 2024-04-26T21:07:31.264 INFO:teuthology.orchestra.run.smithi028.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.83845.log.gz 2024-04-26T21:07:31.265 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44892.log 2024-04-26T21:07:31.270 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45148.log: /var/log/ceph/ceph-client.admin.44892.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39180.log 2024-04-26T21:07:31.271 INFO:teuthology.orchestra.run.smithi028.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45148.log.gz 2024-04-26T21:07:31.272 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44892.log.gz 2024-04-26T21:07:31.272 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43638.log 2024-04-26T21:07:31.275 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44598.log 2024-04-26T21:07:31.276 INFO:teuthology.orchestra.run.smithi028.stderr: 89.5%/var/log/ceph/ceph-client.admin.44598.log: -- replaced with /var/log/ceph/ceph-client.admin.39180.log.gz 2024-04-26T21:07:31.277 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43638.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44598.log.gz 2024-04-26T21:07:31.278 INFO:teuthology.orchestra.run.smithi028.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43638.log.gz 2024-04-26T21:07:31.279 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42444.log 2024-04-26T21:07:31.281 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42612.log 2024-04-26T21:07:31.282 INFO:teuthology.orchestra.run.smithi028.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42444.log.gz 2024-04-26T21:07:31.284 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42282.log 2024-04-26T21:07:31.289 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42612.log: /var/log/ceph/ceph-client.admin.42282.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82599.log 2024-04-26T21:07:31.289 INFO:teuthology.orchestra.run.smithi028.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42612.log.gz 2024-04-26T21:07:31.290 INFO:teuthology.orchestra.run.smithi028.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42282.log.gz 2024-04-26T21:07:31.291 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83197.log 2024-04-26T21:07:31.292 INFO:teuthology.orchestra.run.smithi028.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-26T21:07:31.294 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82599.log: 88.5%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.82599.log.gz --verbose -- /var/log/ceph/ceph-client.admin.43890.log 2024-04-26T21:07:31.294 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-04-26T21:07:31.295 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83197.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.83197.log.gz 2024-04-26T21:07:31.296 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84493.log 2024-04-26T21:07:31.297 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43890.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43890.log.gz 2024-04-26T21:07:31.298 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84493.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84493.log.gz 2024-04-26T21:07:31.299 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39600.log 2024-04-26T21:07:31.301 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45064.log 2024-04-26T21:07:31.302 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39600.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39600.log.gz 2024-04-26T21:07:31.304 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45064.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45064.log.gz 2024-04-26T21:07:31.304 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45526.log 2024-04-26T21:07:31.306 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44682.log 2024-04-26T21:07:31.307 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45526.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45526.log.gz 2024-04-26T21:07:31.308 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44682.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44682.log.gz 2024-04-26T21:07:31.309 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39558.log 2024-04-26T21:07:31.310 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39744.log 2024-04-26T21:07:31.312 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39558.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39558.log.gz 2024-04-26T21:07:31.313 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39744.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39744.log.gz 2024-04-26T21:07:31.314 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45316.log 2024-04-26T21:07:31.315 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-26T21:07:31.317 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45316.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45316.log.gz 2024-04-26T21:07:31.319 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83887.log 2024-04-26T21:07:31.322 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41793.log 2024-04-26T21:07:31.323 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83887.log: 93.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-26T21:07:31.323 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.83887.log.gz 2024-04-26T21:07:31.324 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43022.log 2024-04-26T21:07:31.325 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40843.log 2024-04-26T21:07:31.326 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41793.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41793.log.gz 2024-04-26T21:07:31.327 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84325.log 2024-04-26T21:07:31.328 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.40843.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43022.log.gz 2024-04-26T21:07:31.328 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40843.log.gz 2024-04-26T21:07:31.330 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39306.log 2024-04-26T21:07:31.332 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.84325.log: 87.9%/var/log/ceph/ceph-client.admin.39306.log: -- replaced with /var/log/ceph/ceph-client.admin.84325.log.gz 2024-04-26T21:07:31.333 INFO:teuthology.orchestra.run.smithi028.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39306.log.gz 2024-04-26T21:07:31.333 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42528.log 2024-04-26T21:07:31.335 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38214.log 2024-04-26T21:07:31.336 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42528.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42528.log.gz 2024-04-26T21:07:31.337 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38214.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38214.log.gz 2024-04-26T21:07:31.338 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44310.log 2024-04-26T21:07:31.339 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43806.log 2024-04-26T21:07:31.341 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.44310.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44310.log.gz 2024-04-26T21:07:31.342 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42938.log/var/log/ceph/ceph-client.admin.43806.log: 2024-04-26T21:07:31.342 INFO:teuthology.orchestra.run.smithi028.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43806.log.gz 2024-04-26T21:07:31.343 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45232.log 2024-04-26T21:07:31.345 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.42938.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42938.log.gz 2024-04-26T21:07:31.346 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43122.log/var/log/ceph/ceph-client.admin.45232.log: 2024-04-26T21:07:31.347 INFO:teuthology.orchestra.run.smithi028.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45232.log.gz 2024-04-26T21:07:31.348 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83929.log 2024-04-26T21:07:31.348 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.43122.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43122.log.gz 2024-04-26T21:07:31.350 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82903.log 2024-04-26T21:07:31.351 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.83929.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.83929.log.gz 2024-04-26T21:07:31.352 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39474.log 2024-04-26T21:07:31.353 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.82903.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.82903.log.gz 2024-04-26T21:07:31.354 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38473.log 2024-04-26T21:07:31.355 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.39474.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39474.log.gz 2024-04-26T21:07:31.357 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46674.log 2024-04-26T21:07:31.358 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.38473.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38473.log.gz 2024-04-26T21:07:31.358 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48184.log 2024-04-26T21:07:31.363 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46674.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46674.log.gz 2024-04-26T21:07:31.363 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45274.log/var/log/ceph/ceph-client.admin.48184.log: 2024-04-26T21:07:31.363 INFO:teuthology.orchestra.run.smithi028.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48184.log.gz 2024-04-26T21:07:31.363 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47910.log 2024-04-26T21:07:31.363 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.45274.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45274.log.gz 2024-04-26T21:07:31.365 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47817.log 2024-04-26T21:07:31.365 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47910.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47910.log.gz 2024-04-26T21:07:31.365 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44016.log 2024-04-26T21:07:31.367 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.47817.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47817.log.gz 2024-04-26T21:07:31.367 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.44016.log: /var/log/ceph/ceph-client.admin.41583.log 2024-04-26T21:07:31.368 INFO:teuthology.orchestra.run.smithi028.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44016.log.gz 2024-04-26T21:07:31.369 INFO:teuthology.orchestra.run.smithi028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46412.log 2024-04-26T21:07:31.370 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.41583.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41583.log.gz 2024-04-26T21:07:31.371 INFO:teuthology.orchestra.run.smithi028.stderr:/var/log/ceph/ceph-client.admin.46412.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46412.log.gz 2024-04-26T21:07:31.400 INFO:teuthology.orchestra.run.smithi028.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-26T21:07:31.524 INFO:teuthology.orchestra.run.smithi028.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-26T21:07:31.552 INFO:teuthology.orchestra.run.smithi028.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-26T21:07:31.625 INFO:teuthology.orchestra.run.smithi028.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-26T21:07:32.946 INFO:teuthology.orchestra.run.smithi028.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.0.48828.log.gz 2024-04-26T21:07:33.697 INFO:teuthology.orchestra.run.smithi028.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-26T21:07:33.703 INFO:teuthology.orchestra.run.smithi028.stderr: 2024-04-26T21:07:33.703 INFO:teuthology.orchestra.run.smithi028.stderr:real 0m2.988s 2024-04-26T21:07:33.703 INFO:teuthology.orchestra.run.smithi028.stderr:user 0m7.370s 2024-04-26T21:07:33.703 INFO:teuthology.orchestra.run.smithi028.stderr:sys 0m1.413s 2024-04-26T21:07:33.704 INFO:tasks.ceph:Archiving logs... 2024-04-26T21:07:33.704 DEBUG:teuthology.misc:Transferring archived files from smithi028:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675182/remote/smithi028/log 2024-04-26T21:07:33.705 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T21:07:34.905 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675182/remote/smithi103/log 2024-04-26T21:07:34.906 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T21:07:35.284 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T21:07:35.295 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T21:07:35.295 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T21:07:35.303 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T21:07:35.446 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T21:07:35.456 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T21:07:35.457 DEBUG:teuthology.orchestra.run.smithi028:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:07:35.465 DEBUG:teuthology.orchestra.run.smithi103:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T21:07:35.514 INFO:teuthology.orchestra.run.smithi028.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:07:35.521 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-04-26T21:07:35.525 INFO:teuthology.orchestra.run.smithi028.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:07:35.525 INFO:teuthology.orchestra.run.smithi028.stdout:=============================================================================== 2024-04-26T21:07:35.525 INFO:teuthology.orchestra.run.smithi028.stdout:^* hv01.front.sepia.ceph.com 3 6 377 54 +1076us[+1077us] +/- 47ms 2024-04-26T21:07:35.525 INFO:teuthology.orchestra.run.smithi028.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 56 -1186us[-1185us] +/- 76ms 2024-04-26T21:07:35.525 INFO:teuthology.orchestra.run.smithi028.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 -519us[ -519us] +/- 88ms 2024-04-26T21:07:35.525 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:07:35.530 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T21:07:35.530 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-04-26T21:07:35.530 INFO:teuthology.orchestra.run.smithi103.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 +1028us[+1028us] +/- 47ms 2024-04-26T21:07:35.530 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 53 -1211us[-1211us] +/- 76ms 2024-04-26T21:07:35.530 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -539us[ -539us] +/- 88ms 2024-04-26T21:07:35.531 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T21:07:35.532 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T21:07:35.542 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T21:07:35.542 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T21:07:35.552 DEBUG:teuthology.orchestra.run.smithi028:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T21:07:35.753 DEBUG:teuthology.orchestra.run.smithi103:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T21:07:35.921 DEBUG:teuthology.orchestra.run.smithi028:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:07:36.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:07:36.022 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T21:07:36.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:07:36.111 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T21:07:36.121 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T21:07:36.162 INFO:teuthology.task.internal:Duration was 1151.713349 seconds 2024-04-26T21:07:36.163 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T21:07:36.173 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T21:07:36.174 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T21:07:36.182 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T21:07:36.315 INFO:teuthology.orchestra.run.smithi028.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:07:36.324 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T21:07:36.685 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T21:07:36.685 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi028.front.sepia.ceph.com 2024-04-26T21:07:36.685 DEBUG:teuthology.orchestra.run.smithi028:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T21:07:36.751 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi103.front.sepia.ceph.com 2024-04-26T21:07:36.752 DEBUG:teuthology.orchestra.run.smithi103:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T21:07:36.826 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T21:07:36.827 DEBUG:teuthology.orchestra.run.smithi028:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T21:07:36.834 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T21:07:36.923 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T21:07:36.924 DEBUG:teuthology.orchestra.run.smithi028:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T21:07:36.930 DEBUG:teuthology.orchestra.run.smithi103:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T21:07:37.107 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T21:07:37.118 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T21:07:37.118 DEBUG:teuthology.orchestra.run.smithi028:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T21:07:37.208 DEBUG:teuthology.orchestra.run.smithi103:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T21:07:37.298 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T21:07:37.309 DEBUG:teuthology.orchestra.run.smithi028:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:07:37.318 DEBUG:teuthology.orchestra.run.smithi103:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:07:37.412 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern = core 2024-04-26T21:07:37.418 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = core 2024-04-26T21:07:37.461 DEBUG:teuthology.orchestra.run.smithi028:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:07:37.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:07:37.513 DEBUG:teuthology.orchestra.run.smithi103:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T21:07:37.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T21:07:37.567 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T21:07:37.577 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T21:07:37.577 DEBUG:teuthology.misc:Transferring archived files from smithi028:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675182/remote/smithi028 2024-04-26T21:07:37.578 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T21:07:37.679 DEBUG:teuthology.misc:Transferring archived files from smithi103:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675182/remote/smithi103 2024-04-26T21:07:37.679 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T21:07:37.775 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T21:07:37.776 DEBUG:teuthology.orchestra.run.smithi028:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T21:07:37.784 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T21:07:37.838 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T21:07:37.851 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T21:07:37.851 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T21:07:37.865 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T21:07:37.865 DEBUG:teuthology.orchestra.run.smithi028:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T21:07:37.873 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T21:07:37.925 INFO:teuthology.orchestra.run.smithi028.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 21:07 /home/ubuntu/cephtest 2024-04-26T21:07:37.929 INFO:teuthology.orchestra.run.smithi103.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 21:07 /home/ubuntu/cephtest 2024-04-26T21:07:37.933 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T21:07:37.946 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T21:07:37.975 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/asok_dump_tree} duration: 1151.7133486270905 flavor: default mon.a-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.b-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 owner: scheduled_rishabh@teuthology success: true 2024-04-26T21:07:37.975 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T21:07:38.052 INFO:teuthology.run:pass